Diapositive 1 - École Polytechnique Fédérale de Lausanne

Week 14 Dynamics and Plasticity 14.1 Reservoir computing - Complex brain dynamics - Computing with rich dynamics Biological Modeling of Neural Networks: 14.2 Random Networks - stationary state - chaos

14.3 Stability optimized circuits Week 14 Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne, Switzerland - application to motor data 14.4. Synaptic plasticity - Hebbian - Reward-modulated 14.5. Helping Humans - oscillations

- deep brain stimulation Week 14-part 1: Review: The brain is complex Neuronal Dynamics Brain dynamics is complex 1mm 10 000 neurons 3 km wire motor cortex frontal

cortex to motor output Week 14-part 1: Review: The brain is complex Neuronal Dynamics Brain dynamics is complex -Complex internal dynamics -Memory -Response to inputs -Decision making -Non-stationary

-Movement planning -More than one activity value motor cortex frontal cortex to motor output Week 14-part 1: Reservoir computing

Liquid Computing/Reservoir Computing: exploit rich brain dynamics Maass et al. 2002, Jaeger and Haas, 2004 Review: Maass and Buonomano, Stream of sensory inputs Readout 1 Readout 2

Week 14-part 1: Reservoir computing 3 4 - if-condition on 1 -calculcate See Maass et al. 2007 Rich neuronal dynamics Week 14-part 1: Rich dynamics Experiments of

Churchland et al. 2010 Churchland et al. 2012 See also: Shenoy et al. 2011 Modeling Hennequin et al. 2014, See also: Maass et al. 2002, Sussillo and Abbott, 2009 Laje and Buonomano, 2012 Shenoy et al., 2011

Week 14-part 1: Rich neuronal dynamics: a wish list -Long transients -Reliable (non-chaotic) -Rich dynamics (non-trivial) -Compatible with neural data (excitation/inhibition) -Plausible plasticity rules Week 14 Dynamics and Plasticity 14.1 Reservoir computing - Complex brain dynamics - Computing with rich dynamics

Biological Modeling of Neural Networks: 14.2 Random Networks - rate model - stationary state and chaos 14.3 Hebbian Plasticity Week 14 Dynamics and Plasticity - excitatory synapses - inhibitory synapses

14.4. Reward-modulated plasticity Wulfram Gerstner EPFL, Lausanne, Switzerland - free solution 14.5. Helping Humans - oscillations - deep brain stimulation Week 14-part 2: Review: microscopic vs. macroscopic An (t )

I(t) Week 14-part 2: Review: Random coupling excitation Homogeneous network: - each neuron receives input from k neurons in network -each neuron receives the same (mean) external input

inhibition Week 14-part 2: Review: integrate-and-fire/stochastic spike arrival Stochastic spike arrival: excitation, total rate Re inhibition, total rate Ri Synaptic current pulses d u (u ueq) R dt

u u0 d u ( u u eq ) dt qe (t

f tk k, f qi (t k ', f '

EPSC RI ) f' tk' ) mean IPSC (t ) (t )

Firing times: Threshold crossing Langevin equation, Ornstein Uhlenbeck process Fokker-Planck equation Week 14-part 2: Dynamics in Rate Networks F-I curve of rate neuron d ri ri F ( w ij r j )

dt j Fixed point with F(0)=0 ri 0 Slope 1 Suppose: d 1 F '( 0 ) F (x 0) dx

stable unstable Suppose 1 dimension d x x F (w x) dt Exercise 1: Stability of fixed point d x x F (w x)

dt Fixed point with F(0)=0 x 0 Next lecture: 9h43 Suppose: d 1 F '( 0 ) F (x 0) dx

Suppose 1 dimension Calculate stability, take w as parameter d x x F (w x) dt Week 14-part 2: Dynamics in Rate Networks Blackboard: d ri ri F ( w ij r j ) Two dimensions!

dt j Fixed point with F(0)=0 ri 0 Suppose: d 1 F '( 0 ) F (x 0) dx Suppose 1 dimension

stable w<1 unstable w>1 d x x F (w x) dt Week 14-part 2: Dynamics in RANDOM Rate Networks d

ri ri F ( w ij r j ) dt j Fixed point: ri 0 stable R e( ) 1 unstable R e( ) 1

Random, 10 percent connectivity d 1 F '( 0 ) F (x 0) dx Chaotic dynamics: Sompolinksy et al. 1988 (and many others: Amari, ... Unstable dynamics and Chaos

d ri ri F ( w ij r j ) i ( t ) dt j chaos Rajan and Abbott, 2006 Image: Ostojic, Nat.Neurosci, 2014 Image: Hennequin et al. Neuron, 2014 Week 14-part 2: Dynamics in Random SPIKING

Networks d f u i u i w ij ( t t j ) dt j f Firing times: Threshold crossing Image: Ostojic, Nat.Neurosci, 2014 Week 14-part 2: Stationary activity: two different

regimes Stable rate fixed point, microscopic chaos Switching/bursts long autocorrelations: Rate chaos R e( ) 1 Ostojic, Nat.Neurosci, 2014

Week 14-part 2: Rich neuronal dynamics: a wish list -Long transients -Reliable (non-chaotic) -Rich dynamics (non-trivial) -Compatible with neural data (excitation/inhibition) -Plausible plasticity rules Week 14 Dynamics and Plasticity 14.1 Reservoir computing - Complex brain dynamics - Computing with rich dynamics

Biological Modeling of Neural Networks: 14.2 Random Networks - stationary state - chaos 14.3 Stability optimized circuits Week 14 Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne, Switzerland - application to motor data

14.4. Synaptic plasticity - Hebbian - Reward-modulated 14.5. Helping Humans - oscillations - deep brain stimulation Week 14-part 3: Plasticity-optimized circuit Re( ) 1

R e( ) 1 Image: Hennequin et al. Neuron, 2014 Optimal control of transient dynamics in balanced networks supports generation of complex movements Hennequin et al. 2014, Random stability-optimized circuit (SOC) Random

connectivity Week 14-part 3: Random Plasticity-optimized circuit Random Week 14-part 3: Random Plasticity-optimized circuit study duration of transients a1= slowest = most amplified

slope 1/linear theory slope 1/linear theory F-I curve of rate neuron Week 14-part 3: Random Plasticity-optimized circuit slope 1/linear theory a1= slowest = most amplified

F-I curve of rate neuron Optimal control of transient dynamics in balanced networks supports generation of complex movements Hennequin et al. NEURON 2014, Week 14-part 3: Application to motor cortex: data and model Churchland et al. 2010/2012

Hennequin et al. 2014 Quiz: experiments of Churchland et al. [ ] Before the monkey moves his arm, neurons in motor-related areas exhibit activity [ ] While the monkey moves his arm, different neurons in motorrelated area show the same activity patterns [ ] while the monkey moves his arm, he receives information which of the N targets he has to choose [ ] The temporal spike pattern of a given neuron is nearly the same, between one trial and the next (time scale of a few milliseconds)

[ ] The rate activity pattern of a given neuron is nearly the same, between one trial and the next (time scale of a hundred milliseconds) Week 14-part 3: Random Plasticity-optimized circuit Comparison: weak random Hennequin et al. 2014 Week 14-part 3: Stability optimized SPIKING network Classic sparse random connectivity (Brunel 2000) Random connections, fast

Stabilizy-optimized random connections distal connections, slow, (Branco&Hausser, 2011) structured Fast AMPA slow NMDA Overall: 20% connectivity 12000 excitatory LIF = 200 pools of 60 neurons 3000 inhibitory LIF = 200 pools of 15 neurons

Week 14-part 3: Stability optimized SPIKING network Spontaneous Classic sparse random connectivity (Brunel 2000) firing rate Neuron 1 Neuron 2 Neuron 3 Single neuron different initial conditions Hennequin et al. 2014

Week 14-part 3: Stability optimized SPIKING network Classic sparse random connectivity (Brunel 2000) Hennequin et al. 2014 Week 14-part 3: Rich neuronal dynamics: a result list -Long transients -Reliable (non-chaotic) -Rich dynamics (non-trivial) -Compatible with neural data (excitation/inhibition) -Plausible plasticity rules

Week 14 Dynamics and Plasticity 14.1 Reservoir computing - complex brain dynamics - Computing with rich dynamics Biological Modeling of Neural Networks: 14.2 Random Networks - stationary state - chaos

14.3 Stability optimized circuits Week 14 Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne, Switzerland - application to motor data 14.4. Synaptic plasticity - Hebbian - Reward-modulated 14.5. Helping Humans

- oscillations - deep brain stimulation Hebbian Learning = all inputs/all times are equal pre j post i w ij w ij F ( pre , post )

Week 14-part 4: STDP = spike-based Hebbian learning Pre-before post: potentiation of synapse Pre-after-post: depression of synapse Modulation of Learning = Hebb+ confirmation confirmation

Functional Postulate Useful for learning the important stuff w ij F ( p r e , p o s t , C O N F IR M ) local global Many models (and experiments) of synaptic plasticity do not take into account Neuromodulators. Except: e.g. Schultz et al. 1997, Wickens 2002, Izhikevich, 2007; Reymann+Frey 2007; Moncada 2007, Pawlak+Kerr 2008; Pawlak et al. 2010 Consolidation of Learning Success/reward

Confirmation -Novel -Interesting -Rewarding -Surprising Neuromodulators dopmaine/serotonin/Ach write now to long-term memory Crow (1968),

Fregnac et al (2010), Izhikevich (2007) Plasticity Stability-optimized curcuits - here: algorithmically tuned BUT - replace by inhibitory plasticity Vogels et al., Science 2011

avoids chaotic blow-up of network avoids blow-up of single neuron (detailed balance) yields stability optimized circuits Plasticity Readout - here: algorithmically tuned BUT Izhikevich, 2007 Fremaux et al. Success signal 2012

- replace by 3-factor plasticity rules Week 14-part 4: Plasticity modulated by reward Dopamine-emitting neurons: Schultz et al., 1997 Izhikevich, 2007 Fremaux et al. Success signal 2012 Dopamine encodes success= reward expected reward

Week 14-part 4: Plasticity modulated by reward Week 14-part 4: STDP = spike-based Hebbian learning Pre-before post: potentiation of synapse Week 14-part 4: Plasticity modulated by reward STDP with pre-before post: potentiation of synapse Quiz: Synpatic plasticity: 2-factor and 3-factor rules [ ] a Hebbian learning rule depends only on presynaptic and postsynaptic variables, pre and post

[ ] a Hebbian learning rule can be written abstractly as w ij F ( p r e , p o s t , C O N F IR M ) [ ] STDP is an example of a Hebbian learning rule [ ] a 3-factor learning rule can be written as w ij F ( p r e , p o s t , g lo b a lfa c to r ) [ ] a reward-modulated learning rule can be written abstractly as w ij F ( p r e , p o s t , s u c c e s s ) [ ] a reward-modulated learning rule can be written abstractly as

w ij F ( p r e , p o s t , n e u r o M O D U L A T O R ) Week 14-part 4: from spikes to movement How can the readouts encode movement? Week 14-part 5: Population vector coding Population vector coding of movements Schwartz et al.

1988 Week 14-part 4: Learning movement trajectories Population vector coding of movements 70000 synapses 1 trial =1 second Output to trajectories via population vector coding Single reward at the END of each trial based on similarity with a target trajectory Fremaux et al., J. Neurosci. 2010

Week 14-part 4: Learning movement trajectories QuickTime and a decompressor are needed to see this picture. Performance Fremaux et al. J. Neurosci. 2010 R-STDP

LTPonly Week 14-part 4: readout Plasticity can tune the network and Hebbian STDP - inhibitory connections, tuned by 2-factor STDP, for stabilization Vogels et al. 2011

Reward-modulated Success signal STDP for movement learning - Readout connections, tuned by 3-factor plasticity rule Fremaux et al. 2012 Last Lecture TODAY Exam: - written exam 17. 06. 2014 from 16:15-19:00

- miniprojects counts 1/3 towards final grade For written exam: -bring 1 page A5 of own notes/summary -HANDWRITTEN! Nearly the end: what can I improve for the students next year? Integrated exercises? Quizzes? Miniproject? Overall workload ?(4 credit course = 6hrs per week) Background/Prerequisites?

-Physics students -SV students -Math students Slides? videos? Week 14 Dynamics and Plasticity 14.1 Reservoir computing - Review:Random Networks - Computing with rich dynamics

Biological Modeling of Neural Networks: 14.2 Random Networks - stationary state - chaos 14.3 Stability optimized circuits Week 14 Dynamics and Plasticity Wulfram Gerstner EPFL, Lausanne, Switzerland

- application to motor data 14.4. Synaptic plasticity - Hebbian - Reward-modulated 14.5. Helping Humans - oscillations - deep brain stimulation Week 14-part 5: Oscillations in the brain Individual neurons

fire regularly two groups alternate Week 14-part 5: Oscillations in the brain Individual neurons fire irregularly Week 14-part 5: STDP Week 14-part 5: STDP during oscillations Oscillation near-synchronous spike arrival

Week 14-part 5: Helping Humans big weights Pfister and Tass, 2010 small weights Week 14-part 5: Helping Humans: Deep brain stimulation Parkinsons disease: Symptom: Tremor -periodic shaking - 3-6 Hz

Brain activity: - thalamus, basal ganglia - synchronized 3-6Hz in Parkinson - Deep brain stimulation Benabid et al, 1991, 2009 Week 14-part 5: Helping Humans Abstract Dynamical Systems

View of Brain states healthy pathological Week 14-part 5: Helping Humans: coordinated reset Pathological, synchronous Coordinated reset stimulus Week 14-part 5: Helping Humans Tass et al. 2012

Theoretical concepts can help to understand brain dynamics contribute to understanding plasticity and learning inspire medical approaches can help humans The End Plasticity and Neuronal Dynamics now QUESTION SESSION! Questions to Assistants possible until June 1

The end and good luck for the exam! Week 14-part 3: Correlations in Plasticity-optimized circuit Hennequin et al. 2014,

Recently Viewed Presentations

  • Plenary Speakers: Federal Panel Libby Doggett Early Learning

    Plenary Speakers: Federal Panel Libby Doggett Early Learning

    THANK YOU. [email protected] Libby: So it looks like we have lots of work ahead. First we need more funding. And then we all need to work to make sure good laws get implemented in the right way.
  • What would happen if there were no border between the United ...

    What would happen if there were no border between the United ...

    Last Day of Egg Experiment& Diffusion Lab - Discard egg into beaker on station 8. NOT THE SINK!!!!! - Throw away empty plastic baggie. - Wash cups out with soap and water.
  • Family Advisory Comittee - WCAAP.org

    Family Advisory Comittee - WCAAP.org

    Skagit Peds creates its firstParent Advisory Group. First created in 2003 by Dr. Andrea Lowe, clinic pediatrician who was Skagit County Representative of Washington State Medical Home Network at the time.
  • Présentation PowerPoint

    Présentation PowerPoint

    Research Institute Foundation Etat Others 642.0 1133.0 370.0 58.0 55.0 99.0 Number of projects ETHZ - Eidgenössische Technische Hochschule Zürich CHUV - Centre hospitalier universitaire vaudois Université de Genève Université de Lausanne CNRS - Centre National de la Recherche Scientifique...
  • June 2010 - IEEE Standards Association

    June 2010 - IEEE Standards Association

    Abstract: Information on 802.11 technology for inclusion in the June 2010 NIST PAP#2 Report email Phone Address Company Name [email protected] +1-321-751-3988 5488 Marvell Lane, Santa Clara, CA, 95054 Marvell Bruce Kraemer Kaberi Banerjee
  • Safety design parameters for Li-ion propulsion batteries and

    Safety design parameters for Li-ion propulsion batteries and

    Several cathodes are now in use in large-format batteries. Spinel (LiMn. 2 O 4) NMC (LiNiMnCoO. 2)NCA (LiNiCoAlO. 2)LFP (LiFePO. 4).For design purposes, balance is sought among safety, power, energy, life and cost.
  • #ISUCIT Welcome to CIT 2016 Session: Personally Identifiable

    #ISUCIT Welcome to CIT 2016 Session: Personally Identifiable

    An Important Point "A common misconception is that PII only includes data that can be used to directly identify or contact an individual (e.g., name, e-mail address), or personal data that is especially sensitive (e.g., Social Security number, bank account...
  • Chapter 8: Major Elements

    Chapter 8: Major Elements

    Rotoinversion e. 6-fold rotoinversion ( 6 ) Note: this is the same as a 3-fold rotation axis perpendicular to a mirror plane (combinations of elements follows) 3-D Symmetry New Symmetry Elements 4. Rotoinversion e. 6-fold rotoinversion ( 6 ) A...