CS 182 Sections 103 & 104 - University of California, Berkeley

CS 182 Sections 103 & 104 - University of California, Berkeley

CS 182 Sections 101 - 102 Eva Mok ([email protected]) Feb 11, 2004 bad puns alert! (http://www2.hi.net/s4/strangebreed.htm) Announcements a3 part 1 is due tonight (submit as a3-1) The second tester file is up, so pls. start part 2. The quiz is graded (get it after class). Where we stand Last Week Backprop This Week Recruitment learning color Coming up Imagining techniques (e.g. fMRI) The Big (and complicated) Picture Psycholinguistics Experiments Spatial Relation Motor Control Metaphor Grammar

Cognition and Language abstraction Computation Chang Model Bailey Model Structured Connectionism Neural Net Regier & Learning Triangle Nodes Narayanan Model Model SHRUTI Computational Neurobiology Visual System Neural Development Quiz Biology Midterm Finals

Quiz 1. What is a localist representation? What is a distributed representation? Why are they both bad? 2. What is coarse-fine encoding? Where is it used in our brain? 3. What can Back-Propagation do that Hebbs Rule cant? 4. Derive the Back-Propagation Algorithm 5. What (intuitively) does the learning rate do? How about the momentum term? Distributed vs Localist Repn John 1 1 0 0 John 1 0 0 0 Paul 0

1 1 0 Paul 0 1 0 0 George 0 0 1 1 George 0 0 1 0 Ringo

1 0 0 1 Ringo 0 0 0 1 What are the drawbacks of each representation? Distributed vs Localist Repn John 1 1 0 0 John 1 0

0 0 Paul 0 1 1 0 Paul 0 1 0 0 George 0 0 1 1 George 0

0 1 0 Ringo 1 0 0 1 Ringo 0 0 0 1 What happens if you want to represent a group? What happens if one neuron dies? How many persons can you represent with n bits? 2^n How many persons can you represent with n bits? n

Visual System 1000 x 1000 visual map For each location, encode: orientation direction of motion speed size color depth Blows up combinatorically! Coarse Coding info you can encode with one fine resolution unit = info you can encode with a few coarse resolution units Now as long as we need fewer coarse units total, were good Coarse-Fine Coding Coarse in F2, Fine in F1 Feature 1 e.g. Orientation Y-Orientation Y G

X-Orientation G X Coarse in F1, Fine in F2 Y-Dir X-Dir but we can run into ghost images Feature 2 e.g. Direction of Motion Back-Propagation Algorithm yj wij xi f xi = j wij yj yi ti:target yi = f(xi) Sigmoid: 1

y i f ( xi ) 1 e xi We define the error term for a single node to be ti - yi Gradient Descent i2 i1 global mimimum: this is your goal it should be 4-D (3 weights) but you get the idea The output layer learning rate wjk k wij j yi ti: target E Wij Wij Wij E Wij Wij i

E = Error = i (ti yi)2 The derivative of the sigmoid is just E E y x i i ti yi f ' ( xi ) y j Wij yi xi Wij y i 1 y i Wij ti yi yi 1 yi y j Wij y j i i ti yi yi 1 yi The hidden layer wjk wij yi ti: target E W jk W jk E E y j x j W jk y j x j W jk k j

i E = Error = i (ti yi)2 E E yi xi (ti yi ) f ' ( xi ) Wij y j y x y i i i i j E W jk W jk ( t y )

y 1 y W i i i i i ij y j 1 y j yk W jk yk j j (ti yi ) yi 1 yi Wij y j 1 y j i j Wij i y j 1 y j i (t i i yi ) f ' ( xi ) Wij f ' ( x j ) yk Lets just do an example 0 i 1 0 i 2 b=1

w01 0.8 w02 0.6 w0b 0.5 x0 1/(1+e^-0.5) f y0 0.6224 i2 y0 0 0 0 0 1 1 1 0 1

1 1 1 E = Error = i (ti yi)2 E = (t0 y0)2 0.5 0.4268 E = (0 0.6224)2 = 0.1937 i ti yi yi 1 yi Wij y j i W01 y1 0 i1 i1 0 0 0 W02 y2 0 i2 0 W0b yb 0 b 0 0.1463 learning rate 0 t0 y0 y0 1 y0 0 0 0.6224 0.62241 0.6224 0 0.1463 suppose = 0.5 W0b 0.5 0.1463 0.0731

Recently Viewed Presentations

  • TGn Report to EC - IEEE Standards Association

    TGn Report to EC - IEEE Standards Association

    Conversion from MSWord to FrameMaker is non-trivial. While it is possible to use the clipboard to copy/paste blocks of text (using paste-special, and text-only mode), tables, formulas, and figures are especially troublesome. A table with 20 rows and 5 columns...
  • Superpowers - LT Scotland

    Superpowers - LT Scotland

    Happy Harry. He finds it hard to be calm and happy when things don't go his way. He is a bit like me because ….
  • VIR358 Hyper-V Architecture, Scenarios And Networking

    VIR358 Hyper-V Architecture, Scenarios And Networking

    VIR358 Hyper-V Architecture, Scenarios And Networking Subject: Tech Ed 2008 TechEd Tech-Ed Author: Jeff Woolsey Keywords: Technical, partners and customers, dev, developers, developer, IT IT Pro Pros Professionals, Description
  • Cheating Applies to Many Things 1 Introduction  Cheating

    Cheating Applies to Many Things 1 Introduction Cheating

    Synonyms -- deceiving, deception, defrauding, dishonesty (dictionary.com) ... The fear of the LORD is the beginning of knowledge: but fools despise wisdom and instruction. Proverbs 18:15 [15] The heart of the prudent getteth knowledge; and the ear of the wise...
  • Assessing scales of thermal influence in alluvial floodplains

    Assessing scales of thermal influence in alluvial floodplains

    Excellent opportunity for experimentation on a partially constrained, partially wild alluvial floodplain Presence of 4 species of Pacific Salmon Opportunity to change existing land use patterns Two decades of salmonid life stage monitoring Primary drivers of hyporheic exchange 1- Scale...
  • Artificial Photosynthesis - Mars Home

    Artificial Photosynthesis - Mars Home

    Artificial Photosynthesis a low power recycling life support system by Robert B. Dyck Ardeco Consulting Ltd. Purpose A closed system that includes human metabolism Existing systems Apollo and Shuttle carried bottled oxygen and lithium-hydroxide fine for a few days supplies...
  • Knaqsia dhe stresi n pun si dhe prkushtimi

    Knaqsia dhe stresi n pun si dhe prkushtimi

    Karasek et al. (1982) kishte studiuar më tepër se 1000 punëtorë mashkujë në Suedi dhe ishte treguar që përkrahja e tyre nga mbikqyrësit dhe kolegët zbusin efektin e kërkesave të punës dhe ndihmojnë në mbajtjen e kënaqësisë në punë.
  • Passage 1 An assistance dog is a dog

    Passage 1 An assistance dog is a dog

    The best ones don't just say what you like, they say why you like whatever it is that you like. If you compliment someone on a new dress, don't just say, "I like that dress." Instead, you could say, "I...