USC Brain Project Specific Aims

USC Brain Project Specific Aims

Brain Theory and Artificial Intelligence Lecture 5: Introduction to Vision. Reading Assignments: None Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 1

Projection Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 2 Projection Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision

3 Convention: Visual Angle Rather than reporting two numbers (size of object and distance to observer), we will combine both into a single number: visual angle e.g., the moon: about 0.5deg visual angle your thumb nail at arms length: about 1.5deg visual angle

1deg visual angle: 0.3mm on retina Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 4 Optics limitations: acuity Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision

5 Eye Anatomy Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 6 Visual Pathways

Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 7 Image Formation Accomodation: ciliary muscles can adjust shape of lens, yielding an effect equivalent to an autofocus. Laurent Itti: CS564 Brain Theory and Artificial Intelligence -

Introduction to Vision 8 Phototransduction Cascade Net effect: light (photons) is transformed into electrical (ionic) current. Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 9

Rods and Cones Roughly speaking: 3 types of cones, sensitive to red, green and blue. Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 10 Processing layers in retina Laurent Itti: CS564 Brain Theory and Artificial Intelligence -

Introduction to Vision 11 Retinal Processing Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 12

Center-Surround Center-surround organization: neurons with receptive field at given location receive inhibition from neurons with receptive fields at neighboring locations (via inhibitory interneurons). Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 13 Early Processing in Retina

Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 14 Color Processing Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 15

Over-representation of the Fovea Fovea: central region of the retina (1-2deg diameter); has much higher density of receptors, and benefits from detailed cortical representation. Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 16 Fovea and Optic Nerve

Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 17 Blind Spot Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 18

Retinal Sampling Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 19 Retinal Sampling Laurent Itti: CS564 Brain Theory and Artificial Intelligence -

Introduction to Vision 20 Seeing the world through a retina Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 21 Sampling & Optics

Because of blurring by the optics, we cannot see infinitely small objects Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 22 Sampling & optics The sampling grid optimally corresponds to the amount of

blurring due to the optics! Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 23 from the eye to V1 Image is decomposed and analyzed in terms of: - orientation - spatial frequency - size - color

- direction of motion - binocular disparity Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 24 Visual Field Mapping Laurent Itti: CS564 Brain Theory and Artificial Intelligence -

Introduction to Vision 25 Retina to Lateral Geniculate Nucleus Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 26 Location of LGN in Brain

LGN = lateral geniculate nucleus of the thalamus. Thalamus = deep gray-matter nucleus; relay station for all senses except olfaction. Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 27 Lateral Geniculate Nucleus Receives input from both eyes, but these remain segregated (no binocular neurons).

LGN consists of 6 layers: - 4 parvocellular (P-pathway): small RFs, input from cones, sensitive to color, fine detail and slow motion - 2 magnocellular (M-pathway): large RFs, very sensitive to faster motion. Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 28 Origin of Center-Surround

Neurons at every location receive inhibition from neurons at neighboring locations. Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 29 LGN to V1 - V1 = primary visual cortex = striate cortex (in contrast to higher,

extrastriate areas). - V1 is the first region where neurons respond to a combination of inputs from both eyes. - Some neurons respond equally well to patterns presented on both eyes - Some

respond best to one eye Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 30 Calcarine sulcus Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision

31 Neuronal Tuning In addition to responding only to stimuli in a circumscribed region of the visual space, neurons typically only respond to some specific classes of stimuli (e.g., of given color, orientation, spatial frequency). Each neuron thus has a preferred stimulus, and a tuning curve that describes the decrease of its response to stimuli increasingly different from the preferred

stimulus. Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 32 Orientation Tuning in V1 First recorded by Hubel & Wiesel in 1958. Laurent Itti: CS564 Brain Theory and Artificial Intelligence -

Introduction to Vision 33 Origin of Orientation Selectivity Feedforward model of Hubel & Wiesel: V1 cells receive inputs from LGN cells arranged along a given orientation. Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision

34 Feedforward model Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 35 But the feedforward model has shortcomings E.g., does not explain independence of tuning with respect to contrast. Hence, another model includes recurrent feedback (intra-cortical)

connections which sharpen tuning and render it contrast-independent. Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 36 Excitatory vs. Inhibitory Input Activation of excitatory synapse increases activity of postsynaptic cell.

Activation of inhibitory synapse decreases activity of postsynaptic cell. Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 37 Tuning is General It is also found, for example, in somatosensory cortex. Somatosensory neurons also have a receptive field, a preferred stimulus, and a tuning

curve. Also note that these properties are highly adaptive and trainable. Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 38 More Complex Neuronal Tuning Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision

39 Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 40 Oriented RFs Gabor function: product of a grating and

a Gaussian. Feedforward model: equivalent to convolving input image by sets of Gabor filters. Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 41 Receptive fields Summary

Retina: center-surround, circular, monocular LGN: center-surround, circular, monocular V1: oriented (Gabor): respond best to bar stimuli sensitive to motion monocular or binocular Simple cells: respond best to bars of given orientation at given location within receptive field. Complex cells: less sensitive to stimulus position within RF, sensitive to stimulus motion. Hypercomplex cells: like complex, but with inhibitory region at one end.

Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 42 Cortical Hypercolumn A hypercolumn represents one visual location, but many visual attributes. Basic processing module in V1.

Blobs: discontinuities in the columnar structure. Patches of neurons concerned mainly with color vision. Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 43 Laurent Itti: CS564 Brain Theory and Artificial Intelligence -

Introduction to Vision 44 Cortical Magnification Much more neuronal hardware dedicated to the center of the field of view than to the periphery. 1000x more neurons in fovea than far periphery for same size input. Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision

45 Cortical Hierarchy Some highlights: - more feedback than feedforward - specialization by area - what/where

- interactions Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 46 Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 47

Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 48 Extrastriate Cortex Over 25 visually responsive areas outside of striate cortex Many

of visual areas have retinotopic maps Maps become less precise upstream from striate cortex Receptive fields increase upstream from striate cortex Many

of these areas contain neurons selective for various stimulus dimensions (orientation, direction of motion, disparity, color) Two streams of processing through visual cortex: motion and "where" (occipito-parietal, magnocellular) and color & form (occipito-temporal; parvocellular) pathway. Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision

49 Area V2 Located within the lunate sulcus; immediately adjacent to V1 Orderly retinotopic map Receptive

fields larger than those in V1 A pattern of "thick", "thin" and "interstripes" perpendicular to the cortical surface with inputs from specific regions in V1 (interblob ->interstripe; layer 4B-->thick; blobs-->thin). Cells selective for orientation, direction, disparity, color (similar to V1); responses to subjective contours. Laurent Itti: CS564 Brain Theory and Artificial Intelligence -

Introduction to Vision 50 Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 51 Laurent Itti: CS564 Brain Theory and Artificial Intelligence -

Introduction to Vision 52 Contour Perception and V2 Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 53 Area V3

Inputs from layer 4B (with magnocellular inputs) of V1. Retinotopic Responses map split into upper (VP) and lower field. to lower spatial and higher temporal frequencies than in V2. Receptive

fields larger than in V2; many selective for orientation, direction, disparity and color. Emergence of new properties: evidence for integration of complex motion ("pattern" motion; like MT). Possible site for interaction between color and motion. Laurent Itti: CS564 Brain Theory and Artificial Intelligence -

Introduction to Vision 54 Area V4 Inputs from V2 (thin stripes and interstripes) and V3. Projects to inferotemporal cortex ( IT). Orderly Cells

retinotopic map; larger receptive fields than in V2 and V3 selective for orientation and color; some directionally selective cells. Lesions result in deficits in some aspects of complex form and/or color perception and not in motion perception. Laurent Itti: CS564 Brain Theory and Artificial Intelligence -

Introduction to Vision 55 Area V5 (MT) Inputs from V1 (layer 4B) , V2 (thick stripes) and V3 Projections to MST and parietal cortex

Retinotopic map. Larger receptive fields selective for motion direction, disparity and stimulus orientation; no selectivity for color; responses to complex motion ("pattern" motion). Lesions: selectively affect direction and speed discrimination, as well

as motion integration. deficits more pronounced in the presence of motion noise. Partial or complete recovery with training. Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 56 Response to Motion Stimuli in MT Laurent Itti: CS564 Brain Theory and Artificial Intelligence -

Introduction to Vision 57 Area MST Inputs from MT and V3 Projections Large to parietal cortex

receptive fields that include the fovea; no retinotopy Cells respond well to large-field motion; selective for direction of complex motion (rotation, contraction, expansion, spiral); responses to optic flow. Likely involvement in the analysis of optic flow

Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 58 Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 59 Area IT

Inputs from V4 Large receptive fields include the fovea and covering most of the visual field Selectivity to length, size, shape, faces and textures High

selectivity for complex images (10% of cells selective to faces and hands). Evidence that stimulus selectivity can be acquired through learning Lesions in humans result in prosopagnosia (deficit in face recognition); lesions in monkeys result in deficits in learning of complex pattern discriminations.

Involvement in short-term memory (delay related activity) Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 60 Face Cells

Laurent Itti: CS564 Brain Theory and Artificial Intelligence - Introduction to Vision 61

Recently Viewed Presentations

  • O , my sustainer! Open my Heart and

    O , my sustainer! Open my Heart and

    Anteriorly divided by aponeurosis of levator palpabrae superioris into: Upper orbital part Lower palpabral part Ducts are 12 in number, pass through the palpabral part of the gland and open into the conjunctival sac, 4.5 mm above the upper border...
  • Intro to Musical Theatre - St Pius X School

    Intro to Musical Theatre - St Pius X School

    The Lion King plays in London's Lyceum Theatre in the West End. ... One of the oldest styles of theaters that was originally built outside, this theater is a half circle of seating raised up to look down on the...
  • Step 2. If you dont have an account

    Step 2. If you dont have an account

    Step 4. To access online training, click on this navigation link Step 5. Click on the "Available Training" navigation link Step 5. Find the course you are interested in, and then click on the navigation link If you complete training...
  • RTTT Narrative and Budget Update BRIGHT FUTURE FOR

    RTTT Narrative and Budget Update BRIGHT FUTURE FOR

    Expanded Learning Time and Model Turn-Around Grants: State supports for persistently lowest performing schools will include competitive funding and technical assistance for expanded learning time, using the NCTL model. Model turn-around grants will also be provided supporting district implementation of...
  • Energy Flow through Ecological Systems

    Energy Flow through Ecological Systems

    Implications We observe this dynamic balance of energy flow at all levels of scale from the ecological system to the individual organism. Also, this is the algebra with which we can understand the consequences of our actions. Energy Flow through...
  • Parent Meeting - Eltham Church of England

    Parent Meeting - Eltham Church of England

    Here at Eltham C of E we also use Tapestry to assess the children's learning. Under each observation, where appropriate, we will give you an indication of where your child is within there age range and for what area of...
  • Introducing Object-Oriented Programming (OOP)

    Introducing Object-Oriented Programming (OOP)

    Introducing Object-Oriented Programming (OOP) Goals By the end of this lecture, you should … Understand the three pillars of Object-Oriented Programming: Inheritance, Encapsulation and Polymorphism. Understand what an object is. Understand object attributes, methods and events. Understand how programmers use...
  • Who Took My Dirndl?

    Who Took My Dirndl?

    Who Took My Dirndl? A History of German Fashion By Erin "Hildegaard" Hanson Works Cited What People Wore When. New York, New York: The Ivy Press, 2008.