Music and Imagery Wodzisaw Duch Neurocognitive Laboratory, Center for Modern Interdisciplinary Technologies, & Department of Informatics, Nicolaus Copernicus University Google: W. Duch 2nd International Scientific Conference "Hearing Implants and Music", Kajetany, 14.07.2016 Questions
1. 2. 3. 4. 5. Do we all hear the same things when we hear music? Do we need ears to hear music? Is it enough to have ears to hear music? If I have a good musical ear can I imagine music?
Can musical imagery be controlled? Even more questions in the end Do we all hear the same things when we hear music? Pitch anomia Color anomia is rare: most of us can name about 12 colors. Pitch anomia is common: few have absolute pitch. Lack of training?
Absolute pitch in population of music students in the USA: Caucasians 9% Japanese 26% Korean 37% Chinese 65%. In general Caucasian population < 0.01%
Sound in the brain Acoustic pressure changes auditory nerve impulses. Either by normal hearing mechanism, or via implants. Limited: vibrotactile activations on skin, bone conduction. Time, place, energy, frequency All brain activity is just trains of neural impulses and microcircuit activations. Neural representation of sound can be analyzed by a 4-dimensional spectrograms of the auditory cortex activity.
Pasley et al. Reconstructing Speech from Human Auditory Cortex. PLOS Biology 2012. Sound in the brain A mesh of electrodes measuring cortical electric field potentials allows for reconstruction of speech from measured brain activity. Decomposition of sound S. Koelsch, Toward a neural basis of music perception a review and
updated model. Front. in Psychology 2 (110), 1-20, 2011 Memory imaginary music experience? Neurocognitive Model S. Koelsch, Toward a neural basis of music perception a review and updated model. Front. in Psychology 2 (110), 1-20, 2011 Processing Music
Cognitive model of music processing point to a widely distributed network, focused on pitch and rhythm processing: pitch in lateral Heschls gyrus, timbre in posterior superior-temporal lobes, rhythm in motor/mesiolimbic areas. Conscious hearing requires activation of the auditory cortex (temporal gyrus). We do not have names for internal aspects of music processing in the brain. Auditory Perception Sound is converted into subnetwork
activity binding Many brain areas. In case of speech at least 6 areas are involved and many others activated to give meaning to
words. From: Garagnani et al, 2006 Activation of specific concept/mental state/musical phrase leads to activation of a network of specific structures in the whole brain, contributing to sematic interpretation of the perceived meaning through global brain activity. This activation is sparse and may be better observed by looking at the flattened cortex: http://gallantlab.org/brainviewer/huthetal2012/
It has not been done yet for music Brain modules and cognitive processes Simple and more difficult tasks, requiring the wholebrain network reorganization. Left: 1-back Right: 2back Average over 35 participants.
Left and midline sections. K. Finc et al (HBM, in rev, with World Hearing Center, MPI for Human Development). Sound in the brain Development of connectome
Why such particular networks? Because of long range (white matter) connections in the brain. Connectomes develop before birth and due to the neuroplasticity throughout the whole life, depending on the lowlevel (genetic, epigenetic, signaling pathways) processes, influenced by experience and learning. Individual differences are big. Children connectomes are quite
different, their brain states are simpler than adults. Yet child prodigies plays classical music puzzling! Reward network Why music pleasure? Anatomical connections of some limbic/mesolimbic structures involved in the
emotional processing of music. ACC: anterior cingulate cortex; ant Ins: anterior insula; Am (BL): basolateral amygdala; Am (CM) corticomedial Am, Hipp: hippocampal formation; NAc: nucleus accumbens; OFC: orbitofrontal cortex; PH: parahippocampal gyrus; Temp P: temporal pole.
S. Koelsch, Towards a neural basis of music-evoked emotions. TICS 2010 Do we need ears to hear music? Music perception with implants L. Timm et al. Residual neural processing of musical sound features in adult cochlear implant users. Front Hum Neurosci. 2014 Apr 3;8:181 Our results suggest that even though cochlear implant (CI) users are not
performing at the same level as normal hearing controls in neural discrimination of pitch-based features, they do possess potential neural abilities for music processing. The current behavioral and mismatch negativity findings highlight the residual neural skills for music processing even in CI users who have been implanted in adolescence or adulthood. Challenge: how to automatize improvements of correct discrimination
of all features? Neurofeedback on filtered EEG for automatic adaptation? Sensory substitution Auditory => vision, tactile => visual, auditory, vestibular. Vibrotactile stimulation: change sound to vibrations on skin or tongue. Sense Organ synthesizer patent. Magnetic=>vibrotactile, new sense.
The Sound Shirt connected to a computer system picks up audio signals from microphones and converts it into vibrations delivered by actuators, little motors placed around the shirt, high violins sounds in sleeves, low doublebass on the waist level. A German orchestra, the Jungen Symphoniker Hamburg, organizes concerts for deaf people that feel the music through Sound Shirts.
Direct brain activation Sony patent for direct streaming of multimedia to the brain. Method and system for generating sensory data onto the human neural cortex. US Patent 6536440 B1 It should enable sensory experiences by firing pulses of ultrasound at the head to modify firing patterns in targeted parts of the brain. This would allow the device to trigger various senses, including taste and sound, and even allow the deaf to hear again. Conscious Brain-to-Brain Communication ?
Is it enough to have ears to hear music? Amusia and spatial processing Anatomical locus of amusia, neuroimaging/lesion studies: auditory areas along the STG in pitch discrimination and melodic contour processing; Douglas, K.M. & Bilkey, D.K. Amusia
is associated with deficits in spatial processing. Nature Neuroscience 10, 915-921 (2007) There is no evidence for morphological correlates of amusia in parietal regions. The deficit may derive from changes in neural functioning that are invisible If I have a good musical ear
can I imagine music? The Listener James C. Christensen Varieties of musical imagery Involuntary (or intrusive) musical imagery (INMI), internal perception of spontaneous melodies, repetitive musical sounds.
Liikkanen 2012 Do we need eyes to see? Mental images (visual, auditory) arise with closed eyes. Mental images activate sensory cortex, the processing path is reversed. Reported Vividness of Visual Imagination (VVIQ) correlates well with the early visual cortex activity (Cui, X et al. (2007) . Music Imagery fMRI hemodynamic increase during
an Auditory Imagery Task performed in silence, in the auditory cortex posterior superior temporal gyrus. Zatorre & Halpern, Mental Concerts: Musical Imagery and Auditory Cortex, Neuron 47, 9-12, 2004. Aural imagery or inner hearing is an important aspect of musical development. connect the sound .. with a "feel" they know will produce that sound. The goal of music performance is the reproduction of the internal auditory image. (D.R. Allen musicology thesis, 2007)
An anticipatory image of feedback from an action participates in the selection and initiation or that action. [...] In the closed-loop formulation, the image may serve as a template for comparison with current feedback and need not be activated prior to performance. A.G. Greenwald, Psych. Rev, 77, 73-99, 1970. Aphantasia or imagery agnosia How vividly can people imagine different aspects of music? Even if they can hear precisely melody, pitch, timbre, rhythm, musical space and instruments, can they recall all these aspects? Faw (2009) : ~2.5% of people have no visual imagination.
Zeeman found a patient who abruptly lost the ability to generate visual Images after coronary angioplasty. Later he identified 21 people with no imagery; 19 male, 9 substantial and 12 complete lack of voluntary visual imagery. 5 had affected relatives significant genetic component? 10 had all modalities of imagery affected. Majority described some involuntary imagery, flashes and during dreams. No data on auditory imagery yet
Faw, B. Conflicting intuitions may be based on differing abilities - evidence from mental imaging research. Journal of Consciousness Studies, 2009 Duch, W. Amuzja Wyobraeniowa (Imagery Amusia), book chapter, 2013 Zeeman A, et al, Lives without imagery. Congenital aphantasia. Cortex 2015 Can musical imagery be controlled? Involuntary musical imagery (INMI) Involuntary (or intrusive) musical imagery (INMI), internal perception of spontaneous melodies,
repetitive musical sounds. How common is INMI? How people react to INMI? Can they control it? Is it always pleasant? Passive acceptance and enjoyment is frequent, but a significant number of people want to stop the unwanted earworms. It happens at different time of the day, subjective evaluation of the INMI experience depends on the context situation. Activity and context situation => the experience of mind wandering => enables INMI.
INMI is a common internal experience recruiting brain networks involved in perception, emotions, memory and spontaneous thoughts. Frequency of INMI and its affective aspects has been related to cortical thickness in several cortical and limbic areas. INMI Frequency Internet questionnaires 12.500 people (Liikkanen 2010) Reactions to INMI
Responses from the Finnish study using Internet questionnaires, 12.500 people (Liikkanen 2010). Have you ever done any of the following because of the music that is playing in your head? 1. 74.6% Hum, sing or talk aloud 2. 60.2% Try to figure out the identity of the song 3. 57.3% Listen to the particular song 4. 50.5% Listen to music, radio or television to prevent songs playing 5. 40.7% Sing or play the particular song 6. 29.5% Try to focus on doing something else
7. 0% Avoid listening INMI happens even if you have imagery agnosia, but to know about it one must act: hum, sing, play. Few answers, many questions Individual differences in musical imagery are an interesting subject for
further study. Develop an analog of vividness of visual imagery test. Various forms of imagery agnosia have not yet been studied but should be important in understanding musical talent. Sudden loss of visual imagery is a big surprise. Can deaf people imagine music? Beethoven certainly could. How prevalent is INMI in deaf people? Can we use neurofeedback techniques combined with filtering of EEG
artifacts to significantly improve adaptation to implants? Will training on elementary phonetic contrast discrimination help? How can child prodigies play complex music with their connectomes? Neurohistory: Why particular music forms have appeared around the world? Why some musical forms have become popular at certain times? Why some stayed and other vanished? How is this related to perception mechanism? Technical developments in construction of instruments? Social millieu? Soul or brain: what makes us human?
Interdisciplinary Workshop, Toru 19-21.10.2016 Thank you for synchronization of your neurons! Google: W. Duch => papers, talks, lectures
Political Risk MOF & RMU List of Public Sector Projects Sub Sector List of PPP Projects from Initial Screening with MVA Ministry proposes Sector PPP Projects Tendering Negotiation Contract Management Sector List of PPP Projects from Sector-wide Screening with MVA...
Waste frame directive target: 50% household waste recycled by 2020. 70% construction and demolition waste by 2020. www.carpetrecyclinguk.com. Municipal Waste Landfilled in 2006 compared to targets of the European Landfill Directive (European Environment Agency figures)
This is where the Dictionary ADT and its various implementations which we will study soon come up. decreaseKey: percolateUp increaseKey: percolateDown * Why do I insist on a pointer to each item in decreaseKey, increaseKey, and remove? Because it's hard...
It requires less power input than other mixing configurations .Shear stress levels are low in these reactors Internal gaslift loop reactor (reactor mixed by gas recirculation) was selected for digester studies Gaslift reactor is simple in design and is equipped...
AMITA Health Anti-Harassment and Non-Discrimination Policy It is the policy of AMITA Health to maintain an environment free of discrimination, harassment, or intimidation for all associates and other individuals working at AMITA Health including vendors, contractors, suppliers, and volunteers.
Written annotation Students will be expected to demonstrate the ability to record ideas, observations, insights and independent judgements visually and through informative written annotation, using suitable specialist vocabulary. ... * For GCSE Art and Design we generally have a team...
Ready to download the document? Go ahead and hit continue!