Insert Presentation title here

Insert Presentation title here

Mental Simulations of Voice and Speech during Reading of Quotations Christoph Scheepers Bo Yao Pascal Belin Introduction Direct versus indirect speech Two quoting styles that exist in many languages Often imply differences in perspective (cf. change in corresponding person and tense features), e.g. Mary said: I am hungry. Mary said (that) she was hungry. Important differences in usage While the main function of indirect speech appears to be to provide a description of what was said by another person, the use of direct speech typically also involves a demonstration (or depiction) of how it was said, e.g. ..and she was like: Oh my god! Im so hungry! Direct

speech as vivid demonstration Tannen (1986; 1989) Clark & Gerrig (1990) Wade & Clark (1993) Speakers are more likely to use direct rather than indirect speech quotations when instructed to entertain, as opposed to just inform, a listener Comprehension of Quotations Implications for reading of quotations? Intuitive experience of an inner voice, especially during silent reading of direct speech quotations No direct, objective confirmation of this intuition so far Embodied cognition Language processing is grounded in mental re-enactment (perceptual simulation) of perceptual states and actions (Barsalou, 1999; 2008) Given that the use of direct speech is grounded in vivid demonstrations, direct speech should trigger more enriched perceptual simulations (spontaneous auditory imagery) of, e.g., voice and/or manner of speaking than indirect speech Present investigations Study 1 (fMRI+eye-tracking): Voice-selective auditory cortex areas (cf. Belin et al., 2000)

become more active during silent reading of direct rather than indirect speech quotations Study 2 (eye-tracking): The context preceding a quotation (implying a fast- or a slowspeaking quoted speaker) has an influence on how fast one would read a direct-speech quotation, but not how fast one would read an indirect speech quotation Study 1 Method Event-related fMRI combined with eye-tracking (silent reading of text) Participants 16 native English speakers aged 18 - 44 years, 10 female Stimuli 90 short stories containing direct or meaning-equivalent indirect speech quotations (counterbalanced via Latin square) from fictitious protagonists, e.g. PhD student Ella was summoned to her supervisor Jims office to give a report on her current progress. Ella asked for an extension but Jim looked concerned. He said: Hmm, we really need those data in by next month for that conference. (direct speech) PhD student Ella was summoned to her supervisor Jims office to give a report on her current progress. Ella asked for an extension but Jim looked concerned. He said that they really needed those data in by next month for that conference. (indirect speech)

20 vocal (e.g. spoken vowels; laughing; coughing..) and 20 non-vocal (e.g. telephone ringing; dog barking..) sound-clips for a functional voice localizer session after the main reading session Yao, Belin, Scheepers (2011) Journal of Cognitive Neuroscience Study 1: Tasks (1) Main reading session (2) Voice localizer session Listen to 20 vocal and 20 non-vocal soundclips (8 sec each) Eyes closed! Same as in Belin et al. (2000, Nature) Read stories silently for comprehension Yao, Belin, Scheepers (2011) Journal of Cognitive Neuroscience Study 1: Results Auditory voice-localizer regions of interest (ROIs) in green

Within those regions, significantly higher activation (BOLD response) for direct speech than for indirect speech quotations Measured from 1st fixation into the critical quotation passage to first progressive saccade out of the critical quotation passage (eye-tracking) Strongly right-lateralized pattern Two main clusters of enhanced activity for direct (as opposed to indirect) speech quotations were located in voice-selective areas along posterior and middle parts of the right STS Interestingly, similar right-lateralized activation patterns have previously been found in association with emotional prosody (e.g. Wiethoff et al., 2008; Wildgruber et al., 2006) No differences in reading rate between direct vs. indirect speech quotations

Mean reading time per word: 204 ms vs. 203 ms Suggests no difference in processing difficulty between the two quotation styles Yao, Belin, Scheepers (2011) Journal of Cognitive Neuroscience Study 1: Discussion Evidence for enhanced auditory mental simulation/imagery of voice during silent reading of direct speech quotations (compared to meaning-equivalent indirect speech quotations) Auditory cortex regions that are particularly interested in human voices (cf. Belin et al., 2000) show more top-down activation during silent reading of direct speech quotations Can auditory voice simulation effects be demonstrated behaviourally as well?... Study 2 Stimuli 24 short stories containing direct or meaning-equivalent indirect speech quotations The context preceding those quotations implied either a fast or a slow-speaking quoted protagonist:

Fast context: Teenage pianist Bobby was to take tart in a local talent competition. He was extremely nervous before his performance. His mother encouraged him but he was all shaking and said: No! I cant do it! This is the end of the journey because I wont make it this time. His mother (direct speech) but he was all shaking and said that he couldnt do it and that this was the end of the journey because he wouldnt make it this time. His mother (indirect speech) Slow context: At the Royal Infirmary, an old man was dying, and too weak to sit up. He wanted to say something, so his daughter placed a cushion under his head. Slowly, he looked around and said: Im grateful you are all here. This is the end of the journey because I wont make it this time. Then he closed his eyes (direct speech) and said that he was grateful for their coming and that this was the end of the journey because he wouldnt make it this time. Then he closed his eyes (indirect speech) Yao & Scheepers (2011) Cognition Study 2 Method Oral reading (20 participants): Read the stories aloud in a random order Digital audio recordings throughout the session

Reading rate for critical quotation passages analysed in numbers of syllables per second Silent reading (48 participants): Read the stories (plus fillers) silently in a random order Answer yes/no comprehension question after each story Eye-tracking (EyeLink 1000) throughout the session Reading rate for the critical quotation passages analysed in terms of ~ firstpass reading time (time from fixating the passage for the first time until another region is fixated) Yao & Scheepers (2011) Cognition Study 2: Hypotheses Prior research (Alexander & Nygaard, 2008) has demonstrated that oral and silent reading rates are modulated by auditory imagery People read faster when instructed to imagine familiar fast speakers as authors of text for reading Here, we do not use instructions, but rather look at spontaneous

reading rate adjustments as a function of quoting style (direct / indirect speech) and context (implying fast or slow quoted speaker) Inspired by Alexander & Nygaard (2008), we expect reading rates to be modulated by context (fast or slow quoted speaker) Informed by Study 1, we expect contextual modulation of reading rate to be stronger for direct than for indirect speech quotations Yao & Scheepers (2011) Cognition Study 2: Results Oral reading (syllables/sec) Silent reading (FPRT in ms) 6.2 2100 6.1 2000 Context fast Context slow

6 5.9 1900 Context fast Context slow 1800 5.8 5.7 1700 5.6 1600 Reading aloud task: Oral reading rates (in syllables/second) for the critical quotation passages Clear Context Quoting Style interaction Context-fast > Context-slow in direct speech

No such context effect in indirect speech With direct speech quotations, participants spontaneously act out the contextually implied speech rate of the quoted speaker Silent reading task (eye-tracking): First-pass reading times (in ms) for the critical quotation passages Clear Context Quoting Style interaction Context-fast < Context-slow in direct speech No such context effect in indirect speech With direct speech quotations, silent readers adjust their reading rates to the contextually implied speech rate of the quoted speaker Yao & Scheepers (2011) Cognition Conclusions Converging evidence from fMRI and eye-tracking, suggesting that readers are more likely to mentally simulate (or spontaneously

imagine) aspects of the quoted speakers voice (and manner of speaking) during silent reading of direct speech quotations Consistent with direct speech as vivid demonstration (Clark & Gerrig, 1990) Perceptual grounding (cf. Barsalou, 1999; 2008) of direct speech: Since direct speech typically occurs in the context of vivid demonstrations/re-enactments of actual speech acts, a direct quotation style is more likely to be taken as a cue to engage in enriched perceptual simulations of voice than an indirect quotation style (focus on content) and maybe not only of voice! (to be continued)

Recently Viewed Presentations

  • Ch. 29 - Sponges &amp; Cnidarians

    Ch. 29 - Sponges & Cnidarians

    Ch. 32 - 33 - Sponges, Cnidarians, & Ctenophores. What is an invertebrate? An animal without a backbone. What are some examples? Animals are classified by their shape or symmetry. What is symmetry? Arrangement of body parts around a central...
  • MILLER/SPOOLMAN LIVING IN THE ENVIRONMENT Chapter 19 Climate

    MILLER/SPOOLMAN LIVING IN THE ENVIRONMENT Chapter 19 Climate

    According to the IPCC, a warming of 2 C° (3.6 F°) over 2005 levels is unavoidable, and an increase of at least 3 C° (5.4 F°) is likely sometime during this century (Figure 19-B). (Data from 2007 Intergovernmental Panel on...
  • Duffy was inspired to write this poem by

    Duffy was inspired to write this poem by

    "spools of suffering set out in ordered rows" (Alliteration, Assonance, Word choice/ Imagery) Reel of pictures taken in war zone. Photos have captured and taken on the pain of those in the picture. The spools themselves now represent something agonising...
  • LEADING ORGANISATIONAL CHANGE - CA Sri Lanka

    LEADING ORGANISATIONAL CHANGE - CA Sri Lanka

    e.g. All examiners allocate marks to the correct encoding of e-assignment files and the practice becomes faculty or USQ policy. Turnitin and other software mandated by the USQ similarly place the onus on the student to guarantee the integrity of...
  • Stereotypes and Barriers: Effects On C-suite Career ...

    Stereotypes and Barriers: Effects On C-suite Career ...

    Stereotypes and Barriers: Effects On C-suite Career Progression of African American Females: A Narrative Inquiry . Research methodology and design. 1. Qualitative method. 2. Narrative inquiry design. Qualitative method
  • Barmah-Millewa Forest

    Barmah-Millewa Forest

    Oncountry Learning: Indigenous Studies 10th Anniverary, 2013 Introductory Lecture and Overview of Yorta Yorta culture, history and connections
  • Using Realistic MHD Simulations for Modeling and Interpretation

    Using Realistic MHD Simulations for Modeling and Interpretation

    Comparison of the mean Doppler shift for different distances from the disc center reveals a mean blue shift of about 50 - 100 m/s at the disc center, and a 100 - 210 m/s red-shift at the 600 distance. In...
  • How effective was the Tsar at ruling Russia?

    How effective was the Tsar at ruling Russia?

    "Learning to Lead our Lives" How effective was the Tsar at ruling Russia? Skill: ICT/ Numeracy NGfL: Russia 1900-1924 1900 The Tsar had absolute and divine rule Autocracy Censorship Secret Police 1905 Sailors Mutinied The Russo-Japanese War Economic Depression Bloody...