3. Speech Recognition, Analysis, and Synthesis

3. Speech Recognition, Analysis, and Synthesis

MUSIC 318 MINI-COURSE ON SPEECH AND SINGING 3. SPEECH RECOGNITION, ANALYSIS, AND SYNTHESIS Science of Sound, Chapter 16 The Speech Chain, Chapters 7, 8

SPEECH RECOGNITION OUR ABILITY TO RECOGNIZE THE SOUNDS OF LANGUAGE IS TRULY PHENOMENAL. WE CAN RECOGNIZE MORE THAN 30 PHONEMES PER SECOND SPEECH CAN BE UNDERSTOOD AT RATES AS HIGH AS 400 WORDS PER MINUTE. ARTICULATION TESTS

A SET OF SPOKEN WORDS IS PRESENTED AND A LISTENER OR GROUP OF LISTENERS WRITES DOWN WHAT THEY HEAR. THE PERCENTAGE OF WORDS CORRECTLY HEARD IS CALLED THE ARTICULATION SCORE. ARTICULATION SCORES DEPEND UPON THE TEST WORDS USED. ONE TYPE OF WORD LIST CONSISTS OF SINGLE SYLLABLE WORDS SELECTED SO THAT SPEECH SOUNDS IN THE LISTS OCCUR WITH THE SAME RELATIVE FREQUENCY AS THEY DO IN SPOKEN ENGLISH. THESE ARE THE SO-CALLED PHONETICALLY BALANCED OR PB LISTS. ANOTHER TYPE OF WORD LIST IS MADE UP OF TWO-SYLLABLE WORDS LIKE ARMCHAIR, SHOTGUN, OR RAILROAD IN WHICH EACH WORD IS PRONOUNCED WITH EQUAL

STRESS ON BOTH SYLLABLES. ANALYSIS OF SPEECH THREE-DIMENSIONAL DISPLAY OF SOUND LEVEL VERSUS FREQUENCY AND TIME SPEECH SPECTROGRAPH

AS DEVELOPED AT BELL LABORATORIES (1945) DIGITAL VERSION SPEECH SPECTROGRAM SPEECH SPECTROGRAM OF A SENTENCE: This is a speech spectrogram

SPEECH SPECTROGRAM WITH COLOR ADDING COLOR ADDS ADDITIONAL INFORMATION PATTERN PLAYBACK MACHINE CONSONANT SOUNDS, CHANGE VERY RAPIDLY, ARE DIFFICULT TO ANALYZE. THE SOUND CUES, BY WHICH THEY ARE RECOGNIZED, OFTEN OCCUR IN THE FIRST

FEW MILLISECONDS. MUCH EARLY KNOWLEDGE ABOUT THE RECOGNITION OF CONSONANTS RESULTED FROM THE PATTERN PLAYBACK MACHINE, DEVELOPED AT THE HASKINS LABORATORY, WHICH WORKS LIKE A SPEECH SPECTROGRAPH IN REVERSE. PATTERNS MAY BE PRINTED ON PLASTIC BELTS IN ORDER TO STUDY THE EFFECTS OF VARYING THE FEATURES OF SPEECH ONE BY ONE. A DOT PRODUCES A POP LIKE A PLOSIVE CONSTANT STIMULUS PATTERN FOR PRODUCING /t/, /k/, AND /p/ SOUNDS

TRANSITIONS MAY OCCUR IN EITHER THE FIRST OR SECOND FORMANT A FORMANT TRANSITION WHICH MAY PRODUCE /t/, /p/, OR /k/ DEPENDING ON THE VOWEL WHICH FOLLOWS TRANSITIONS THAT APPEAR TO ORIGINATE FROM 1800 Hz

SECOND-FORMANT TRANSITIONS PERCEIVED AS THE SAME PLOSIVE CONSONANT /t/ (after Delattre, Liberman, and Cooper, 1955) PATTERNS FOR SYNTHESIS OF /b/, /d/, /g/

PATTERNS FFOR THE SYNTHESIS OF /b/, /d/, AND /g/ BEFORE VOWELS (THE DASHED LINE SHOWS THE LOCUS FOR /d/) PATTERNS FOR SYNTHESIZING /d/ (a) SECOND FORMANT TRANSITIONS THAT START AT THE / d/-LOCUS

(b) COMPARABLE TRANSITIONS THAT MERELY POINT AT THE /d/-LOCUS TRANSITIONS IN (a) PRODUCE SYLLABLES BEGINNING WITH /b/, /d/, OR /g/ DEPENDING ON THE FREQUENCY LEVEL OF THE FORMANT; THOSE IN (b) PRODUCE ONLY SYLLABLES BEGINNING WITH /d/ SPEECH INTELLIGIBILITY vs SPL

FILTERED SPEECH FILTERS MAY HAVE HIGH-PASS, LOW-PASS, BAND-PASS, OR BAND-REJECT CHARACTERISTICS. SPEECH INTELLIGIBILITY IS USUALLY MEASURED BY ARTICULATION TESTS IN WHICH A SET OF WORDS IS SPOKEN AND LISTENERS ARE ASKED TO IDENTIFY THEM. ARTICULATION SCORES FOR SPEECH FILTERED WITH HIGH-PASS AND LOWPASS FILTERS. THE CURVES CROSS OVER AT 1800 Hz WHERE THE

ARTICULATION SCORES FOR BOTH ARE 67%. NORMAL SPEECH IS INTELLIGIBLE WITH BOTH TYPES OF FILTERS ALTHOUGH THE QUALITY CHANGES. WAVEFORM DISTORTION PEAK CLIPPING IS A TYPE OF DISTORTION THAT RESULTS FROM OVERDRIVING AN AUDIO AMPLIFIER. IT IS SOMETIMES USED DELIBERATELY TO REDUCE BANDWIDTH

ORIGINAL SPEECH MODERATE CLIPPING SEVERE CLIPPING EVEN AFTER SEVERE CLIPPING IN (c) THE INTELLIGIBILITY REMAINS 50-90% DEPENDING ON THE LISTENER

EFFECT OF NOISE ON SPEECH INTELLIGIBILITY THE THRESHOLDS OF INTELLIGIBILITY AND DETECTABILITY AS FUNCTIONS OF NOISE LEVEL CATEGORICAL PERCEPTION OUR EXPECTATIONS INFLUENCE OUR ABILITY TO PERCEIVE SPEECH. EXPECTATIONS ARE STRONGER WHEN THE TEST VOCABULARY HAS FEWER WORDS

SYNTHESIS OF SPEECH AN EARLY ATTEMPT (1791) TO SYNTHESIZE SPEECH WAS VON KEMPELENS TALKING MACHINE. A BELLOWS SUPPLIES AIR TO A REED WHICH SERVES AS THE VOICE SOURCE. A LEATHER VOCAL TRACT IS SHAPED BY THE FINGERS OF ONE HAND. CONSONANTS ARE SIMULATED BY FOUR CONSTRICTED PASSAGES CONTROLLED BY THE FINGERS OF THE OTHEER HAND. WHEATSTONES RECONSTRUCTION OF

KEMPELENS TALKING MACHINE SPEECH SYNTHESIS ACOUSTIC SYNTHESIZERSMECHANICAL DEVICES BY VON KEMPELEN, WHEATSTONE, KRATZENSTEIN, VON HELMHOLTZ, etc. CHANNEL VOCODERS (voice coders)---CHANGES IN INTENSITY IN NARROW BANDS IS TRANSMITTED AND USED TO REGENERATE SPEECH SPECTRA IN THESE BANDS. FORMANT SYNTHESIZERS---USES A BUZZ GENERATOR (FOR VOICED SOUNDS) AND A

HISS GENERATOR (FOR UNVOICED SOUNDS) ALONG WITH A SERIES OF ELECTRICAL RESONATORS (TO SIMULATE FORMANTS). LINEAR PREDICTIVE CODING (LPC)---TEN OR TWELVE COEFFICIENTS ARE CALCULATED FROM SHORT SEGMENTS OF SPEECH AND USED TO PREDICT NEW SPEECH SAMPLES USING A DIGITAL COMPUTER HMM-BASED SYNTHESIS OR STATISTICAL PARAMETRIC SYNTHESIS---BASED ON HIDDEN MARKOV MODELS. USES MAXIMUM LIKELIHOOD TO COMPUTE WAVEFORMS AUTOMATIC SPEECH RECOGNITION BY COMPUTER

AUTOMATIC SPEECH RECOGNITION IS THE HOLY GRAIL OF COMPUTER SPEECH RESEARCH HUMAN LISTENERS HAVE LEARNED TO UNDERSTAND DIFFERENT DIALECTS, ACCENTS, VOICE INFLECTIONS, AND EVEN SPEECH OF RATHER LOW QUALITY FROM TALKING COMPUTERS. IT IS STILL DIFFICULT FOR COMPUTERS TO DO THIS. A COMMON STRATEGY FOR RECOGNIZING INDIVIDUAL WORDS IS TEMPLATE MATCHING. TEMPLATES ARE CREATED FOR THE WORDS IN THE DESIRED VOCABULARY AS SPOKEN BY SELECTED SPEAKERS. SPOKEN WORDS ARE THEN MATCHED TO THESE TEMPLATES, AND THE CLOSEST MATCH IS ASSUMED TO BE THE WORD SPOKEN. CONTINUOUS SPEECH RECOGNITION IS MUCH MORE DIFFICULT THAN INDIVIDUAL WORDS

BECAUSE IT IS DIFFICULT TO RECOGNIZE THE BEGINNING AND END OF WORDS, SYLLABLES, AND PHONEMES. RECOGNIZING WORD BOUNDARIES THE SPACE NEARBY WORD BOUNDARIES CAN BE LOCATED BY THE INITIAL OR FINAL CONSONANTS THE AREA AROUND

WORD BOUNDARIES ARE DIFFICULT TO LOCATE HIDDEN MARKOV MODELS (HMMs) INVENTED (IN THE EARLY 1900s) BY RUSSIAN MATHEMATICIAN A.A. MARKOV DURING HIS STUDIES OF WORD STATISTICS IN LITERARY TEXTS. DURING THE 1980s HMMs BECAME THE MOST POPULAR SPEECH RECOGNITION METHOD. HIDDEN MARKOV MODEL REFPRESENTATION. (a) Example of a word represented by four

internal states 1,2,3,4. (b) Abstract representation of (a) snowing states 1-4 sequential transition probabilites a1. . . .a4; self-transition probabilities d1 .d4; and within-state probability distribution p1 . . .p4 (DENES et al.) SPEAKER IDENTIFICATION: VOICEPRINTS SPEECH SPECTROGRAMS PORTRAY SHORT-TERM VARIATIONS IN INTENSITY AND FREQUENCY IN GRAPHICAL FORM. THUS THEY GIVE MUCH USEFUL INFORMATION ABOUT SPEECH ARTICULATION. WHEN TWO PERSONS SPEAK THE SAME WORD, THEIR ARTICULATION IS SIMILAR

BUT NOT IDENTICAL. THUS SPECTROGRAMS OF THEIR SPEECH WILL SHOW SIMILARITIES BUT ALSO DIFFERENCES. SPECTROGRAMS OF THE SPOKEN WORD SCIENCE. WHICH TWO SPECTROGRAMS WERE MADE BY THE SAME SPEAKER? THE TWO SPECTROGRAMS AT THE TOP WERE MADE BY THE SAME SPEAKER. THE TWO SPECTROGRAMS AT THE BOTTOM WERE MADE BY TWO OTHER SPEAKERS

FROM THE WINTER 2010 ISSUE OF ECHOES SPEECH RECOGNITION CAN BE IMPOROVED BY JOINT ANALYSIS OF THROAT AND ACOUSTIC MICROPHONE RECORDINGS, ACCORDING TO A PAPER IN THE SEPTEMBER ISSUE OF IEEE TRANSACTION ON AUDIO. SPEECH, AND LANGUAGE PROCESSING. A PROPOSED MULTIMODAL SYSTEM IMPROVES PHONEME RECOGNITION RATE. A PAPER IN THE NOVEMBER 2010 ISSUE OF NATURE PROPOSES THAT THE AMINO ACID COMPOSITION IN THE GENE FOXP2 HAS UNDERGONE ACCELERATED EVOLUTION,, AND THIS TWO-AMINO-ACID CHANGE OCCURRED AROUND THE TIME OF LANGUAGE EMERGENCE IN HUMANS AND MAY HAVE PLAYED AN IMPORTANT ROLE.

HUMANS USE TACTILE INFORMATION DURING AUDITORY SPEECH PERCEPTION, ACCORDING TO A PAPER IN THE 26TH NOVEMBER ISSUE OF NATURE. APPLYING TINY BURSTS OF ASPIRATION (SUCH AS WOULD BE PRODUCED BY PLOSIVE CONSONANT

TO THE RIGHT HAND OR THE NECK MADE THE SYLLABLES MORE APT TO BE HEARD AS SPIRATED (

RATHER THAN , FOR EXAMPLE).

Recently Viewed Presentations

  • The 'Lost' of the First World War

    The 'Lost' of the First World War

    You may wish to use our short introductory film as a starter, to introduce the work of the Commonwealth War Graves Commission to your students. The film is a part of this pack. The images here are from the Commonwealth...
  • Optical Fiber Communications - Ryerson University

    Optical Fiber Communications - Ryerson University

    They can be fabricated either from optical fibers or by means of planar optical waveguides using material such as LiNbO3, InP, silica, silicon oxynitride, or various polymers. * The 2 2 Fiber Coupler P0 is the input power, P1 is...
  • Warmup We know that the first civilizations formed

    Warmup We know that the first civilizations formed

    Ancient Egypt. World History I. The Sphinx is an iconic structure from Egyptian Civilization. The Sphinx is located in the town of Giza. It sits in front of three massive pyramids and several smaller ones/a whole burial complex in Giza;...
  • metabolomics - Gordon Medical Associates

    metabolomics - Gordon Medical Associates

    What is Metabolomics? It is the measurement of the 2000-3000 small organic molecules (< 1200 daltons) that the body produces. These are the amino acids , lipids, smaller nucleotides, vitamins, and carbohydrates. These are the building blocks, not the larger...
  • Evidence-Based Instruction Module Pre-Test Please complete the Pre-Test

    Evidence-Based Instruction Module Pre-Test Please complete the Pre-Test

    Systematic phonics instruction A planned sequence of phonic elements 2. ... words Introduce graphemes Emphasize learning to chunk or break words into their basic parts Stevenson Language Program Spalding Method Corrective Reading Lindamood Phonemic Sequencing Program for Reading, Spelling, and...
  • Welcome to McLaughlin College - York University

    Welcome to McLaughlin College - York University

    Terry Conlin, Academic Advisor, McLaughlin College 227 McLaughlin [email protected] 736-5128 Enroll with permission: Jan 31 Reading Week: Feb 20 - 24 Feb. 10: drop date for full-year courses March 9: drop date for winter term 3 credit courses Look for...
  • Vocabulary 10

    Vocabulary 10

    To destroy completely; to reduce to utter ruin. It appears that Syria's President Assad is trying to . annihilate. the people living in Aleppo. Benefactor (noun) A person who provides help, especially by giving money. People who donated more than...
  • Concrete Poetry - ednet.ns.ca

    Concrete Poetry - ednet.ns.ca

    Concrete Poetry Concrete Poetry Concrete poetry involves arranging the letters or words that describe an object into a visual image that also describes the object. It is a kind of painting with letters or words as the medium. Creating Concrete...