Augmenting Speech Recognition Systems with Myoelectric Pattern Classification

Augmenting Speech Recognition Systems with Myoelectric Pattern Classification Dawn MacIsaac, PhD Candidate Institute of Biomedical Engineering University of New Brunswick Outline 20-20 speech V5 Continuous Speech Recognition System MES Pattern Classification system based on wavelet transform features, a PCA dimensionality block and a Linear discriminator Experimental protocol - nonrandom and random signal acquisition Electrode Placement signal segmentation signal classification Results

- classification errors - effects of Pre-triggering Conclusion Department of National Defense, UK Attempting to enhance fighter pilot performance and safety by automating cockpit controls with speech - currently, pilots of the Eurojet are in danger of crashing because of the amount of time they spend looking at their controls instead of their flying environment - using speech recognition systems like the 20-20 speech continuous speech recognition system, pilots can use words to control some of the jet instrumentation and displays. - The problem is that systems like the 20-20 speech system use auditory inputs which can be severely corrupted by the noise conditions inside the cockpit of a jet. - myoelectric signals dont have this problem so they are an obvious alternative for assisting the conventional

speech recognition system. Advantage of Myoelectric Signals in Speech Recognition - Not corrupted by audio noise - there are similar sounding words with unique mouth positions implying unique myoelectric signals Thus, if the myolectric signals of the mouth can be sufficiently classified, they can be used to augment the traditional speech classifier Is there speech informatiuon in the myolectric signal? Experimental Protocol We collected two different types of data - random data and nonrandom data - 7 subjects of non-random data repeating numbers 1-9 in blocks of 60 seconds with breaks between - 2 subjects of random data repeating numbers 1-9 randomly in blocks of 60 seconds with breaks

between - 5 myoelectric signals were collected along with the acoustic signal levator Anguli Oris Zygomaticus Major Platysma Depressor Anguli Oris Anterior Belly of the Digastric Electrode Placement in Mask Data Segmentation Pretrigger Used the onset of sound as determined by the audio acoustic signal as the trigger full-wave rectification, followed by a moving average filter window length of 1024 ms Pre-triggered because the

onset of myo activity precedes the onset of the auditory signal LAI ZYG PLT DAO ABD Data Classification Wavelet transform feature set yk n k 1

x m h 2 n m k m kn t h *k a k nT t - simplet Wavelet (Daublets modification) Data Classification

Principle Component analysis for Dimensionality Reduction v p L M p z i u i i 1 bi u i i L 1 1 P M p E L z i bi 2 p 1 L1

1 P p bi zi P p 1 1 M T EL ui vui 2 i L1 Which is minimized as long as v u i i u i Which defines ui, the principle component 2 Data Classification Linear Discrimenant Analysis

d k ln P y k x T C 1m k 1 T 1 mk C mk 2 Where - C is the pooled covariance matrix - x are your observations - m are the means of your observations - P(y) are your probabilities of obtaining your observations Data Classification Audio Signal Data Segmentation Myoelectric Signal

Feature Extraction Data Reduction Data Classification Word Results Non-Random Data Random Data 15 30 10

5 S1-2 S2-2 Exp1Avg 25 Classification Error (%) Classification Error (%) S1-1 S2-1 S3-1 S4-1 S5-1 20

15 10 5 0 0 100 200 300 400 Pretrigger (ms) 500 600

700 0 0 100 200 300 400 Pretrigger (ms) 500 600 700

Conclusions and Future Work The myoelectric signal does contain speech information Myoelectric signal precedes the acoustic information Optimal pretrigger 400 500 ms In combination with auditory signal, better classification accuracy may be possible Acknowledgements Supervisors: Dr. Kevin Englehart Dr. Bernard Hudgins Dr. Dennis Lovely Defense Evaluation Research Agency National Sciences and Engineering Research Council of Canada

Recently Viewed Presentations

  • COMMON CORE READING DISCUSSION QUESTIONS (Complete at least

    COMMON CORE READING DISCUSSION QUESTIONS (Complete at least

    RL.3.1. Fiction. Explain an important event, setting or character. Use details from the text in your description. Mark with a sticky note a section of the text that is confusing or makes you wonder.
  • Air Car - CSUSM

    Air Car - CSUSM

    Air Car Zero Pollution Motors Specs Est. price of $17,800 106 MPG Compressed air fuel tank: 3200 ft3 @ 4500 psi 75hp engine with a top speed of 96 850 mile range on a tank of gas Co2 emissions -...
  • Chapter 8: Mechanical Failure - Faculty Web

    Chapter 8: Mechanical Failure - Faculty Web

    Figure 8.8 Fatigue corresponds to the brittle fracture of an alloy after a total of N cycles to a stress below the tensile strength. * Section 6.17 Application of Fatigue Testing (c)2003 Brooks/Cole, a division of Thomson Learning, Inc. Thomson...
  • Teknik perbaikan tanah - WordPress.com

    Teknik perbaikan tanah - WordPress.com

    Unit penggetar kemudian diangkat secara bertahap Untuk dry method air diganti dengan udara KONTROL KUALITAS Pemadatan Dangkal: Density & kadar air dengan sand cone, Nuclear Density Test, Rubber Ballon, Dynamic Cone Penetration, Cone Penetrometer, Plate Bearing Pemadatan Dalam: DCPT, SPT,...
  • SOCIALIZATION All encompassing educational process from which values,

    SOCIALIZATION All encompassing educational process from which values,

    SOCIALIZATION All encompassing educational process from which values, goals, beliefs, attitudes, and sex roles are acquired. Viewgraph #5-1 SOURCES THAT IMPACT SOCIALIZATION Viewgraph #5-2 Nation Region Community Family Media Peers / friends DR.
  • AP World History Review - White Plains Middle School

    AP World History Review - White Plains Middle School

    cultural diffusion to JAPAN elements of Tang-Song economic prosperity to 1250 CE The full incorporation of southern China into the economy as a major food-producing region, center of trade; commercial expansion with West, southern Asia, southeast Asia establishment of Chinese...
  • 0 9 IAT 102 Graphic Design  The dispute

    0 9 IAT 102 Graphic Design The dispute

    Typography helps readers navigate through the flow of content. A designer's task is to provide ways into and out of the sea of words by breaking up text into smaller segments and offering alternate routes through the mass of information....
  • Diapositiva 1

    Diapositiva 1

    OBJETIVOS. Agroalimentación. 4.-Potenciar especialmente la formación de jóvenes investigadores (master y doctores). 5.-Propiciar la transferencia de know-how de los organismos que integran la red a empresas del sector y cooperativas de productores, con el objetivo de mejorar su rendimiento, la...