Practical Prediction and Prefetch for Faster Access to Mobile ...

Practical Prediction and Prefetch for Faster Access to Mobile ...

Recognizing Smoking Gestures with Inertial Measurements Unit (IMU) Abhinav Parate Mengh-Chieh Chiu Chaniel Chadowitz Deepak Ganesan Evangelos Kalogerakis University of Massachusetts, Amherst UNIVERSITY OF MASSACHUSETTS AMHERST Smoking According to CDC, smoking is responsible for 440,000 deaths in the United States $96 billion in medical costs $97 billion in lost productivity Over a billion smokers worldwide! UNIVERSITY OF MASSACHUSETTS AMHERST Smoking Cessation 40% smokers try to quit each year. Most efforts end in relapse.

Well-timed interventions help! Less than 10 % success rate Requires presence of a ubiquitous agent UNIVERSITY OF MASSACHUSETTS AMHERST RisQ: A Mobile Solution for Intervention Smartphone Always with the user Can sense user environment Real-time intervention Wristband Equipped with 9-axis Inertial Measurement Unit (IMU) Real-time smoking detection

UNIVERSITY OF MASSACHUSETTS AMHERST Hand-to-mouth gesture characteristics (a) Smoking when standing still (c) Eating with a spoon Dwell Time Dwell Time cigarette towards mouth return to normal (b) Smoking when walking food towards mouth (d) Drinking from a cup Dwell Time cigarette towards mouth return to normal

Dwell Time return to normal cup towards mouth return to normal IMU Signals for various hand-to-mouth gestures UNIVERSITY OF MASSACHUSETTS AMHERST 5 1: Orientation-dependent Characteristics Gesture characteristics for the same smoking gesture Signal change users orientation Signal characteristics characteristics change with users body

body orientation when the user faceswith in opposite directions UNIVERSITY OF MASSACHUSETTS AMHERST 6 2: Unknown Gesture Boundaries Where does a gesture start? How How to to identify identify gesture gesture boundaries boundaries in in a a passive passive manner? manner?

UNIVERSITY OF MASSACHUSETTS AMHERST 7 3: Collecting labels for training How to collect fine-grained labels for training a classification model? UNIVERSITY OF MASSACHUSETTS AMHERST 8 Outline Introduction Challenges Data Collection using IMUs Data Processing Pipeline Evaluation Conclusion UNIVERSITY OF MASSACHUSETTS AMHERST 9

IMU signal: Background Quaternion Mathematical entity to represent orientation of an object in 3D space q = q_s + q_x i+q_y j+q_z k One scalar and 3 imaginary components y q = cos(a/2) + x sin(a/2) i + y sin(a/ 2) j + z sin(a/2) k UNIVERSITY OF MASSACHUSETTS AMHERST Angle of

rotation: a x z 10 3D coordinates using Quaternions Point p w.r.t. IMUs local frame of reference IMU device orientation in the form of a quaternion q Coordinates of p w.r.t world frame of reference p = q.p.q q = cos(a/2) + x sin(a/2) i + y sin(a/2) j + z sin(a/2) k q = cos(a/2) x sin(a/2) I y sin(a/2) j z sin(a/2) k UNIVERSITY OF MASSACHUSETTS AMHERST 11 Wrist Trajectory using Quaternions Visualizing Visualizing gestures gestures using

using aa wristband wristband and and an an armband armband equipped equipped with with IMUs IMUs UNIVERSITY OF MASSACHUSETTS AMHERST 12 Outline Introduction Challenges Data Collection using IMUs Data Processing Pipeline Evaluation Conclusion UNIVERSITY OF MASSACHUSETTS AMHERST 13 Segment

Extraction Gesture Classificatio n Feature Extraction Session Detection Segment q_s q_x q_y q_z Relative to elbow z z z 1 1

0 0.5 0.5 x 0 0.5 y 0.5 x 10 Wrist Trajectory Computation y 1 0 0.5 0.5

1 0 Trajectory Smoothing Peak Detectio n algorith m UNIVERSITY OF MASSACHUSETTS AMHERST x y 10 Wrist's Rest-point Computation Execute d on phone

14 Segment Extraction Feature Extraction Gesture Classificatio n Session Detection Orientation Independent Features A set of 34 spatio-temporal features Duration-based features (4) Gesture duration, time to raise arm, etc. Velocity-based features (6) Maximum wrist speed, etc.

Displacement-based features (6) Vertical displacement, XY displacement, etc. Angle-based features (18) Angle with the gravity, angular velocity, etc. UNIVERSITY OF MASSACHUSETTS AMHERST 15 Segment Extraction Feature Extraction Gesture Classificatio n Session

Detection Session Detection Layer Segment N-1 Session Segment N+1 Conditional Random Field Model Segment N features Gesture Recognition Layer Supervised Gesture Classification Feature Extraction Vertical Displacement Average Speed

UNIVERSITY OF MASSACHUSETTS AMHERST Angular twist Gesture duration 16 Outline Introduction Challenges Data Collection using IMUs Data Processing Pipeline Evaluation Conclusion UNIVERSITY OF MASSACHUSETTS AMHERST 17 Evaluation Dataset Dataset 28 hours of data from 15 volunteers 17 smoking sessions (369 puffs)

10 eating sessions (252 food bites) 6 drinking sessions UNIVERSITY OF MASSACHUSETTS AMHERST 18 Smoking Session Detection 16 12 17 8 4 15 Statistic 0 al t To

ns o i s s Se s n o si s Se ed t ec t e D Avg Std Dev Duration of smoking sessions

326.21 19.65 s Error in estimation 65.7 30.6 s Leave-one-session-out Cross-validation UNIVERSITY OF MASSACHUSETTS AMHERST 19 Smoking Gesture Recognition Mechanism Performance Metrics Accuracy Recall Precision

FPR Random Forests 93.00% 0.85 0.72 0.023 CRF 95.74% 0.81 0.91 0.005 10-fold Cross-validation 369 puffs 252 bites 4976 other gestures

UNIVERSITY OF MASSACHUSETTS AMHERST CRF improves precision at a cost of slight drop in recall 20 User Study Recruited 4 subjects for 3 days. Used our smoking detection app developed for Android OS. UNIVERSITY OF MASSACHUSETTS AMHERST 21 User Study Day 1 Day 2 Day 3 Rarely

Fewer than missed 2 false any smoking positivessession. per day! UNIVERSITY OF MASSACHUSETTS AMHERST 22 Conclusion An algorithm to recognize hand-gestures using a wristband Demonstrated an application to detect smoking in real-time. Smartphones in conjunction with wearable accessories present a great platform to sense health-related behaviors like smoking, eating, and so on. Remarkable opportunity to create effective intervention strategies using smartphones. Software/Code available at: http://people.cs.umass.edu/~aparate/risq.html

UNIVERSITY OF MASSACHUSETTS AMHERST 23 Eating Gesture Recognition Mechanism Eating Sessions All data Recall Precision Recall Precision Bite-Counter 0.60 0.57

0.65 0.03 Random Forests 0.92 0.78 0.69 0.64 CRF N/A N/A 0.64 0.78 Eating gesture recognition Bite-Counter detects food bites when user explicitly indicates that eating session is in progress.

UNIVERSITY OF MASSACHUSETTS AMHERST 24 System Overhead Statistic Value Time for segmentation 92.34ms Time for feature extraction 79.88ms Time for CRF inference Memory 5.89ms 12-20MB Binary Size

1.7MB Measured on Samsung Galaxy Nexus UNIVERSITY OF MASSACHUSETTS AMHERST 25 Optimizing Performance Optimal cost for best performance Use a cost function during RF classifier training to assign penalty for missing a smoking gesture. High cost results in lower precision Low cost results in lower recall and low FPR UNIVERSITY OF MASSACHUSETTS AMHERST 26 3: Concurrent Activities

User is stationary User is walking Concurrent activities can modify the characteristic patterns of gestures UNIVERSITY OF MASSACHUSETTS AMHERST 27

Recently Viewed Presentations

  • Electronic Coordination Tool (ECT) Training Slides ...

    Electronic Coordination Tool (ECT) Training Slides ...

    End User Training Materials Acquisition Strategy Workflow Acquisition Resources and Analysis (ARA) Version 1.1, 07/07/2014 This provide a high-level view of the stages at which the PM uploads versions of the documents being coordinated to ECT, as well as the...
  • Myth - University of Kentucky

    Myth - University of Kentucky

    Myth Traditional and contemporary approaches ... This saga also involves a galaxy teeming with inhabited worlds held together by a collapsing galactic empire using hyperdrives (for long-distance transportation). ... One can speak of . . . the mythic elements within...
  • 2001 Paradigm Publishing Inc. Computer Concepts 1-1 Types

    2001 Paradigm Publishing Inc. Computer Concepts 1-1 Types

    Public-Key Encryption (Asymmetrical) Conventional Encryption (Symmetrical) File encrypted with public key Locked file sent to recipient File decrypted with secure key File encrypted Locked file sent to recipient File decrypted with identical key The Future of Computing: Parallel Processing Deep...
  • What Educational & Research Networks Need To Know

    What Educational & Research Networks Need To Know

    ASN assignments. Transfers. Reverse DNS. Record Maintenance. ... Initially, specified recipient transfers were allowed only in the ARIN region. Some years after the first policy was implemented, the community created an inter-RIR transfer policy, which in essence is the same...
  • Financial Information Management Data quality Stefano Grazioli Critical

    Financial Information Management Data quality Stefano Grazioli Critical

    The quality of the data stored in organizational databases is often poor. 10-25% of the records have inaccuracies or missing elements. Data frequently misinterpreted
  • Cumberland Mills Elementary School  Black History Presentation Dr.

    Cumberland Mills Elementary School Black History Presentation Dr.

    Cumberland Mills Elementary School Black History Presentation Dr. Linda Wilson-Jones Fayetteville State University Harriet Tubman Frederick Douglass Sojourner Truth Phillis Wheatley Jackie Robinson Hank Aaron Booker T. Washington Marian Anderson William Christopher Handy W.E.B. Du Bois Rosa Parks Martin Luther...
  • Wandering In The Wilderness: - Lake Forest church of Christ

    Wandering In The Wilderness: - Lake Forest church of Christ

    Psalm 8 (NKJV) 1 O Lord, our Lord, How excellent is Your name in all the earth, Who have set Your glory above the heavens! 2 Out of the mouth of babes and nursing infants, You have ordained strength, Because...
  • VSEPR PowerPoint - Tetrahedral, Trigonal Bipyramidal, etc.

    VSEPR PowerPoint - Tetrahedral, Trigonal Bipyramidal, etc.

    Valence Shell Electron Pair Repulsion Theory Tetrahedral Trigonal pyramidal Bent