Practical Prediction and Prefetch for Faster Access to Mobile ...
Recognizing Smoking Gestures with Inertial Measurements Unit (IMU) Abhinav Parate Mengh-Chieh Chiu Chaniel Chadowitz Deepak Ganesan Evangelos Kalogerakis University of Massachusetts, Amherst UNIVERSITY OF MASSACHUSETTS AMHERST Smoking According to CDC, smoking is responsible for 440,000 deaths in the United States $96 billion in medical costs $97 billion in lost productivity Over a billion smokers worldwide! UNIVERSITY OF MASSACHUSETTS AMHERST Smoking Cessation 40% smokers try to quit each year. Most efforts end in relapse.
Well-timed interventions help! Less than 10 % success rate Requires presence of a ubiquitous agent UNIVERSITY OF MASSACHUSETTS AMHERST RisQ: A Mobile Solution for Intervention Smartphone Always with the user Can sense user environment Real-time intervention Wristband Equipped with 9-axis Inertial Measurement Unit (IMU) Real-time smoking detection
UNIVERSITY OF MASSACHUSETTS AMHERST Hand-to-mouth gesture characteristics (a) Smoking when standing still (c) Eating with a spoon Dwell Time Dwell Time cigarette towards mouth return to normal (b) Smoking when walking food towards mouth (d) Drinking from a cup Dwell Time cigarette towards mouth return to normal
Dwell Time return to normal cup towards mouth return to normal IMU Signals for various hand-to-mouth gestures UNIVERSITY OF MASSACHUSETTS AMHERST 5 1: Orientation-dependent Characteristics Gesture characteristics for the same smoking gesture Signal change users orientation Signal characteristics characteristics change with users body
body orientation when the user faceswith in opposite directions UNIVERSITY OF MASSACHUSETTS AMHERST 6 2: Unknown Gesture Boundaries Where does a gesture start? How How to to identify identify gesture gesture boundaries boundaries in in a a passive passive manner? manner?
UNIVERSITY OF MASSACHUSETTS AMHERST 7 3: Collecting labels for training How to collect fine-grained labels for training a classification model? UNIVERSITY OF MASSACHUSETTS AMHERST 8 Outline Introduction Challenges Data Collection using IMUs Data Processing Pipeline Evaluation Conclusion UNIVERSITY OF MASSACHUSETTS AMHERST 9
IMU signal: Background Quaternion Mathematical entity to represent orientation of an object in 3D space q = q_s + q_x i+q_y j+q_z k One scalar and 3 imaginary components y q = cos(a/2) + x sin(a/2) i + y sin(a/ 2) j + z sin(a/2) k UNIVERSITY OF MASSACHUSETTS AMHERST Angle of
rotation: a x z 10 3D coordinates using Quaternions Point p w.r.t. IMUs local frame of reference IMU device orientation in the form of a quaternion q Coordinates of p w.r.t world frame of reference p = q.p.q q = cos(a/2) + x sin(a/2) i + y sin(a/2) j + z sin(a/2) k q = cos(a/2) x sin(a/2) I y sin(a/2) j z sin(a/2) k UNIVERSITY OF MASSACHUSETTS AMHERST 11 Wrist Trajectory using Quaternions Visualizing Visualizing gestures gestures using
using aa wristband wristband and and an an armband armband equipped equipped with with IMUs IMUs UNIVERSITY OF MASSACHUSETTS AMHERST 12 Outline Introduction Challenges Data Collection using IMUs Data Processing Pipeline Evaluation Conclusion UNIVERSITY OF MASSACHUSETTS AMHERST 13 Segment
Extraction Gesture Classificatio n Feature Extraction Session Detection Segment q_s q_x q_y q_z Relative to elbow z z z 1 1
0 0.5 0.5 x 0 0.5 y 0.5 x 10 Wrist Trajectory Computation y 1 0 0.5 0.5
1 0 Trajectory Smoothing Peak Detectio n algorith m UNIVERSITY OF MASSACHUSETTS AMHERST x y 10 Wrist's Rest-point Computation Execute d on phone
14 Segment Extraction Feature Extraction Gesture Classificatio n Session Detection Orientation Independent Features A set of 34 spatio-temporal features Duration-based features (4) Gesture duration, time to raise arm, etc. Velocity-based features (6) Maximum wrist speed, etc.
Displacement-based features (6) Vertical displacement, XY displacement, etc. Angle-based features (18) Angle with the gravity, angular velocity, etc. UNIVERSITY OF MASSACHUSETTS AMHERST 15 Segment Extraction Feature Extraction Gesture Classificatio n Session
Detection Session Detection Layer Segment N-1 Session Segment N+1 Conditional Random Field Model Segment N features Gesture Recognition Layer Supervised Gesture Classification Feature Extraction Vertical Displacement Average Speed
UNIVERSITY OF MASSACHUSETTS AMHERST Angular twist Gesture duration 16 Outline Introduction Challenges Data Collection using IMUs Data Processing Pipeline Evaluation Conclusion UNIVERSITY OF MASSACHUSETTS AMHERST 17 Evaluation Dataset Dataset 28 hours of data from 15 volunteers 17 smoking sessions (369 puffs)
10 eating sessions (252 food bites) 6 drinking sessions UNIVERSITY OF MASSACHUSETTS AMHERST 18 Smoking Session Detection 16 12 17 8 4 15 Statistic 0 al t To
ns o i s s Se s n o si s Se ed t ec t e D Avg Std Dev Duration of smoking sessions
326.21 19.65 s Error in estimation 65.7 30.6 s Leave-one-session-out Cross-validation UNIVERSITY OF MASSACHUSETTS AMHERST 19 Smoking Gesture Recognition Mechanism Performance Metrics Accuracy Recall Precision
UNIVERSITY OF MASSACHUSETTS AMHERST CRF improves precision at a cost of slight drop in recall 20 User Study Recruited 4 subjects for 3 days. Used our smoking detection app developed for Android OS. UNIVERSITY OF MASSACHUSETTS AMHERST 21 User Study Day 1 Day 2 Day 3 Rarely
Fewer than missed 2 false any smoking positivessession. per day! UNIVERSITY OF MASSACHUSETTS AMHERST 22 Conclusion An algorithm to recognize hand-gestures using a wristband Demonstrated an application to detect smoking in real-time. Smartphones in conjunction with wearable accessories present a great platform to sense health-related behaviors like smoking, eating, and so on. Remarkable opportunity to create effective intervention strategies using smartphones. Software/Code available at: http://people.cs.umass.edu/~aparate/risq.html
UNIVERSITY OF MASSACHUSETTS AMHERST 23 Eating Gesture Recognition Mechanism Eating Sessions All data Recall Precision Recall Precision Bite-Counter 0.60 0.57
0.65 0.03 Random Forests 0.92 0.78 0.69 0.64 CRF N/A N/A 0.64 0.78 Eating gesture recognition Bite-Counter detects food bites when user explicitly indicates that eating session is in progress.
UNIVERSITY OF MASSACHUSETTS AMHERST 24 System Overhead Statistic Value Time for segmentation 92.34ms Time for feature extraction 79.88ms Time for CRF inference Memory 5.89ms 12-20MB Binary Size
1.7MB Measured on Samsung Galaxy Nexus UNIVERSITY OF MASSACHUSETTS AMHERST 25 Optimizing Performance Optimal cost for best performance Use a cost function during RF classifier training to assign penalty for missing a smoking gesture. High cost results in lower precision Low cost results in lower recall and low FPR UNIVERSITY OF MASSACHUSETTS AMHERST 26 3: Concurrent Activities
User is stationary User is walking Concurrent activities can modify the characteristic patterns of gestures UNIVERSITY OF MASSACHUSETTS AMHERST 27
End User Training Materials Acquisition Strategy Workflow Acquisition Resources and Analysis (ARA) Version 1.1, 07/07/2014 This provide a high-level view of the stages at which the PM uploads versions of the documents being coordinated to ECT, as well as the...
Myth Traditional and contemporary approaches ... This saga also involves a galaxy teeming with inhabited worlds held together by a collapsing galactic empire using hyperdrives (for long-distance transportation). ... One can speak of . . . the mythic elements within...
Public-Key Encryption (Asymmetrical) Conventional Encryption (Symmetrical) File encrypted with public key Locked file sent to recipient File decrypted with secure key File encrypted Locked file sent to recipient File decrypted with identical key The Future of Computing: Parallel Processing Deep...
ASN assignments. Transfers. Reverse DNS. Record Maintenance. ... Initially, specified recipient transfers were allowed only in the ARIN region. Some years after the first policy was implemented, the community created an inter-RIR transfer policy, which in essence is the same...
Cumberland Mills Elementary School Black History Presentation Dr. Linda Wilson-Jones Fayetteville State University Harriet Tubman Frederick Douglass Sojourner Truth Phillis Wheatley Jackie Robinson Hank Aaron Booker T. Washington Marian Anderson William Christopher Handy W.E.B. Du Bois Rosa Parks Martin Luther...
Psalm 8 (NKJV) 1 O Lord, our Lord, How excellent is Your name in all the earth, Who have set Your glory above the heavens! 2 Out of the mouth of babes and nursing infants, You have ordained strength, Because...