Activity Recognition Using Cell Phone Accelerometers Jennifer Kwapisz

Activity Recognition Using Cell Phone Accelerometers Jennifer Kwapisz, Gary Weiss, Samuel Moore Department of Computer & Info. Science Fordham University July 25, 2010 Sensor. KDD 2010 1

We are Interested in WISDM: WIreless Sensor Data Mining Powerful portable wireless devices are becoming common and are filled with sensors n Smart phones: Android phones, i. Phone n Music players: i. Pod Touch n n Sensors on smart phones include: n Microphone, camera, light sensor, proximity sensor, temperature sensor, GPS, compass, accelerometer July 25, 2010 Sensor. KDD 2010 2

Accelerometer-Based Activity Recognition The Problem: use accelerometer data to determine a user’s activity n Activities include: n Walking and jogging n Sitting and standing n Ascending and descending stairs n More activities to be added in future work n July 25, 2010 Sensor. KDD 2010 3

Applications of Activity Recognition n Health Applications Generate activity profile to monitor overall type and quantity of activity n Parents can use it to monitor their children n Can be used to monitor the elderly n n Make the device context-sensitive Cell phone sends all calls to voice mail when jogging n Adjust music based on the activity n n Broadcast (Facebook) your every activity July 25, 2010 Sensor. KDD 2010 4

Our WISDM Platform n Platform based on Android cell phones Android is Google’s open source mobile computing OS n Easy to program, free, will have a large market share n n Unlike most other work on activity recognition: No specialized equipment n Single device naturally placed on body (in pocket) n July 25, 2010 Sensor. KDD 2010 5

Our WISDM Platform n Current research was conducted off-line n Data was collected and later analyzed off-line In future our platform will operate in real-time n In June we released real-time sensor data collection app to Android marketplace n n Currently collects accelerometer and GPS data July 25, 2010 Sensor. KDD 2010 6

Accelerometers n Included in most smart phones & other devices All Android phones, i. Pod Touches, etc. n Tri-axial accelerometers that measure 3 dimensions n n Initially included for screen rotation and advanced game play July 25, 2010 Sensor. KDD 2010 7

Examples of Raw Data Next few slides show data for one user over a few seconds for various activities n Cell phone is in user’s pocket n Earth’s gravity is registered as acceleration n Acceleration values relative to axes of the device, not Earth n n In theory we can correct this given that we can determine orientation of the device July 25, 2010 Sensor. KDD 2010 8

Standing July 25, 2010 Sensor. KDD 2010 9

Sitting July 25, 2010 Sensor. KDD 2010 10

Walking July 25, 2010 Sensor. KDD 2010 11

Jogging July 25, 2010 Sensor. KDD 2010 12

Descending Stairs July 25, 2010 Sensor. KDD 2010 13

Ascending Stairs July 25, 2010 Sensor. KDD 2010 14

Data Collection Procedure n User’s move through a specific course Perform various activities for specific times n Data collected using Android phones n Activities labeled using our Android app n Data collection procedure approved by Fordham Institutional Review Board (IRB) n Collected data from 29 users n July 25, 2010 Sensor. KDD 2010 15

Data Preprocessing Need to convert time series data into examples n Use a 10 second example duration (i. e. , window) n n n 3 acceleration values every 50 ms (600 total values) Generate 43 total features Ave. acceleration each axis (3) n Standard deviation each axis (3) n Binned/histogram distribution for each axis (30) n Time between peaks (3) Sensor. KDD 2010 July 25, n 2010 Ave. resultant acceleration (1) n 16

Final Data Set July 25, 2010 Sensor. KDD 2010 17

Data Mining Step n Utilized three WEKA learning methods Decision Tree (J 48) n Logistic Regression n Neural Network n n Results reported using 10 -fold cross validation July 25, 2010 Sensor. KDD 2010 18

Summary Results July 25, 2010 Sensor. KDD 2010 19

J 48 Confusion Matrix Predicted Class A c t u a l C l a s s July 25, 2010 Walk Jog Up Down Sit Stand Walk 1513 14 72 82 2 0 Jog 16 1275 16 12 1 1 Up 88 23 323 107 2 2 Down 99 13 92 258 1 2 Sit 4 0 2 3 270 3 Stand 4 1 2 7 1 208 Sensor. KDD 2010 20

Conclusions n Able to identify activities with good accuracy Hard to differentiate between ascending and descending stairs. To limited degree also looks like walking. n Can accomplish this with a cell phone placed naturally in pocket n Accomplished with simple features and standard data mining methods n July 25, 2010 Sensor. KDD 2010 21

Related Work n At least a dozen papers on activity recognition using multiple sensors, mainly accelerometers n n n Activity recognition also done via computer vision Actigraphy uses devices to study movement n n Typically studies only 10 -20 users Used by psychologists to study sleep disorders, ADD A few recent efforts use cell phones n n Yang (2009) used Nokia N 95 and 4 users Brezmes (2009) used Nokia N 95 with real-time recognition n July 25, 2010 One model per user (requires labeled data from each user) Sensor. KDD 2010 22

Future Work Add more activities and users n Add more sophisticated features n Try time-series based learning methods n Generate results in real time n Deploy higher level applications: activity profiler n July 25, 2010 Sensor. KDD 2010 23

Other WISDM Research n Cell Phone-Based Biometric identification 1 n n Same accelerometer data and same generated features but added 7 users (36 in total) If we group all of the test examples from one cell phone and apply majority voting, achieve 100% accuracy Can be used for security or automatic personalization Interested in GPS spatio-temporal data mining 1 Kwapisz, Weiss, and Moore, Cell-Phone Based Biometric Identification, Proceedings of the IEEE 4 th International Conference on Biometrics: Theory, Applications, and Systems (BTAS-10), September 2010. July 25, 2010 Sensor. KDD 2010 24

Thank You Questions? July 25, 2010 Sensor. KDD 2010 25