Asia CCS 2016 SmartwatchBased Keystroke Inference Attacks and
Asia. CCS 2016 Smartwatch-Based Keystroke Inference Attacks and Context-Aware Protection Mechanisms Anindya Maiti, Oscar Armbruster, Murtuza Jadliwala, Jibo He {axmaiti, oxarmbruster, murtuza. jadliwala, jibo. he}@wichita. edu
Keystrokes and Privacy • We type more than ever before. • It is important to protect from eavesdropping, because often the typed information is sensitive. Tuesday, November 3, 2020 2
Keystrokes and Privacy • Credit Card Information Tuesday, November 3, 2020 3
Keystrokes and Privacy • Tax Filing Tuesday, November 3, 2020 4
Keystrokes and Privacy • Emails/Messages And so on… Tuesday, November 3, 2020 5
Side-Channel Attacks Channel Type Common Limitation Berger et al. [CCS’ 06] Acoustic Marquardt et al. [CCS’ 11] Surface Vibration Ali et al. [Mobi. Com’ 15] Wi-Fi Channel State Information Change in position of either keyboard or eavesdropping device renders previous training data useless! Tuesday, November 3, 2020 6
Day 1 Day 2 Tuesday, November 3, 2020 7
Smartwatches • Wristwatch with functionality well beyond timekeeping. • Miniaturized computer. Tuesday, November 3, 2020 8
Behind the Scenes • Sensors • • Motion Microphone GPS Camera Ambient Light Temperature … Tuesday, November 3, 2020 9
Problems • We can’t turn off motion sensors. All applications have access to motion sensors by default. • Permissions allows control of access to data directly sensed by the sensors, but not to information that can be inferred indirectly from the sensors! Tuesday, November 3, 2020 10
Our Previous Work • “(Smart)Watch Your Taps” ISWC’ 15 Tuesday, November 3, 2020 11
New Target: QWERTY Keyboards Tuesday, November 3, 2020 12
We Asked Ourselves • Is it Possible to Infer What is Being Typed on the Keyboard Based on the Wrist Movements Observable by the Smartwatch Motion Sensors? Q M Linear Accelerometer Readings Tuesday, November 3, 2020 13
Dividing the Keyboard • Thus, an attacker can infer which side of the keyboard was pressed based on the level of activity during a key press. Tuesday, November 3, 2020 14
Further Observations • We can also categorize the direction of movement for the watch wearing hand. Assuming watch is worn on left hand Tuesday, November 3, 2020 15
Forming “Word-Profiles” • Word-profile for the word “boards”: bo oa ar rd ds RXR. RXL. LEL. LSL. LWL Tuesday, November 3, 2020 16
Learning Phase Tuesday, November 3, 2020 17
Attack Phase Tuesday, November 3, 2020 18
Evaluation • 25 participants aged between 19 -32 years. Samsung Gear Live smartwatch Anker A 7726121 Bluetooth keyboard • Matlab and Py. Brain Tuesday, November 3, 2020 19
Results: Basic Text Recovery • Dictionary: Ten sentences in List 6 of Harvard sentences • Typed: The same ten sentences above • L-R classifier misclassifications: 0% • N-E-S-W-O classifier misclassifications: ~5% • Word Recovery Error: Out of 48 words of four letters or more, only 3 were not recovered correctly (93. 75% success in recovery) Tuesday, November 3, 2020 20
Similarity Score • Closest Matching Word-Profile • Based on number of matching features • Frequency of Use • As in Dictionary Pool or English Literature Tuesday, November 3, 2020 21
Results: Contextual Dictionary • Participants typed a paragraph of 40 words (of length four or more) that appear in a National Public Radio (NPR) news article on Greece debt crisis, and this experiment simulates eavesdropping on a reporter typing the NPR news article. • The dictionary is formed with words that appear in six other news articles related to Greece debt crisis, that were published a week before the target article. Tuesday, November 3, 2020 22
Results: Contextual Dictionary: Percentage of words recovered per participant, presented in descending order of typing speed of the participants. Tuesday, November 3, 2020 23
Results: Typing Behavior and Speed • We observed that in many instances participants did not follow our assumed layout. Some of the participants frequently used their left hand to press a key on the right side of the keyboard, and vice versa. • We also found that participant who typed slower, were less likely to follow the left and right division of the keyboard. Tuesday, November 3, 2020 24
Results: Typing Behavior and Speed Contextual Dictionary: Percentage of words recovered per participant, presented in descending order of typing speed of the participants. Tuesday, November 3, 2020 25
Results: Large Dictionary • 38 English words typed by participants. • English dictionary of 60, 000 words, sorted by frequency of use in English literature. • Problem of colliding word-profiles: Show: LXR. RXL Sums: LXR. RXL Tuesday, November 3, 2020 26
Results: Large Dictionary A comparison of accuracy of our attack with Marquardt et al. [surface vibration] and Berger et al. [acoustic emanation]. Note that in spite of not having wrist movement information available from the non-watch-wearing hand, our results are roughly comparable for a large (60, 000 words) dictionary. Tuesday, November 3, 2020 27
Limitations • Ambient Wrist Movement • Left and Right Handedness. But… • Inferring Non-Dictionary Text Tuesday, November 3, 2020 28
Smart Mitigation • Access control over seemingly innocuous sensors required. • But should not be the old-fashioned way. • Must be context-aware in order to automatically manage sensor permissions, without having the user to manually change these settings repetitively. Tuesday, November 3, 2020 29
Proposed Protection Framework Tuesday, November 3, 2020 30
r. TAD Parameters Tuesday, November 3, 2020 31
Motion Sensor Access. Controller (MSAC) • Complete Blocking • Reduced Sampling Rate • Random Out of Order Blocks Tuesday, November 3, 2020 32
r. TAD Evaluation: High Sensitivity Tuesday, November 3, 2020 33
r. TAD Evaluation: Low Sensitivity Tuesday, November 3, 2020 34
r. TAD Evaluation Results Tuesday, November 3, 2020 35
Conclusion • A new keystroke inference attack which utilizes wrist-motion data gathered from a smartwatch as side-channel information. • A smart protection framework to detect typing activity and automatically regulate sensor access, aimed to improve privacy without degrading utility of the device. • Thank You! Tuesday, November 3, 2020 http: //sprite. cs. wichita. edu/ 36
- Slides: 36