Pattern Listener Cracking Android Pattern Lock Using Acoustic

Pattern. Listener: Cracking Android Pattern Lock Using Acoustic Signals Man Zhou 1, Qian Wang 1, Jingxiao Yang 1, Qi Li 2, Feng Xiao 1, Zhibo Wang 1, Xiaofeng Chen 3 1 School of Cyber Science and Engineering, Wuhan University, China 2 Graduate School at Shenzhen & Department of Computer Science, Tsinghua University, China 3 School of Cyber Engineering, Xidian University, China {zhouman, qianwang, yangjingxiao, f 3 i, zbwang}@whu. edu. cn, qi. li@sz. tsinghua. edu. cn, xfchen@xidian. edu. cn

Outline • Motivation • Patternlistener • Evaluation • Discussion • Conclusion

Wide Usage of Pattern Lock • 40% of the participants use pattern lock as the screen lock • 33% of other participants often use pattern lock for identity authentication on apps Screen unlock App unlock Guixin Ye, Zhanyong Tang, Dingyi Fang, Xiaojiang Chen, Kwang In Kim, Ben Taylor, and Zheng Wang. 2017. Cracking Android pattern lock in five attempts. In Proc. of NDSS.

Existing Security Mechanisms • Many security mechanisms have been developed to ensure that the screen of devices cannot be captured by other apps Sandbox Trust. Zone

Shared Hardware Resources • Apps can still access certain shared hardware resources to generate side channel information, which may allow an attacker to infer unlock pattern by using such information

Side Channel Attacks to PINs • Emiliano et al. infer the location of taps on screens using motion sensor data with a machine learning approach • Simon et al. leverage the camera to estimate orientation changes and correlate the changes to the position of the tapped digitals • Emiliano Miluzzo, Alexander Varshavsky, Suhrid Balakrishnan, and Romit Roy Choudhury. 2012. Tapprints: your finger taps have fingerprints. In Proc. Of Mobi. Sys. ACM, 323– 336. • Laurent Simon and Ross Anderson. 2013. Pin skimmer: Inferring pins through the camera and microphone. In Proc. of SPSM. ACM, 67– 78.

Attacks to Pattern Lock • Aviv et al. analyze oily residual left on screen to infer unlock pattern[3] • Zhang et al. infer the pattern by analyzing Wi. Fi hotspots signals[4] • Ye et al. crack pattern lock by using footage that records the victim’s fingertip motions[5] [3] Adam J Aviv, Katherine L Gibson, Evan Mossop, Matt Blaze, and Jonathan M Smith. 2010. Smudge Attacks on Smartphone Touch Screens. Woot 10 (2010), 1– 7. [4] Jie Zhang, Xiaolong Zheng, Zhanyong Tang, Tianzhang Xing, Xiaojiang Chen, Dingyi Fang, Rong Li, Xiaoqing Gong, and Feng Chen. 2016. Privacy leakage in mobile sensing: Your unlock passwords can be leaked through wireless hotspot functionality. Mobile Information Systems (2016). [5] Guixin Ye, Zhanyong Tang, Dingyi Fang, Xiaojiang Chen, Kwang In Kim, Ben Taylor, and Zheng Wang. 2017. Cracking Android pattern lock in five attempts. In Proc. of NDSS.

Key Idea • Pattern. Listener, a novel and robust attack to crack Android pattern lock by leveraging imperceptible acoustic signals

Overview of Pattern. Listener

Unlocking Detection • Capture the broadcasted state transition information • Detect the raise actions, which usually indicated screen unlock

Audio Capturing •

Audio Preprocessing • Pre-process the sound signals and obtain the components corresponding fingertip motions • Extract the envelopes of recorded signals to capture the signals reflected by the fingertip • Leverage the coherent detection to demodulate signals and obtain the cophase(C)/orthogonal(O) components

Signal Segmentation • Segment the C/O components into fragments corresponding to each line included in the pattern • The acoustic signal reflected by the fingertip is relatively stable at a turning point

Measuring Phase Changes • Observe that C/O waveforms are similar to the sine and cosine curves that the C/O trace forms circle shapes • Calculate the accumulated degrees of rotation of each node to obtain the values of the phase changes

Relative Movement Measurement • Convert the phase changes into the changes of the path lengths of the signals reflected by the fingertip

Inferring Pattern Lines • Compute eight features vectors for each node in the pattern grid according to the distances to other nodes • Compare the extracted features vector of path changes with the ground truth and calculate the similarity • There are two cases of the path computation, which are similar

Candidate Patterns Generation • Map all candidates of the lines into the pattern grid to generate the patterns tree • The weight of each edge indicates the similarity of each candidate • The top five patterns with the highest similarities are selected as the candidate patterns

Implementation & Evaluation • Mare ware on devices: SAMSUNG C 9 Pro and HUAWEI P 9 Plus • Data analysis on server: 2. 9 GHz CPU and 8 GB memory • Pattern data set: 130 collected patterns • The sampling rate of signals: 48 k. Hz

Overall Accuracy

Impact of Pattern Complexity

Impact of Gesture

Impact of Drawing Speed

Impact of Surrounding Objects

Impact of Smartphones and Noise

Countermeasures • Prevent the usage of microphone in the background when sensitive tasks are executed(e. g. drawing pattern) • Notify users(e. g. a pop-up notification) during the usage of microphone in the background • Restrict the frequency range that the sound hardware(i. e. speakers and microphones) supports • Randomize sizes and layouts of the pattern grid

Conclusion • Uncover a new vulnerability of pattern lock by leveraging speakers and microphones of mobile devices • Propose Pattern. Listener, a novel attack to crack android pattern lock of a large number of mobile devices by utilizing imperceptible acoustic signals • Pattern. Listener achieved high accuracy in reconstructing the unlock pattern on smartphones with various practical considerations