Do You Hear What I Hear Fingerprinting Smart
- Slides: 25
Do You Hear What I Hear? Fingerprinting Smart Devices Through Embedded Acoustic Components Anupam Das (UIUC), Nikita Borisov (UIUC), Matthew Caesar (UIUC) CCS 2014 1 May 24, 2021
Smartphone Usage • How many people use smartphones? Source: Business Insider Source: Gartner • Shipment of smartphone is increasing every year. Tablet Mobile Phone 2, 250, 000 2, 000 Thousands of Units 75% of the mobile phones are smartphones and highlyfeatured phones PC(Desktop & Notebook) 1, 750, 000 1, 500, 000 1, 250, 000 1, 000 750, 000 500, 000 250, 000 0 2012 2 2013 Year 2014 2015 May 24, 2021
A Closer Look at Smartphones • Today, smartphones come with a wide range of sensors. All of which are useful for a variety of tasks. • Motion detection • Gesture detection • Audio Genre detection • Location detection • Interaction with nearby devices • Compass • However, sensors could also be potential source for unique fingerprints. 3 May 24, 2021
Why Fingerprint Smartphones? Smartphones can be fingerprinted for: • Targeted Advertisement • Secondary Authentication factor The main drawback for software based approaches is the source ofapproaches: the fingerprints are not too stable. Øthat Software based • Browser based features, Cookies Moreover, device IDs are not. Device alwaysdrivers usable. UDID for • Different Firmware and Apple devices has been removed since May 1, 2013 and for an based Androidapproaches: Device requires explicit permission. ØIMEI Hardware • Clock Skew rate • Radio Transmitter • Accelerometer 4 May 24, 2021
Our Goal We focus on fingerprinting smart devices through their embedded speakers and microphones. • Scenario 1: External attacker locally present Requires: Deployed microphones 1. Attacker records audio signal from distance 2. Create a fingerprint of the recorded audio and link the fingerprint to a unique smartphone • Scenario 2: Stealthy App 2. Audio signal 1. P lay au dio o App 3. R udi a rd eco 4. Extract Fingerprint 5 Need access to only: 1. Microphone 2. Internet connection May 24, 2021
Top Android Permissions Top 17 Android permissions out of 173 total permissions 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 Storage : modify/delete USB storage and SD card contents Network communication : full Internet access Network communication : view network state System tools : prevent device from sleeping Phone calls : read phone state and identity Hardware controls : control vibrator System tools : automatically start at boot Network communication : view Wi−Fi state Your location : fine (GPS) location Your location : coarse (network−based) location System tools : retrieve running applications Your personal information : read contact data Your messages : read SMS or MMS Your messages : receive SMS Hardware controls : take pictures and videos Hardware controls : record audio System tools : modify global system settings Source: Mario Frank, Ben Dong, Adrienne Porter Felt, Dawn Song. Mining Permission Request Patterns from Android and Facebook Applications, ICDM, 2012 6 May 24, 2021
Real World Apps There are popular apps that people use that require similar permissions. For example: My Talking Tom 4, 405, 876 7 May 24, 2021
Closer Look at Microphones MEMS microphone: Sound Wave Mechanical Energy Capacitive Change Voltage Change The sensitivity of the microphone depends on how well the diaphragm deflects to acoustic pressure Imperfections can arise due to: 1. Slight variations in the chemical composition of components from one batch to the next 2. Wear and tear in the manufacturing machines 3. Changes in temperature and humidity Source: STMicroelectronics 8 May 24, 2021
Closer Look at Speakers Basic functionality of a speaker : Electrical current Magnetic field around voice coil Mechanical Energy Sound Wave The frequency of the sound wave produced is dictated by the rate at which the voice coil moves. 9 Source: Center Point Audio May 24, 2021
Experimental Setup Maker Model # Apple i. Phone 5 1 HTC Nexus One 14 Nexus S 8 Galaxy S 3 3 Galaxy S 4 10 Motorola Droid A 855 15 Sony Ericsson W 518 1 Samsung To emulate an attacker we use a laptop Total 52 Audio Type Description Variations Instrumental Musical instruments playing together 4 Human Speech Small segments of human speech 4 Song Combination of human voice & instrumental sound 3 10 May 24, 2021
Fingerprinting Acoustic Components 1. Fingerprinting Speakers This scenario is suited for the case where the attacker has deployed nearby microphones to capture audio signal (e. g. , ringtone) from user device. 11 May 24, 2021
Fingerprinting Acoustic Components 2. Fingerprinting Microphones 3. Fingerprinting Both Speakers and Microphones In these scenarios the attacker has stealthy convinced the user to provide access to microphone. 12 May 24, 2021
Acoustic Features We investigate a total of 15 acoustics features # Feature Dimension 1 Root Mean Square 1 2 Zero Crossing Rate 1 3 Low-Energy-Rate 1 4 Spectral Centroid 1 5 Spectral Entropy 1 6 Spectral Irregularity 1 7 Spectral Spread 1 8 Spectral Skewness 1 9 Spectral Kurtosis 1 10 Spectral Rolloff 1 11 Spectral Brightness 1 12 Spectral Flatness 1 13 Mel-Frequency-Cepstral-Coefficients 13 14 Chromagram 12 15 Tonal Centroid 6 13 May 24, 2021
Feature Selection Are there any dominant features? Identifying the dominant feature set benefits us in two ways: 1. Less computation (potentially shifting the feature extraction component into mobile devices) 2. Improve accuracy as there might be conflicting features Feature Reduction (Dimensionality reduction) Feature Extraction [new features=function(old features)] e. g. , PCA, LDA Feature Selection [subset of old features] Feature selection is preferable to feature extraction when dimensionality and numerical transformations of the features are inappropriate. 14 May 24, 2021
Sequential Feature Selection We adopt a well-known machine learning technique called sequential forward selection (SFS). It is a greedy approach where features are added only if it increases accuracy. We found Mel-Frequency-Cepstral-Coefficients (MFCCs) as the dominating features. MFCCs: • Compactly represent the spectrum along the nonlinear mel-scale of frequency. • Distinguish the low and fast varying spectral envelopes of the signal. 15 May 24, 2021
Evaluation Algorithms & Metrics We evaluate using two classification algorithm: • k-NN: k-nearest neighbors • GMM: Gaussian Mixture Model TP: True Positive FP: False Positive FN: False Negative 16 May 24, 2021
Different Make & Model Sets We consider one set from each make & model, giving us a total of 7 different sets. 1 - F 1_score (%) Speaker 10 9 8 7 6 5 4 3 2 1 0 Microphone Speaker & Microphone Maker Model # Apple i. Phone 5 1 HTC Nexus One 14 Nexus S 8 Galaxy S 3 3 Galaxy S 4 10 Motorola Droid A 855 15 Sony Ericsson W 518 1 Samsung 2. 599999 999 0 0 0 Instrument 0 0 Human Speech 0 0 Song 0 Total 52 Genre So we can accurately distinguish smartphones of different make and model. 17 May 24, 2021
Same Make & Model Sets We consider 15 Motorola Droid A 855 handsets. 1 - F 1_score (%) Speaker 10 9 8 7 6 5 4 3 2 1 0 Microphone Speaker & Microphone 5. 5 3. 9000000 01 4. 7 1. 7 0 Instrument 0 0 0 Human Speech Song Maker Model # Apple i. Phone 5 1 HTC Nexus One 14 Nexus S 8 Galaxy S 3 3 Galaxy S 4 10 Motorola Droid A 855 15 Sony Ericsson W 518 1 Samsung Total 52 Genre Fingerprinting both the speaker and microphone seems to provide better results. 18 May 24, 2021
Heterogeneous Smartphones We develop an Android App to collect audio samples from 50 Android sets. In this case the audio clip is played and recorded on the phone set. Maker Model 10 9 1 - F 1_score (%) 8 Apple i. Phone 5 1 HTC Nexus One 14 Nexus S 8 Galaxy S 3 3 Galaxy S 4 10 Motorola Droid A 855 15 Sony Ericsson W 518 1 7 6 Samsung 5 4 3 2 1. 7 1 0. 7000003 0 0 Instrument Human Speech Genre Song We were able to correctly classify ~98% of the recorded audio clips. 19 # Total 52 May 24, 2021
Sensitivity Analysis We analyze how different factors impact our fingerprinting accuracy to better understand the feasibility of our approach. We analysis the following factors: • Sampling rate • Training set size • Distance between speaker and microphone • Ambient background noise We only consider same make and model handsets for the following set of experiments (as this is a harder problem) 20 May 24, 2021
Impact of Distance We vary the distance between the speaker and microphone. k-NN GMM 100 90 80 We used an Audio. Technica ATR-6550 shotgun microphone (~$45) for this experiment F 1 Score 70 60 50 40 30 20 10 0 0. 1 1 2 3 4 Distance (in meters) 5 After a distance of 2 meters the accuracy tend to go down at a faster rate. 21 May 24, 2021
Impact of Background Noise We emulate four types of ambient background noise using external speakers. Played audio from the instrument category Environment 2 Meters SNR (d. B) GMM Shopping Mall 15. 84 (16%) 94. 2 Restaurant 17. 77 (13%) 91. 6 City Park 15. 43 (17%) 94. 6 Airport Gate 14. 92 (18%) 93. 9 Even with relative amount of background noise we can successfully fingerprint smartphones. 22 May 24, 2021
Discussions Providing a counter measure against such a side channel attack is still a open research question. One could attempt to shift the frequencies of the audio signal in a way so as to not deter the quality of the audio stream too much. But a thorough analysis of the impact of such a counter measure is required to fully understand its feasibility as audio streams are used for legitimate purposes too. 23 May 24, 2021
Conclusion • We see that it is possible to fingerprint smartphones through embedded microphones and speakers. • We were able to accurately attribute ~98% of all recorded audio excerpts from 50 different Android devices. So, the next time you talk to Tom keep in mind whether you are giving Tom a fingerprint of your device. More details of the project is available at the following linkhttp: //web. engr. illinois. edu/~das 17/Smartphone. Fingerprint. html 24 May 24, 2021
The End 25 May 24, 2021
- Wisdom and goodness to the vile seem vile
- Interserf
- It's not how smart you are it's how you are smart
- Let him who has ears
- Hear ye definition
- Perceive family words
- You hear me but are you listening
- When you hear the word scientist what comes to mind
- I can hear you quite well. you not shout
- You hear: el café you say: yo ceno en el café.
- What do you think of when you hear
- Lausd fingerprinting
- 3 basic principles of fingerprints
- Dna fingerprinting minilab answers
- Dna fingerprinting minilab answers
- Biorad dna fingerprinting
- Vntr vs str in dna fingerprinting
- Who ate the cheese
- Dna fingerprinting rflp
- Miami dade county public schools contractor badge
- Dr. henry p. de forest
- Fingerprint minutiae
- Dhcp fingerprint
- Golden rice zanichelli
- Dna fingerprinting lesson plan
- Fingerprinting