Face Revelio A Face Liveness Detection System for

Face. Revelio: A Face Liveness Detection System for Smartphones with a Single Front Camera Habiba Farrukh, Reham Mohamed Aburas, Siyuan Cao, He Wang Purdue University

User Authentication

2 D Spoofing Attacks Attacker 2 D photos and video replay attacks

Liveness Detection Live human subject Spoofing attempt

Texture Analysis Image Degradation Diffusion speed Wen et al. [2015], Patel et al. [2016] Kim et al. [2015], Aziz et al. [2017] Live Spoof Diffusion Rely on ideal lighting conditions, photo quality Scale Texture Difference of Gaussian (Do. G) Peng et al. [2018] Tan et al. [2014] Live Spoof

User Interaction Time constrained, difficult for elderly usage and emergency cases Eye Blinking Facial Expressions Jee et al. [2006], Pan et al. [2011] Kollreider et al. [2007], Tang et al. [2018] Phone movement Chen et al. [2014], Li et al. [2016], Li et al. [2019]

We want a face liveness detection system, which is accurate, environment independent and does not require user interaction or specialized hardware.

Use screen as a light source to reconstruct 3 D face

We use this core idea to design Face. Revelio: A Face Liveness Detection System for Smartphones with Single Front Camera

Images under different illuminations Stereo Image Recovery 3 D reconstructed face Photometric Stereo Decision model Recorded reflections from the face Human or Spoof?

Photometric Stereo Images Light directions (scaled by intensity) Surface normals (scaled by albedo) M L S Number of images = Number of pixels M = U∑VT L = U√∑A and S = A-1√∑VT * Both L and S are now unknown! This is a matrix factorization problem.

Light source 3 D reconstructed face via photometric stereo

How can we defend replay attacks?

Light Passcode . . . t light passcode p 1 p 2 p 3 p 4 … … t p 1 p 2 p 3 p 4

Stereo Image Recovery mixture of reflections of the four patterns 4 number of frames, f separate reflections from each pattern number of pixels in each frame, n Holds true if the camera response is linear to the light reflected from the face light passcode (fx 4)

Camera Model linearization white balancing demosaicing raw sensor data final video frames from the camera Linear camera model four stereo images color space correction Gamma calibration non-zero black level of camera sensor final image apply inverse gamma calibration

What if ambient light is present?

Zero-mean and Orthogonal Patterns ambient lighting independent, zero-mean and orthogonal p 1 p 2 p 3 p 4 … … Gram Schmidt Process … … … t t

How can we make our system more comfortable for users?

Low Pass Filtering FFT before and after Gram Schmidt Process

Recovered stereo images Four images representing face illuminated from four different light sources region with a higher intensity value compared to the intensity in the same region in all other images

Normal Map 3 x 3 matrix ambiguity X L … … … = * … Num of Pixels Template mesh for finding A

3 D reconstruction from human subjects Person A Person B Person C 3 D reconstruction from photographs of the human subjects Person A Person B Person C

Classification Binary Classification Reconstruction from human Reconstruction from spoof Deep Siamese Neural Network 1 0 One shot learning Sample human depth map & reconstruction from human Sample human depth map & reconstruction from spoof 1 0

Face. Revelio

Evaluation

Implementation & Data Collection • Prototype on Samsung S 10 • Experiment with 30 volunteers • Photo and Video Replay attacks • Natural daylight, dark and indoor light settings • Various screen-to-face distances and face orientations

Overall Performance § > 99. 3% detection accuracy in various lighting conditions. § Replay attacks detected with an EER of 0. 15%. Performance in various lighting conditions Distribution of correlation between video and light passcode video human

Robustness Performance with various screen to face distances Performance with different face orientations

Computation Cost § Passcode duration 1 s. § Total time cost < 2 s.

Conclusion By using smartphone’s front camera only, we can perform secure face liveness detection which is not environment dependent or requires any user interaction Thank you!

Thank you!