Robust RealTime Face Detection PAUL VIOLA MICHAEL J

Robust Real-Time Face Detection PAUL VIOLA MICHAEL J. JONES Microsoft Research Mitsubishi Electric Research Lab.

OUTLINE INTRODUCTION RELATED WORK FEATURES Adaptive Boosting Cascade of Classifier RESULTS CONCLUSIONS

INTRODUCTION FEATURES This face detection system is most clearly distinguished from previous approaches in its ability to detect faces extremely fast. Adaptive Boosting Cascade of Classifier RESULTS Operating on 384 by 288 pixel images, faces are detected at 15 frames per second on a conventional 700 MHZ Intel Pentium III. Achieves higher frame rates by working only with the information present in a single gray scale image.

INTRODUCTION 3 main contributions of this system. FEATURES Adaptive Boosting Cascade of Classifier RESULTS A new image representation called “integral image” that allows very fast feature evaluation. Selecting a small number of important features from a huge library of potential features using Ada. Boost (Adaptive Boosting). Combining classifiers in a cascade structure which increases the speed of the detector by focusing on promising regions of the image.

FEATURES INTRODUCTION Integral image FEATURES Adaptive Boosting Cascade of Classifier RESULTS The value of the integral image at point (x, y) is the sum of all the pixels above and to the left.

FEATURES INTRODUCTION Rectangle features FEATURES Adaptive Boosting Cascade of Classifier RESULTS The sum of the pixels which lie within the white rectangle are subtracted from the sum of pixels in the grey rectangles.

FEATURES INTRODUCTION FEATURES Adaptive Boosting Cascade of Classifier RESULTS Rectangle features

FEATURES INTRODUCTION Scanning the image FEATURES Adaptive Boosting Cascade of Classifier RESULTS The detector is scanned across the input image at multiple scales and locations. Our detector scans the input at many scales; starting at the base size of 24 × 24 pixels, a 384 by 288 pixel image is scanned at 12 scales each a factor of 1. 25 larger than the last. The detector is also scanned across location. Subsequent locations are obtained by shifting the window some number of pixels △.

Adaptive Boosting INTRODUCTION FEATURES Adaptive Boosting Cascade of Classifier RESULTS Given a feature set and a training set of positive and negative images, adaptive boosting could be used to learn a classification function. Each feature is used as a weak classifier. A weak classifier ( h (x, f, p, θ)) thus consists of a feature ( f ), a threshold (θ) and a polarity (p) indicating the direction of the inequality:

Adaptive Boosting INTRODUCTION Algorithm FEATURES Adaptive Boosting Given example images (x 1, y 1), . . . , (xn , yn ) where yi = 0, 1 for negative and positive examples respectively. Cascade of Classifier RESULTS Initialize weights for yi = 0, 1 respectively, where m and l are the number of negatives and positives respectively.

Adaptive Boosting INTRODUCTION FEATURES Adaptive Boosting Cascade of Classifier RESULTS Algorithm

Adaptive Boosting INTRODUCTION The first two features selected by Ada. Boost. FEATURES Adaptive Boosting Cascade of Classifier RESULTS The first feature measures the difference in intensity between the region of the eyes and a region across the upper cheeks. The second feature compares the intensities in the eye regions to the intensity across the bridge of the nose.

Cascade of Classifier INTRODUCTION FEATURES Adaptive Boosting Cascade of Classifier RESULTS Within an image, most sub-images are non-face instances. Use smaller and efficient classifiers to reject many negative examples at early stage while detecting almost all the positive instances. Simpler classifiers are used to reject the majority of sub-windows. More complex classifiers are used at later stage to examine difficult cases.

Cascade of Classifier INTRODUCTION FEATURES Adaptive Boosting Cascade of Classifier RESULTS

Cascade of Classifier INTRODUCTION FEATURES • Given a trained cascade of classifiers, the false positive rate and detection rate at each stage are: Adaptive Boosting Cascade of Classifier RESULTS • And 3 parameters should decide in the optimization framework: • the number of classifier stages • the number of features, ni , of each stage • the threshold of each stage

Cascade of Classifier INTRODUCTION FEATURES Adaptive Boosting Cascade of Classifier RESULTS Algorithm

RESULTS INTRODUCTION FEATURES Adaptive Boosting Cascade of Classifier RESULTS

RESULTS INTRODUCTION FEATURES Adaptive Boosting Cascade of Classifier RESULTS

RESULTS INTRODUCTION FEATURES Adaptive Boosting Cascade of Classifier RESULTS

RESULTS INTRODUCTION FEATURES Adaptive Boosting Cascade of Classifier RESULTS FAILURE MODES • Informal observation suggests that the face detector can detect faces that are tilted up to about ± 15 degrees in plane and about ± 45 degrees out of plane. • We have also noticed that harsh backlighting in which the faces are very dark while the background is relatively light sometimes causes failures. • Our face detector fails on significantly occluded faces.