Preprocessing Overview Preprocessing to enhance recognition performance in

  • Slides: 25
Download presentation
Preprocessing Overview • Preprocessing to enhance recognition performance in the presence of: - Illumination

Preprocessing Overview • Preprocessing to enhance recognition performance in the presence of: - Illumination variations - Pose/expression/scale variations - Resolution enhancement (deblurring) • Stand-alone recognition system • Preprocessing/recognition results - Face Recognition Grand Challenge (NIST)

Face Recognition Problem: Current state-of-the-art face recognition systems degrade significantly in performance due to

Face Recognition Problem: Current state-of-the-art face recognition systems degrade significantly in performance due to variations in pose, illumination, and blurring. Solution: IMAGE CAPTURE PREPROCESSING RESTORATION/ ENHANCEMENT FACE RECOGNITION SYSTEM POSE CORRECTION due to mismatch in facial position, facial expression and scale ILLUMINATION CORRECTION due to mismatch in lighting conditions in both indoor and outdoor environments DEBLURRING due to mismatch in camera focus, camera lenses, camera resolution and motion blur

Highlights of the approach • No a priori information with regards to pose orientation,

Highlights of the approach • No a priori information with regards to pose orientation, camera parameters, etc • No laser scanned images for 3 D reconstruction • No manual detection of feature points • Preprocessing & Stand-Alone Recognition

Principle • Find a function which maps a given test (probe) image into the

Principle • Find a function which maps a given test (probe) image into the correct train (gallery) image • Approach where M is the number of training images • Select that is maximally bijective

Recognition Principle • A function ’f ‘ is found which maps points in the

Recognition Principle • A function ’f ‘ is found which maps points in the test (probe) to equivalent points in the train (gallery) domain (X) f range(X) where X = Test image (domain) Y X Y = Train image (co-domain) = Bijective function mapping X Y Test Train One to One and Onto (bijection)

Inverse Estimation • A function ’g ‘ is found which maps points in the

Inverse Estimation • A function ’g ‘ is found which maps points in the train (gallery) to equivalent points in the test (probe) domain (Y) g range(Y) where Y = Train image (domain) X = Test image (co-domain) X Y = Bijective function mapping Y X Test Train

Measure of Bijectivity X f Y Blue, Green, Cyan Red Partition X where n

Measure of Bijectivity X f Y Blue, Green, Cyan Red Partition X where n is the total number of distinct blocks in X

Measure of Bijectivity X g Y Blue, Green, Cyan Red Partition Y where p

Measure of Bijectivity X g Y Blue, Green, Cyan Red Partition Y where p is the total number of distinct blocks in Y

Measure of Bijectivity The Bijectivity score is given by: = Forward (test train) =

Measure of Bijectivity The Bijectivity score is given by: = Forward (test train) = Backward (train test) = Adaptive Forward (test train) = Adaptive Backward (train test) = constants and

Mapping Properties Train Image No. 1

Mapping Properties Train Image No. 1

Preprocessing Example

Preprocessing Example

Preprocessing Performance

Preprocessing Performance

Face recognition performance Metric used One-to-None Mapping Yale Subset-I Block Size-8 x 8; Search-56

Face recognition performance Metric used One-to-None Mapping Yale Subset-I Block Size-8 x 8; Search-56 x 56 Yale Subset-II Block Size-8 x 8; Search-88 x 88 Exhaustive Search 100 % 80 % Exhaustive Search with Constraints 100 % 83. 33 % Fast Search 98 % 83. 33 % Table 4. 2 Yale Results-Stand Alone Recognition-II

Face recognition performance PIE Subset- Exhaustive Search Block Size-8 x 8; Search-72 x 72

Face recognition performance PIE Subset- Exhaustive Search Block Size-8 x 8; Search-72 x 72 One-to-None Mapping One-to-One Mapping Stand Alone Recognition Performance 91. 18 % 95. 59 % Table 4. 4 PIE Database-Stand Alone Recognition PCA-based Approach recognition accuracy: 5. 88 %

Preprocessing for Illumination Correction Training Test Preprocessed

Preprocessing for Illumination Correction Training Test Preprocessed

Preprocessing for Illumination Correction • Algorithm based on image adaptive least squares illumination correction

Preprocessing for Illumination Correction • Algorithm based on image adaptive least squares illumination correction Training image A Image A illuminated as B Testing image B Adaptive segmentation Image B illuminated as A Least squares estimate of illumination

Preprocessing Results Illumination Correction Enrollment : Process of accepting the image and creating a

Preprocessing Results Illumination Correction Enrollment : Process of accepting the image and creating a feature set for recognition. Set tested (Yale) Enrollment Rate (Commercial face recognition system) Original 56 % Preprocessed 90 %

Comparison with Existing Methods Test subset 3 D Morphable Model Our algorithm PIE frontal

Comparison with Existing Methods Test subset 3 D Morphable Model Our algorithm PIE frontal 99. 8 % 99. 6 % • 3 D morphable models : • Good results (FRVT 2002). • Very complex, computationally expensive, • manual labeling of features 1. T. Vetter and V. Blanz, “Face Recognition Based on Fitting a 3 D Morphable Model, ” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1063 --1075, Sept. 2003.

Preprocessing Example train Vector Field Representation of f test Preprocessed Test Image

Preprocessing Example train Vector Field Representation of f test Preprocessed Test Image

Preprocessing Example Notre Dame Database

Preprocessing Example Notre Dame Database

Preprocessing Example Notre Dame Database

Preprocessing Example Notre Dame Database

Recognition Example With the correct gallery Gallery Probe Bijective Mapping White Region measure of

Recognition Example With the correct gallery Gallery Probe Bijective Mapping White Region measure of bijectivity (52. 91%)

Recognition Example With the incorrect gallery Gallery Probe Bijective Mapping White Region measure of

Recognition Example With the incorrect gallery Gallery Probe Bijective Mapping White Region measure of bijectivity (33. 94%)

Face Recognition Performance Notre Dame Database

Face Recognition Performance Notre Dame Database

Conclusion and Future Work • New algorithm for registration and illumination correction to enhance

Conclusion and Future Work • New algorithm for registration and illumination correction to enhance the performance of face recognition systems • Algorithm is based on properties of the mapping between test and train data • Mapping produces similarity scores which can be used for a stand-alone face recognition algorithm • Extend algorithm for high resolution data • Reduce algorithm complexity