Faces in the Wild Detection Alignment and Recognition

Faces in the Wild Detection, Alignment and Recognition of Real World Faces Erik Learned-Miller with Vidit Jain, Gary Huang, Andras Ferencz, et al. Computer Science Department

Is Face Recognition Solved? Computer Science 2

Is Face Recognition Solved? “ 100% Accuracy in Automatic Face Recognition” [!!!] Science 25 January 2008 Computer Science 3

Is Face Recognition Solved? “ 100% Accuracy in Automatic Face Recognition” [!!!] Science 25 January 2008 A history of overstated results. Computer Science 4

The Truth § Many different face recognition problems • Out of context, accuracy is meaningless! § Many problems are REALLY HARD! • For some problems state of the art is 70% or worse! § We have a long way to go! Computer Science 5

Face Recognition at UMass § § § Labeled Faces in the Wild The Detection-Alignment-Recognition pipeline Congealing and automatic face alignment Hyper-features for face recognition New directions in recognition Computer Science 6

Labeled Faces in the Wild http: //vis-www. cs. umass. edu/lfw/ Computer Science 7

The Many Faces of Face Recognition Labeled Faces in the Wild Computer Science 8

The Many Faces of Face Recognition Labeled Faces in the Wild Computer Science 9

The Many Faces of Face Recognition Labeled Faces in the Wild Computer Science 10

The Many Faces of Face Recognition Labeled Faces in the Wild Computer Science 11

The Many Faces of Face Recognition Labeled Faces in the Wild Computer Science 12

Labeled Faces in the Wild § § § 13, 233 images, with name of each person 5749 people 1680 people with 2 or more images § Designed for the “unseen pair matching problem”. • • § § Train on matched or mismatched pairs. Test on never-before-seen pairs. Distinct from problems with “galleries” or training data for each target image. Best accuracy: currently about 73%! Computer Science 13

Detection-Alignment-Recognition Pipeline Detection Alignment Recognition “Same” Computer Science 14

Detection-Alignment-Recognition Pipeline Detection Alignment Recognition “Same” Parts should work together. Computer Science 15

Labeled Faces in the Wild § All images are output of a standard face detector. § Also provides aligned images. § Consequence: any face recognition algorithm that works well on LFW can easily be turned into a complete system. Computer Science 16

Congealing (CVPR 2000) Computer Science 17

Criterion of Joint Alignment § Minimize sum of pixel stack entropies by transforming each image. A pixel stack Computer Science 18

Congealing Complex Images Window around pixel SIFT vector and clusters SIFT clusters vector representing probability of each cluster, or “mixture” of clusters Computer Science 19

Crash Course on Martian Identification Martian training set Test: Find Bob after one meeting ? = = Bob = Computer Science 21

Training Data “same” “different” Computer Science 22

General Approach to Hyper-feature method § Carefully align objects § Develop a patch-based model of image differences. § Score match/mismatch based on patch differences. Computer Science 23

Three Models 1. Universal patch model: P(patch. Distance|same) P(patch. Distance|different) 2. Spatially dependent patch model: P(patch. Distance |same, x, y) P(patch. Distance |different, x, y) 3. Hyper-feature dependent model: 1. P(patch. Distance |same, x, y, appearance) 2. P(patch. Distance |different, x, y, appearance) Computer Science 24

Universal Patch Model A single P(dist | same) for all patches Different blue patches are evidence against a match! Computer Science 25

Spatial Patch Model P(dist|same, x 1, y 1) estimated separately from P(dist|same, x 2, y 2) Greatly increases discriminativeness of model. Computer Science 26

Hyper-Feature Patch Model Is the patch from a matching face going to match this patch? Computer Science 27

Hyper-Feature Patch Model Is the patch from a matching face going to match this patch? Probably yes Computer Science 28

Hyper-Feature Patch Model What about this patch? Computer Science 29

Hyper-Feature Patch Model What about this patch? Probably not. Computer Science 30

Ridiculous Errors from the World’s Best Unconstrained Face Recognition System Computer Science 31

Ridiculous Errors from the World’s Best Unconstrained Face Recognition System Computer Science 32

The New Mission: Estimate Higher Level Features Computer Science 33

The New Mission: Estimate Higher Level Features Can we guess pose? Computer Science 34

The New Mission: Estimate Higher Level Features Can we guess gender? Computer Science 35

The New Mission: Estimate Higher Level Features Can we guess degree of balding, beardedness, moustache? Computer Science 36

The New Mission: Estimate Higher Level Features Can we say that none of these individuals are the same person? Computer Science 37

What can we do with a good segmentation? Computer Science 38

CRF Segmentations Computer Science 39

CRF Segmentations Computer Science 40

Who’s This? Computer Science 41

Who’s This? Computer Science 42

Who’s This? from www. coolopticalillusions. com Computer Science 43

Thanks Computer Science Department