Largescale Realworld facial recognition in movie trailers Alan

Large-scale, Real-world facial recognition in movie trailers Alan Wright Presentation 7

recap of last week

Cast selector 3

Cast selector • Retrieves cast list from Rotten Tomatoes using their API. • Ignore tracks we don’t want. • Type custom names. • Allows two people to simultaneously label tracks and no labeling will be repeated. 4

Cast selector • All 2400+ tracks have now been labeled with the correct faces. • Faces not in Pub. Fig were still labeled. • Easily label more tracks if new trailers are added. • If faces are added to Pub. Fig, the labeling will not need to be redone. 5

Labeling results • 635 Unknown tracks • 712 Pub. Fig tracks • 1113 labeled tracks (faces not in Pub. Fig) • 4 ignored tracks.

Labeling results # of labels Pub. Fig Ids

Labeling results • Katherine Heigl was labeled the most with 51 tracks. • Each Pub. Fig face (in the trailers) has an average of 12 tracks.

Labeling Results • The most labeled face, not in Pub. Fig, was Edward Norton with 53 tracks. • 218 faces were labeled, but not in Pub. Fig. • Average of 5 tracks per face.

New pr curve Accurate with labeled faces

How can we add more faces? • Look at the distribution of faces that aren’t in Pub. Fig • Pick a threshold that will give us faces that appear often, and extend Pub. Fig. • Note: We want a good threshold because the average number of tracks person (not in Pub. Fig) is 5.

Track distribution Faces not in Pub. Fig # of labels Face IDs

Track distribution Faces not in Pub. Fig # of labels Threshold of 20 Face IDs

New faces • Choosing a threshold of 20 or more tracks gives us 9 new people: 1. Edward Norton - 53 2. Amanda Seyfried - 37 3. Jason Bateman - 34 4. Hilary Swank - 31 5. Paul Rudd - 30 6. Robert De Niro - 27 Leelee

new faces • Downloaded images for these 9 people and added them to Pub. Fig. (eye aligned, extracted features, etc)

new labeling distribution • 635 Unknown tracks • 998 Extended Pub. Fig tracks • 827 labeled tracks (faces not in Pub. Fig) • 4 ignored tracks.

What’s next? • Run over new supplemented data (Server will be up this afternoon) • Implement other voting methods: 1. Logarithmic pooling 2. Borda Count • Look at other ways to create a single confidence score for non-avg SRC and SVM methods • Experiment with different parameters: crop, pca dimensions, features, voting