Extreme Nonparametric Object Recognition 80 million tiny images

Extreme, Non-parametric Object Recognition 80 million tiny images (Torralba et al)

Our World is Boring… Slide by Antonio Torralba

Lots Of Images A. Torralba, R. Fergus, W. T. Freeman. PAMI 2008

Lots Of Images

Automatic Colorization Result Grayscale input High resolution Colorization of input using average A. Torralba, R. Fergus, W. T. Freeman. 2008

Automatic Orientation • Many images have ambiguous orientation • Look at top 25% by confidence: • Examples of high and low confidence images: Slide by Antonio Torralba

Automatic Orientation Examples A. Torralba, R. Fergus, W. T. Freeman. 2008

What If we have Labels…

Are 32 x 32 images enough?

Label. Me Tiny images Caltech 101 10% of the objects account for 90% of the data ~Zipf’s law Slide by Antonio Torralba

Do people do this?

What’s the Capacity of Visual Long Term Memory? What we know… Standing (1973) What we don’t know… … what people are remembering for each item? 10, 000 images According to Standing “Basically, my recollection is that we just separated the pictures into distinct thematic categories: e. g. cars, animals, singleperson, 2 -people, plants, etc. ) Only a few slides were selected which fell into each category, and they were visually distinct. ” 83% Recognition … people can remember thousands of images High Fidelity Visual Memory is possible (Hollingworth 2004) Dogs Playing Cards “Gist” Only Sparse Details Highly Detailed Slide by Aude Oliva

Massive Memory I: Methods 1024 -back 1 -back . . Showed 14 observers 2500 categorically unique objects 1 at a time, 3 seconds each 800 ms blank between items Study session lasted about 5. 5 hours Repeat Detection task to maintain focus Followed by 300 2 -alternative forced choice tests Slide by Aude Oliva

Slide by Aude Oliva

how far can we push the fidelity of visual LTM representation ? Same object, different states Slide by Aude Oliva

Massive Memory I: Recognition Memory Results 92% Replication of Standing (1973) Visual Cognition Expert Predictions Slide by Aude Oliva

Massive Memory I: Recognition Memory Results 92% 88% 87% Slide by Aude Oliva

Extrapolation of Repeat Detection Data Human performances for n = 1024 Quadratic (r 2=. 988) Power law (r 2=. 988) Brady, Konkle, Alvarez, Oliva (submitted) Slide by Aude Oliva

Past and future of image datasets in computer vision Number of pictures Human Click Limit 1020 1015 Lena (all humanity taking one picture/second during 100 years) COREL a dataset in one picture 1010 2 billion 105 40. 000 1972 1996 2007 2020? Time Slide by Antonio Torralba