Components of a computer vision system Camera Lighting

Components of a computer vision system Camera Lighting Computer Scene Interpretation Srinivasa Narasimhan’s slide

Computer vision vs Human Vision What we see What a computer sees Srinivasa Narasimhan’s slide

A little story about Computer Vision In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision)

A little story about Computer Vision Founder, MIT AI project In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision)

A little story about Computer Vision Founder, MIT AI project In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision) Professor of Electrical Engineering, MIT

A little story about Computer Vision In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision) Image Understanding

A little story about Computer Vision In 1966, Marvin Minsky at MIT asked his undergraduate student Gerald Jay Sussman to “spend the summer linking a camera to a computer and getting the computer to describe what it saw”. We now know that the problem is slightly more difficult than that. (Szeliski 2009, Computer Vision) Image Understanding Image Sensing

Continue on CAPTCHA stands for "Completely Automated Public Turing test to Tell Computers and Humans Apart". Picture of a CAPTCHA in use at Yahoo. http: //www. cs. sfu. ca/~mori/research/gimpy/

Breaking a Visual CAPTCHA On EZ-Gimpy: a success rate of 176/191=92%! Other examples http: //www. cs. sfu. ca/~mori/research/gimpy/ez / http: //www. cs. sfu. ca/~mori/research/gimpy/

Breaking a Visual CAPTCHA On more difficult Gimpy: a success rate of 33%! Other examples http: //www. cs. sfu. ca/~mori/research/gimpy/ha rd/ http: //www. cs. sfu. ca/~mori/research/gimpy/

Breaking a Visual CAPTCHA YAHOO’s current CAPTCHA format http: //en. wikipedia. org/wiki/CAPTCHA

Face Detection and Recognition Applications: Security, Law Enforcement, Surveillance

Face Detection and Recognition Smart cameras: auto focus, red eye removal, auto color correction

Face Detection and Tracking

Face Detection and Tracking

Face Detection and Tracking Lexus LS 600 Driver Monitor System

General Motion Tracking Hidden Dragon Crouching Tiger

General Motion Tracking Application Andy Serkis, Gollum, Lord of the Rings

Segmentation http: //www. eecs. berkeley. edu/Research/Projects/CS/vision/bsds/

Segmentation using Graph Cuts Application Medical Image Processing

Segmentation using Graph Cuts Input Matting: Soft Segmentation Composition

Segmentation using Graph Cuts State-of-the-art Tool (videosnapcut. mp 4) http: //juew. org/projects/Snap. Cut/snapcut. htm

From 2 D to 3 D http: //www. eecs. harvard. edu/~zickler/helmholtz. html

Projective Geometry

Single View Metrology • http: //research. microsoft. com/vision/cambridg e/3 d/default. htm

Single View Metrology • http: //research. microsoft. com/vision/cambridg e/3 d/default. htm

Stereo scene point image plane optical center

Stereo Basic Principle: Triangulation • Gives reconstruction as intersection of two rays • Requires – Camera positions – point correspondence

Using 3 D structure to organize photos http: //phototour. cs. washington. edu/

Using 3 D structure to organize photos http: //photosynth. net/

Reconstructing detailed 3 D models rendered model example input image

Reconstructing detailed 3 D models rendered model example input image

Reconstructing detailed 3 D models http: //grail. cs. washington. edu/ projects/mvscpc/ rendered model example input image

Reconstructing detailed 3 D models rendered model example input image

Reconstructing detailed 3 D models rendered model example input image

Application: View morphing

Application: View morphing

From Static Statues to Dynamic Targets MSR Image based Reality Project http: //research. microsoft. com/~larryz/videoviewinterpolation. htm …|

Spacetime Face Capture System Black & White Cameras Color Cameras Video Projectors

System in Action

Input Videos (640 480, 60 fps)

Spacetime Stereo Reconstruction

Applications Entertainment: Games & Movies Medical Practice: Prosthetics

Computational Photography • High Dynamic Range Conventional Image High Dynamic Range Image Nayar et al 2002

Computational Photography • High Dynamic Range Modulator Sensor Optics Assorted-pixel camera High Dynamic Range Image Nayar et al 2002

Computational Photography • High Dynamic Range Handheld camera Digital Gain Adjustment

Computational Photography • High Dynamic Range Handheld camera High Dynamic Range Image Zhang et al 2010

Summary • Recognize things • Reconstruct 3 D structures • Enhance Photography

If you are interested in, Major Conferences: Computer Vision and Pattern Recognition (CVPR) International Conference on Computer Vision (ICCV) European Conference on Computer Vision (ECCV) ACM SIGGRAPH Conference (SIGGRAPH) Faculty: Chuck Dyer, Vikas Singh, Li Zhang Courses: CS 766 Computer Vision CS 638 Special Topics Computational Photography CS 638 Special Topics Computational Methods in Medical Image Analysis