Introduction to Computer Vision CS 223 B Winter

Introduction to Computer Vision CS 223 B, Winter 2005 1/25/2005 Introduction to Computer Vision

Richard Szeliski – Guest Lecturer • Ph. D. , Carnegie Mellon, 1988 • Researcher, Cambridge Research Lab at DEC, 1990 -1995 • Senior Researcher, Interactive Visual Media Group, Microsoft, 1995 • Research interests: • computer vision (stereo, motion), computer graphics (image-based rendering), parallel programming 1/25/2005 Introduction to Computer Vision 2

What is Computer Vision? 1/25/2005 Introduction to Computer Vision

What is Computer Vision? • • Image Understanding (AI, behavior) A sensor modality for robotics Computer emulation of human vision Inverse of Computer Graphics Computer vision World model 1/25/2005 World model Computer graphics Introduction to Computer Vision 4

Intersection of Vision and Graphics rendering surface design animation user-interfaces modeling - shape - light - motion - optics - images IP shape estimation motion estimation recognition 2 D modeling Computer Graphics Computer Vision 1/25/2005 Introduction to Computer Vision 5

Computer Vision [Trucco&Verri’ 98] 1/25/2005 Introduction to Computer Vision 6

Image-Based Modeling image processing graphics Images (2 D) Geometry (3 D) shape + Photometry appearance vision 3 Image processing 2. 1 Geometric image formation 4 Feature extraction 5 Camera calibration 7 Image alignment 6 Structure from motion 2. 2 Photometric image formation 8 Mosaics 9 Stereo correspondence 11 Model-based reconstruction 12 Photometric recovery 14 Image-based rendering 1/25/2005 Introduction to Computer Vision 7

Syllabus Image Transforms / Representations • filters, pyramids, steerable filters • warping and resampling • blending • image statistics, denoising/coding • edge and feature detection 1/25/2005 Introduction to Computer Vision 11

Image Pyramid Lowpass Images Bandpass Images 1/25/2005 Introduction to Computer Vision 12

Pyramid Blending 1/25/2005 Introduction to Computer Vision 13

Parametric (global) warping Examples of parametric warps: translation affine 1/25/2005 rotation perspective Introduction to Computer Vision aspect cylindrical 14

Syllabus Optical Flow • least squares regression • iterative, coarse-to-fine • parametric • robust flow and mixture models • layers, EM 1/25/2005 Introduction to Computer Vision 15

Image Morphing 1/25/2005 Introduction to Computer Vision 16

Syllabus Projective geometry • points, lines, planes, transforms Camera calibration and pose • point matching and tracking • lens distortion Image registration • mosaics 1/25/2005 Introduction to Computer Vision 17

Panoramic Mosaics + … + 1/25/2005 Introduction to Computer Vision = 18

Syllabus 3 D structure from motion • two frame techniques • factorization of shape and motion • bundle adjustment 1/25/2005 Introduction to Computer Vision 19

3 D Shape Reconstruction Debevec, Taylor, and Malik, SIGGRAPH 1996 1/25/2005 Introduction to Computer Vision 20

Face Modeling 1/25/2005 Introduction to Computer Vision 21

Syllabus Stereo • correspondence • local methods • global optimization 1/25/2005 Introduction to Computer Vision 22

View Morphing Morph between pair of images using epipolar geometry [Seitz & Dyer, SIGGRAPH’ 96] 1/25/2005 Introduction to Computer Vision 23

Z-keying: mix live and synthetic Takeo Kanade, CMU (Stereo Machine) 1/25/2005 Introduction to Computer Vision 24

Virtualized Reality. TM Takeo Kanade, CMU • collect video from 50+ stream reconstruct 3 D model sequences http: //www. cs. cmu. edu/afs/cs/project/Virtualized. R/www/Virtualized. R. html 1/25/2005 Introduction to Computer Vision 25

Virtualized Reality. TM Takeo Kanade, CMU • generate new video • steerable version used for Super. Bowl XXV “eye vision” system 1/25/2005 Introduction to Computer Vision 26

Syllabus Tracking • eigen-tracking • on-line EM • Kalman filter • particle filtering • appearance models 1/25/2005 Introduction to Computer Vision 27

Syllabus Recognition • subspaces and local invariance features • face recognition • color histograms • textures Image editing • segmentation • curve tracking 1/25/2005 Introduction to Computer Vision 28

Edge detection and editing Elder, J. H. and R. M. Goldberg. "Image Editing in the Contour Domain, " Proc. IEEE: Computer Vision and Pattern Recognition, pp. 374 -381, June, 1998. 1/25/2005 Introduction to Computer Vision 29

Image Enhancement High dynamic range photography [Debevec et al. ’ 97; Mitsunaga & Nayar’ 99] • combine several different exposures together 1/25/2005 Introduction to Computer Vision 30

Syllabus Image-based rendering • Lightfields and Lumigraphs • concentric mosaics • layered models • video-based rendering 1/25/2005 Introduction to Computer Vision 31

Concentric Mosaics Interpolate between several panoramas to give a 3 D depth effect [Shum & He, SIGGRAPH’ 99] 1/25/2005 Introduction to Computer Vision 32

Applications • Geometric reconstruction: modeling, forensics, special effects (ILM, Real. Vis, 2 D 3) • Image and video editing (Avid, Adobe) • Webcasting and Indexing Digital Video (Virage) • Scientific / medical applications (GE) 1/25/2005 Introduction to Computer Vision 33

Applications • • • Tracking and surveillance (Sarnoff) Fingerprint recognition (Digital Persona) Biometrics / iris scans (Iridian Technologies) Vehicle safety (Mobil. Eye) Drowning people (Vision. IQ Inc) Optical motion capture (Vicon) 1/25/2005 Introduction to Computer Vision 34

Projects Let’s look at what students have done in previous years … Stanford http: //www. stanford. edu/class/cs 223 b/winter 01 -02/projects. html CMU http: //www-2. cs. cmu. edu/~ph/869/www/869. html UW http: //www. cs. washington. edu/education/courses/cse 590 ss/01 wi/ GA Tech http: //www. cc. gatech. edu/classes/AY 2002/cs 4480_spring/ 1/25/2005 Introduction to Computer Vision 35