Introduction to Computer Vision CS 223 B Winter
Introduction to Computer Vision CS 223 B, Winter 2005 1/25/2005 Introduction to Computer Vision
Richard Szeliski – Guest Lecturer • Ph. D. , Carnegie Mellon, 1988 • Researcher, Cambridge Research Lab at DEC, 1990 -1995 • Senior Researcher, Interactive Visual Media Group, Microsoft, 1995 • Research interests: • computer vision (stereo, motion), computer graphics (image-based rendering), parallel programming 1/25/2005 Introduction to Computer Vision 2
What is Computer Vision? 1/25/2005 Introduction to Computer Vision
What is Computer Vision? • • Image Understanding (AI, behavior) A sensor modality for robotics Computer emulation of human vision Inverse of Computer Graphics Computer vision World model 1/25/2005 World model Computer graphics Introduction to Computer Vision 4
Intersection of Vision and Graphics rendering surface design animation user-interfaces modeling - shape - light - motion - optics - images IP shape estimation motion estimation recognition 2 D modeling Computer Graphics Computer Vision 1/25/2005 Introduction to Computer Vision 5
Computer Vision [Trucco&Verri’ 98] 1/25/2005 Introduction to Computer Vision 6
Image-Based Modeling image processing graphics Images (2 D) Geometry (3 D) shape + Photometry appearance vision 3 Image processing 2. 1 Geometric image formation 4 Feature extraction 5 Camera calibration 7 Image alignment 6 Structure from motion 2. 2 Photometric image formation 8 Mosaics 9 Stereo correspondence 11 Model-based reconstruction 12 Photometric recovery 14 Image-based rendering 1/25/2005 Introduction to Computer Vision 7
Syllabus Image Transforms / Representations • filters, pyramids, steerable filters • warping and resampling • blending • image statistics, denoising/coding • edge and feature detection 1/25/2005 Introduction to Computer Vision 11
Image Pyramid Lowpass Images Bandpass Images 1/25/2005 Introduction to Computer Vision 12
Pyramid Blending 1/25/2005 Introduction to Computer Vision 13
Parametric (global) warping Examples of parametric warps: translation affine 1/25/2005 rotation perspective Introduction to Computer Vision aspect cylindrical 14
Syllabus Optical Flow • least squares regression • iterative, coarse-to-fine • parametric • robust flow and mixture models • layers, EM 1/25/2005 Introduction to Computer Vision 15
Image Morphing 1/25/2005 Introduction to Computer Vision 16
Syllabus Projective geometry • points, lines, planes, transforms Camera calibration and pose • point matching and tracking • lens distortion Image registration • mosaics 1/25/2005 Introduction to Computer Vision 17
Panoramic Mosaics + … + 1/25/2005 Introduction to Computer Vision = 18
Syllabus 3 D structure from motion • two frame techniques • factorization of shape and motion • bundle adjustment 1/25/2005 Introduction to Computer Vision 19
3 D Shape Reconstruction Debevec, Taylor, and Malik, SIGGRAPH 1996 1/25/2005 Introduction to Computer Vision 20
Face Modeling 1/25/2005 Introduction to Computer Vision 21
Syllabus Stereo • correspondence • local methods • global optimization 1/25/2005 Introduction to Computer Vision 22
View Morphing Morph between pair of images using epipolar geometry [Seitz & Dyer, SIGGRAPH’ 96] 1/25/2005 Introduction to Computer Vision 23
Z-keying: mix live and synthetic Takeo Kanade, CMU (Stereo Machine) 1/25/2005 Introduction to Computer Vision 24
Virtualized Reality. TM Takeo Kanade, CMU • collect video from 50+ stream reconstruct 3 D model sequences http: //www. cs. cmu. edu/afs/cs/project/Virtualized. R/www/Virtualized. R. html 1/25/2005 Introduction to Computer Vision 25
Virtualized Reality. TM Takeo Kanade, CMU • generate new video • steerable version used for Super. Bowl XXV “eye vision” system 1/25/2005 Introduction to Computer Vision 26
Syllabus Tracking • eigen-tracking • on-line EM • Kalman filter • particle filtering • appearance models 1/25/2005 Introduction to Computer Vision 27
Syllabus Recognition • subspaces and local invariance features • face recognition • color histograms • textures Image editing • segmentation • curve tracking 1/25/2005 Introduction to Computer Vision 28
Edge detection and editing Elder, J. H. and R. M. Goldberg. "Image Editing in the Contour Domain, " Proc. IEEE: Computer Vision and Pattern Recognition, pp. 374 -381, June, 1998. 1/25/2005 Introduction to Computer Vision 29
Image Enhancement High dynamic range photography [Debevec et al. ’ 97; Mitsunaga & Nayar’ 99] • combine several different exposures together 1/25/2005 Introduction to Computer Vision 30
Syllabus Image-based rendering • Lightfields and Lumigraphs • concentric mosaics • layered models • video-based rendering 1/25/2005 Introduction to Computer Vision 31
Concentric Mosaics Interpolate between several panoramas to give a 3 D depth effect [Shum & He, SIGGRAPH’ 99] 1/25/2005 Introduction to Computer Vision 32
Applications • Geometric reconstruction: modeling, forensics, special effects (ILM, Real. Vis, 2 D 3) • Image and video editing (Avid, Adobe) • Webcasting and Indexing Digital Video (Virage) • Scientific / medical applications (GE) 1/25/2005 Introduction to Computer Vision 33
Applications • • • Tracking and surveillance (Sarnoff) Fingerprint recognition (Digital Persona) Biometrics / iris scans (Iridian Technologies) Vehicle safety (Mobil. Eye) Drowning people (Vision. IQ Inc) Optical motion capture (Vicon) 1/25/2005 Introduction to Computer Vision 34
Projects Let’s look at what students have done in previous years … Stanford http: //www. stanford. edu/class/cs 223 b/winter 01 -02/projects. html CMU http: //www-2. cs. cmu. edu/~ph/869/www/869. html UW http: //www. cs. washington. edu/education/courses/cse 590 ss/01 wi/ GA Tech http: //www. cc. gatech. edu/classes/AY 2002/cs 4480_spring/ 1/25/2005 Introduction to Computer Vision 35
- Slides: 32