Introduction to Computer Vision CS 223 B Winter

  • Slides: 32
Download presentation
Introduction to Computer Vision CS 223 B, Winter 2005 1/25/2005 Introduction to Computer Vision

Introduction to Computer Vision CS 223 B, Winter 2005 1/25/2005 Introduction to Computer Vision

Richard Szeliski – Guest Lecturer • Ph. D. , Carnegie Mellon, 1988 • Researcher,

Richard Szeliski – Guest Lecturer • Ph. D. , Carnegie Mellon, 1988 • Researcher, Cambridge Research Lab at DEC, 1990 -1995 • Senior Researcher, Interactive Visual Media Group, Microsoft, 1995 • Research interests: • computer vision (stereo, motion), computer graphics (image-based rendering), parallel programming 1/25/2005 Introduction to Computer Vision 2

What is Computer Vision? 1/25/2005 Introduction to Computer Vision

What is Computer Vision? 1/25/2005 Introduction to Computer Vision

What is Computer Vision? • • Image Understanding (AI, behavior) A sensor modality for

What is Computer Vision? • • Image Understanding (AI, behavior) A sensor modality for robotics Computer emulation of human vision Inverse of Computer Graphics Computer vision World model 1/25/2005 World model Computer graphics Introduction to Computer Vision 4

Intersection of Vision and Graphics rendering surface design animation user-interfaces modeling - shape -

Intersection of Vision and Graphics rendering surface design animation user-interfaces modeling - shape - light - motion - optics - images IP shape estimation motion estimation recognition 2 D modeling Computer Graphics Computer Vision 1/25/2005 Introduction to Computer Vision 5

Computer Vision [Trucco&Verri’ 98] 1/25/2005 Introduction to Computer Vision 6

Computer Vision [Trucco&Verri’ 98] 1/25/2005 Introduction to Computer Vision 6

Image-Based Modeling image processing graphics Images (2 D) Geometry (3 D) shape + Photometry

Image-Based Modeling image processing graphics Images (2 D) Geometry (3 D) shape + Photometry appearance vision 3 Image processing 2. 1 Geometric image formation 4 Feature extraction 5 Camera calibration 7 Image alignment 6 Structure from motion 2. 2 Photometric image formation 8 Mosaics 9 Stereo correspondence 11 Model-based reconstruction 12 Photometric recovery 14 Image-based rendering 1/25/2005 Introduction to Computer Vision 7

Syllabus Image Transforms / Representations • filters, pyramids, steerable filters • warping and resampling

Syllabus Image Transforms / Representations • filters, pyramids, steerable filters • warping and resampling • blending • image statistics, denoising/coding • edge and feature detection 1/25/2005 Introduction to Computer Vision 11

Image Pyramid Lowpass Images Bandpass Images 1/25/2005 Introduction to Computer Vision 12

Image Pyramid Lowpass Images Bandpass Images 1/25/2005 Introduction to Computer Vision 12

Pyramid Blending 1/25/2005 Introduction to Computer Vision 13

Pyramid Blending 1/25/2005 Introduction to Computer Vision 13

Parametric (global) warping Examples of parametric warps: translation affine 1/25/2005 rotation perspective Introduction to

Parametric (global) warping Examples of parametric warps: translation affine 1/25/2005 rotation perspective Introduction to Computer Vision aspect cylindrical 14

Syllabus Optical Flow • least squares regression • iterative, coarse-to-fine • parametric • robust

Syllabus Optical Flow • least squares regression • iterative, coarse-to-fine • parametric • robust flow and mixture models • layers, EM 1/25/2005 Introduction to Computer Vision 15

Image Morphing 1/25/2005 Introduction to Computer Vision 16

Image Morphing 1/25/2005 Introduction to Computer Vision 16

Syllabus Projective geometry • points, lines, planes, transforms Camera calibration and pose • point

Syllabus Projective geometry • points, lines, planes, transforms Camera calibration and pose • point matching and tracking • lens distortion Image registration • mosaics 1/25/2005 Introduction to Computer Vision 17

Panoramic Mosaics + … + 1/25/2005 Introduction to Computer Vision = 18

Panoramic Mosaics + … + 1/25/2005 Introduction to Computer Vision = 18

Syllabus 3 D structure from motion • two frame techniques • factorization of shape

Syllabus 3 D structure from motion • two frame techniques • factorization of shape and motion • bundle adjustment 1/25/2005 Introduction to Computer Vision 19

3 D Shape Reconstruction Debevec, Taylor, and Malik, SIGGRAPH 1996 1/25/2005 Introduction to Computer

3 D Shape Reconstruction Debevec, Taylor, and Malik, SIGGRAPH 1996 1/25/2005 Introduction to Computer Vision 20

Face Modeling 1/25/2005 Introduction to Computer Vision 21

Face Modeling 1/25/2005 Introduction to Computer Vision 21

Syllabus Stereo • correspondence • local methods • global optimization 1/25/2005 Introduction to Computer

Syllabus Stereo • correspondence • local methods • global optimization 1/25/2005 Introduction to Computer Vision 22

View Morphing Morph between pair of images using epipolar geometry [Seitz & Dyer, SIGGRAPH’

View Morphing Morph between pair of images using epipolar geometry [Seitz & Dyer, SIGGRAPH’ 96] 1/25/2005 Introduction to Computer Vision 23

Z-keying: mix live and synthetic Takeo Kanade, CMU (Stereo Machine) 1/25/2005 Introduction to Computer

Z-keying: mix live and synthetic Takeo Kanade, CMU (Stereo Machine) 1/25/2005 Introduction to Computer Vision 24

Virtualized Reality. TM Takeo Kanade, CMU • collect video from 50+ stream reconstruct 3

Virtualized Reality. TM Takeo Kanade, CMU • collect video from 50+ stream reconstruct 3 D model sequences http: //www. cs. cmu. edu/afs/cs/project/Virtualized. R/www/Virtualized. R. html 1/25/2005 Introduction to Computer Vision 25

Virtualized Reality. TM Takeo Kanade, CMU • generate new video • steerable version used

Virtualized Reality. TM Takeo Kanade, CMU • generate new video • steerable version used for Super. Bowl XXV “eye vision” system 1/25/2005 Introduction to Computer Vision 26

Syllabus Tracking • eigen-tracking • on-line EM • Kalman filter • particle filtering •

Syllabus Tracking • eigen-tracking • on-line EM • Kalman filter • particle filtering • appearance models 1/25/2005 Introduction to Computer Vision 27

Syllabus Recognition • subspaces and local invariance features • face recognition • color histograms

Syllabus Recognition • subspaces and local invariance features • face recognition • color histograms • textures Image editing • segmentation • curve tracking 1/25/2005 Introduction to Computer Vision 28

Edge detection and editing Elder, J. H. and R. M. Goldberg.

Edge detection and editing Elder, J. H. and R. M. Goldberg. "Image Editing in the Contour Domain, " Proc. IEEE: Computer Vision and Pattern Recognition, pp. 374 -381, June, 1998. 1/25/2005 Introduction to Computer Vision 29

Image Enhancement High dynamic range photography [Debevec et al. ’ 97; Mitsunaga & Nayar’

Image Enhancement High dynamic range photography [Debevec et al. ’ 97; Mitsunaga & Nayar’ 99] • combine several different exposures together 1/25/2005 Introduction to Computer Vision 30

Syllabus Image-based rendering • Lightfields and Lumigraphs • concentric mosaics • layered models •

Syllabus Image-based rendering • Lightfields and Lumigraphs • concentric mosaics • layered models • video-based rendering 1/25/2005 Introduction to Computer Vision 31

Concentric Mosaics Interpolate between several panoramas to give a 3 D depth effect [Shum

Concentric Mosaics Interpolate between several panoramas to give a 3 D depth effect [Shum & He, SIGGRAPH’ 99] 1/25/2005 Introduction to Computer Vision 32

Applications • Geometric reconstruction: modeling, forensics, special effects (ILM, Real. Vis, 2 D 3)

Applications • Geometric reconstruction: modeling, forensics, special effects (ILM, Real. Vis, 2 D 3) • Image and video editing (Avid, Adobe) • Webcasting and Indexing Digital Video (Virage) • Scientific / medical applications (GE) 1/25/2005 Introduction to Computer Vision 33

Applications • • • Tracking and surveillance (Sarnoff) Fingerprint recognition (Digital Persona) Biometrics /

Applications • • • Tracking and surveillance (Sarnoff) Fingerprint recognition (Digital Persona) Biometrics / iris scans (Iridian Technologies) Vehicle safety (Mobil. Eye) Drowning people (Vision. IQ Inc) Optical motion capture (Vicon) 1/25/2005 Introduction to Computer Vision 34

Projects Let’s look at what students have done in previous years … Stanford http:

Projects Let’s look at what students have done in previous years … Stanford http: //www. stanford. edu/class/cs 223 b/winter 01 -02/projects. html CMU http: //www-2. cs. cmu. edu/~ph/869/www/869. html UW http: //www. cs. washington. edu/education/courses/cse 590 ss/01 wi/ GA Tech http: //www. cc. gatech. edu/classes/AY 2002/cs 4480_spring/ 1/25/2005 Introduction to Computer Vision 35