Visual Simulation CAP 6938 Dr Hassan Foroosh Dept

Visual Simulation CAP 6938 Dr. Hassan Foroosh Dept. of Computer Science UCF © Copyright Hassan Foroosh 2002

CAP 6938 “Visual Simulation” n Announcements n Class web site n n n Handouts n n Under construction Temporary site: http: //www. cs. ucf. edu/courses/cap 6938 -02 class info access/accounts survey Readings for Wednesday (via web site) n n Paul Heckbert, Survey of Texture Mapping, IEEE Computer Graphics and Applications, 6(11), November 1986, 56 -67. Beier, T. and Neely, S. , Feature-Based Image Metamorphosis, ACM Computer Graphics (SIGGRAPH'92), 26(2), July 1992, 35 -42

Today n Intro n n n 2 D image processing n n Admin Survey Introductions Course overview Blending Filtering Pyramids On Wednesday (1/8) n n image warping, morphing image enhancement

Vision vs. Graphics Image Sequence World model n Vision Engine Graphics Engine World model Image Sequence Vision and Graphics are inverse problems

Vision with Graphics Image Sequence Vision Engine Graphics Engine n Vision and graphics combined Image Sequence

Vision and Graphics rendering surface design animation user-interfaces modeling - shape - light - motion - optics - images IP shape estimation motion estimation recognition 2 D modeling Computer Graphics Computer Vision

Cross Fertilization n Vision’s impact on graphics n n n n image-based rendering model acquisition motion capture perceptual user interfaces special effects image editing Graphics’ impact on vision n reflectance transparency shape modeling

Course Objectives n What to expect n n n Knowledge of vision that is relevant to graphics How to apply your expertise in image analysis to synthesis Fundamentals Explore new avenues for research What not to expect n n Not a graphics course Not a complete vision course

Administrative Stuff n n Web Site: to be announced Grading n n n 2 programming projects 1 final research project Class presentation Class participation Software and Hardware n n n Programming projects in C/C++ Support code for Windows and Linux Lab work: You’re welcome to use your own machines or departmental machines

Prerequisites n n Prior course on vision OR graphics Assume n n n Familiarity with image representations Basic image processing (linear filtering, transforms, etc. ) Differential equations, linear algebra Camera modeling and projection Ability to read research articles, fill in gaps Questions? foroosh@cs. ucf. edu

Image Processing Elder, J. H. and R. M. Goldberg. "Image Editing in the Contour Domain, " Proc. IEEE: Computer Vision and Pattern Recognition, pp. 374 -381, June, 1998. http: //www. fearthis. com/warpimages/pres. shtml

Motion Estimation

Pose Estimation Ascending Stairs, Eadweard Muybridge, 1884 -85

3 D Shape Reconstruction Debevec, Taylor, and Malik, SIGGRAPH 1996 Foroosh, 2001

Image-Based Rendering View Morphing, Seitz and Dyer, SIGGRAPH 96

Modeling light "Interface", courtesy of Lance Williams, 1985 Environment Matting and Compositing, Zongker, Werner, Curless, and Salesin. SIGGRAPH 99

Image Blending

Feathering + 1 0 Encoding transparency I(x, y) = (a. R, a. G, a. B, a) = Iblend = Ileft + Iright See Blinn reading (CGA, 1994) for details

Effect of Window Size 1 left 1 right 0 0

Effect of Window Size 1 1 0 0

Good Window Size 1 0 “Optimal” Window: smooth but not ghosted

What is the Optimal Window? n To avoid seams n n window = size of largest prominent feature To avoid ghosting n window <= 2*size of smallest prominent feature Natural to cast this in the Fourier domain • largest frequency <= 2*size of smallest frequency • image frequency content should occupy one “octave” (power of two) FFT

What if the Frequency Spread is Wide FFT n Idea (Burt and Adelson) n n Compute Fleft = FFT(Ileft), Fright = FFT(Iright) Decompose Fourier image into octaves (bands) n n Feather corresponding octaves Flefti with Frighti n n n Fleft = Fleft 1 + Fleft 2 + … Can compute inverse FFT and feather in spatial domain Sum feathered octave images in frequency domain Better implemented in spatial domain

Octaves in the Spatial Domain Lowpass Images n Bandpass Images

Image Pyramids

Pyramid Creation filter mask “Gaussian” Pyramid n “Laplacian” Pyramid n Created from Gaussian pyramid by subtraction Ll = Gl – expand(Gl+1)

Pyramids n Advantages of pyramids n n n Many applications n n n Faster than Fourier transform Avoids “ringing” artifacts small images faster to process good for multiresolution processing compression progressive transmission Known as “mip-maps” in graphics community Precursor to wavelets n Wavelets also have these advantages

Pyramid Blending

laplacian level 4 laplacian level 2 laplacian level 0 left pyramid right pyramid blended pyramid

Blending Regions Other applications • Removing block artifacts in compressed images

Limitations?

Related Topics n Matting n n Given image and background(s), estimate foreground What if foreground object is refractive? n n Hole filling n n Environment matting Environment Matting and Compositing, Zongker, Werner, Curless, and Salesin. SIGGRAPH 99 Remove scratches, holes in an image Texture synthesis