Interactively Modeling with Photogrammetry Pierre Poulin Mathieu Ouimet

Interactively Modeling with Photogrammetry Pierre Poulin Mathieu Ouimet Marie-Claude Frasson Dép. Informatique et recherche opérationnelle Université de Montréal

Motivation • Photo-realism is difficult to achieve • Important recent progress in rendering • Acquiring realistic 3 D models is still a major hurtle • Important needs for realism, special effects in movies, CAR, etc. Extracting 3 D models from photographs

Computer Vision / Robotics • 3 D models do not satisfy most of the visual accuracy necessary in graphics • Fully automatic systems are challenging : – false correspondences – missed edge detections – noise – textures • Provide much inspiration in our system

Our Interactive Reconstruction System • User knows the 3 D models / textures • User is responsible for everything • User interactions : – User draws 2 D primitives – User puts the 2 D primitives in correspondences – User adds 3 D constraints – User extracts a unified texture

Drawing 2 D Primitives

Correspondences

3 D Constraints Co-planarity Perpendicularity Parallelism

Extracted Textures

Reconstruction Process • • • Incremental Robust Intuitive Provides good graphics models Labor-intensive

The Camera • Our camera is a transformation matrix • No explicit need for real camera parameters

Reconstructing a Camera • 6 or more 2 D-to-3 D point correspondences (0, 1, 0) (1, 1, 0) (0, 1, 1) (1, 0, 0) (0, 0, 1) (1, 0, 1)

Reconstructing a Camera • Least-squares to compute all Ti • Solution with SVD – Fast – Robust – Always provides a solution – Conditions for accuracy similar to non-linear

Reconstructing a 3 D Point • Incidence of 3 D point on planes • Least-squares to compute each (x, y, z) • Polygons as set of 3 D points

Reconstructing a 3 D Line • Plücker coordinates of a 3 D line

Additional 3 D Constraints • Co-planarity • Parallelism • Perpendicularity Weights • Weights can be used to alter the importance of certain constraints

Iterating • Better cameras give better 3 D geometry • Better 3 D geometry give better cameras • Iterations between the two improve both

Convergence

Recovering Texel Colors t v v u Texture map u t s 3 D Polygon 2 D Images s

Occlusion Testing 3 D Model 2 D Image Zones of Occlusion

Linear Fit • Misalignments due to imprecisions in the 3 D model and its cameras • 2 D transformation matrix using least-squares

Unifying Texel Criteria • Clustering to discriminate view-dependent colors for a texel • Other metrics used to weight valid texels : – Projected area (adaptive sampling) – Texture quality

Two Scenes with Cubes

Desktop

Lego Tower

Coffee Pot

Conclusions • User knows best • Satisfying 3 D models and extracted textures • Labor-intensive

Future Work • Better user interface • Error detection • Radiances, reflectances, and global illumination • Displacement maps on 3 D primitives • Bounds on reconstructed information