INFORMATIK HardwareAccelerated Silhouette Matching Hendrik Lensch Wolfgang Heidrich

Overview • • INFORMATIK Motivation Comparing Silhouettes Stitching and Combining Textures Results and Conclusions

Acquiring Real World Models Geometry • 3 D scanner INFORMATIK Texture data • digital

3 D – 2 D Registration INFORMATIK Find the camera setting for each 2

Camera Model INFORMATIK Transformations • to camera coordinates (extrinsic): • to 2 D image

Similarity Measure INFORMATIK Which features to investigate? • no color information on the model

Similarity Measure INFORMATIK Compare silhouettes [Etienne de Silhouette 1709 -1767] • model: render monochrome

Distance Measure for Silhouettes INFORMATIK Point-to-outline distances • slow because points on the outline

Pixel-based Distance Measure Count the number of pixels covered by just one silhouette. •

Approximation of Squared Distances INFORMATIK Use smooth transitions • blur images • integrate squared

Non-linear Optimization INFORMATIK Downhill Simplex Method [Press 1992] • • • works for N

Simplex Method in 3 D INFORMATIK original simplex reflection and/or expansion shrinking random perturbation

Hierarchical Optimization • • INFORMATIK optimize on low resolution first restart optimization to avoid

Starting Point Generation INFORMATIK • • set camera distance tz depending on object size

Texture Stitching INFORMATIK § projective texture mapping § assign one image to each triangle

Blending Across Assignment Borders INFORMATIK • find border vertices • release all triangles around

Entire Texture INFORMATIK Hendrik Lensch

Results and Conclusions INFORMATIK Problems solved: • • automatic texture registration (R, t, f)

Questions? INFORMATIK visit us at www. mpi-sb. mpg. de Hendrik Lensch

Slides: 22

Download presentation

INFORMATIK Hardware-Accelerated Silhouette Matching Hendrik Lensch, Wolfgang Heidrich, and Hans-Peter Seidel Max-Planck-Institut für Informatik, Saarbrücken (Germany) Hendrik Lensch

Overview • • INFORMATIK Motivation Comparing Silhouettes Stitching and Combining Textures Results and Conclusions Hendrik Lensch

Acquiring Real World Models Geometry • 3 D scanner INFORMATIK Texture data • digital camera single sensor vs. multiple sensors Hendrik Lensch

3 D – 2 D Registration INFORMATIK Find the camera setting for each 2 D image. Hendrik Lensch

Camera Model INFORMATIK Transformations • to camera coordinates (extrinsic): • to 2 D image space (intrinsic): determine R, t and f (6+1 dimensions) Hendrik Lensch

Similarity Measure INFORMATIK Which features to investigate? • no color information on the model • correspondence of geometric features hard to find Hendrik Lensch

Similarity Measure INFORMATIK Compare silhouettes [Etienne de Silhouette 1709 -1767] • model: render monochrome • photo: automatic histogram-based segmentation Hendrik Lensch

Distance Measure for Silhouettes INFORMATIK Point-to-outline distances • slow because points on the outline must be determined • speedup by distance maps Hendrik Lensch

Pixel-based Distance Measure Count the number of pixels covered by just one silhouette. • XOR the images • compute histogram (hardware) • gives linear response to the displacement INFORMATIK 1 intensity x 0 difference 1 0 displacement x Hendrik Lensch

Approximation of Squared Distances INFORMATIK Use smooth transitions • blur images • integrate squared differences • • • faster convergence reduced variance higher evaluation cost Hendrik Lensch

Non-linear Optimization INFORMATIK Downhill Simplex Method [Press 1992] • • • works for N dimensions no derivatives easy to control Hendrik Lensch

Simplex Method in 3 D INFORMATIK original simplex reflection and/or expansion shrinking random perturbation Hendrik Lensch

Hierarchical Optimization • • INFORMATIK optimize on low resolution first restart optimization to avoid local minima switch to higher resolution mesh resolution can be adapted Hendrik Lensch

Starting Point Generation INFORMATIK • • set camera distance tz depending on object size set tx and ty to zero select 48 sample rotations run optimization for each of the samples (40 evaluations) • select top 5 results • restart optimization (200 evaluations) • take best result as starting point Hendrik Lensch

Texture Stitching INFORMATIK § projective texture mapping § assign one image to each triangle • triangle visible in image? (test every vertex) • select best viewing angle • discard data near depth discontinuities Hendrik Lensch

Blending Across Assignment Borders INFORMATIK • find border vertices • release all triangles around them • assign boundary vertices to best region • assign alpha-values for each region – 1 to vertices included in the region – 0 to all others. Hendrik Lensch

Entire Texture INFORMATIK Hendrik Lensch

Results and Conclusions INFORMATIK Problems solved: • • automatic texture registration (R, t, f) view-independent texture stitching blending across assignment boundaries rough manual alignment helps (speedup, failures) Further problems: • extract purely diffuse part of texture • generate texture where data is missing Hendrik Lensch

Questions? INFORMATIK visit us at www. mpi-sb. mpg. de Hendrik Lensch