Automatic Posing of a Meshed Human Model Using

  • Slides: 19
Download presentation
Automatic Posing of a Meshed Human Model Using Point Clouds Lei Wang Joint work

Automatic Posing of a Meshed Human Model Using Point Clouds Lei Wang Joint work with Tamal K. Dey, Huamin Wang and Bo Fu DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Problem Statement • Low-end scanning devices are becoming popular. But quality of their output…

Problem Statement • Low-end scanning devices are becoming popular. But quality of their output… • Reconstruct from problematic human scans by deforming a prior high quality template mesh DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Related Work • Model-based Registration • Parametric human body models • Registration without a

Related Work • Model-based Registration • Parametric human body models • Registration without a Model • Manual intervention • Non-rigid registration by nonlinear optimization • Isometric deformations: keep geodesic distance unchanged DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Our Method: Outline computing correspondences DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING posing output

Our Method: Outline computing correspondences DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING posing output

Computing Correspondences DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Computing Correspondences DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Computing Correspondences • The Global Point Signature (GPS) framework by [Rustamov 2007]: • Apply

Computing Correspondences • The Global Point Signature (GPS) framework by [Rustamov 2007]: • Apply Gaussian-weighted Graph Laplacian on the adjacency graph of input point cloud Graph Laplacian GPS in dimension 1, 2 and 3 DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Computing Correspondences • GPS is invariant under isometric deformation • We choose only the

Computing Correspondences • GPS is invariant under isometric deformation • We choose only the first three eigenfunctions DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Computing Correspondences • Correspondences are expanded from 5 extremums of aligned GPS embeddings DEPARTMENT

Computing Correspondences • Correspondences are expanded from 5 extremums of aligned GPS embeddings DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Posing Template Mesh • Formulated as an energy minimization problem: • is the internal

Posing Template Mesh • Formulated as an energy minimization problem: • is the internal energy of template mesh ○ measures difference between template and the point cloud • Solved by Invertible FEM [Irving et al. 2004] DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Posing Template Mesh • It has two stages Initial alignment: deformed by correspondence DEPARTMENT

Posing Template Mesh • It has two stages Initial alignment: deformed by correspondence DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING Point cloud alignment: deformed by input point cloud

Posing Template Mesh • For efficiency, posing is performed on a simplified template mesh,

Posing Template Mesh • For efficiency, posing is performed on a simplified template mesh, called control mesh • Deformed template is recovered by Mean Value Coordinates [Ju et al. 2005], i. e. , weighted sum of the control mesh DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Results DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Results DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Results DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Results DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Results DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Results DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Results (a) input scan (b) our method DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING (c)

Results (a) input scan (b) our method DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING (c) SCAPE

Results • DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING can be used to control the

Results • DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING can be used to control the body size

Supplementary Video DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Supplementary Video DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Discussion • An automatic approach to align a detailed template mesh with human point

Discussion • An automatic approach to align a detailed template mesh with human point clouds in different poses • Robust to noise and occlusions • Sensitive to topological change • Can not handle details like fingers • Does not run in real-time DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Q&A Thank you! DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Q&A Thank you! DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING