Numerical geometry of nonrigid shapes In the Rigid

Numerical geometry of non-rigid shapes In the Rigid Kingdom 1 In the Rigid Kingdom Lecture 4 © Alexander & Michael Bronstein tosca. cs. technion. ac. il/book Numerical geometry of non-rigid shapes Stanford University, Winter 2009

Imagine a glamorous ball… Numerical geometry of non-rigid shapes In the Rigid Kingdom 2

A fairy tale shape similarity problem Numerical geometry of non-rigid shapes In the Rigid Kingdom 3

4 Numerical geometry of non-rigid shapes In the Rigid Kingdom Extrinsic shape similarity n Given two shapes and n Compare as subsets of the Euclidean space and , find the degree of their incongruence. . n Invariance to rigid motion: rotation, translation, (reflection): n is a rotation matrix, n is a translation vector

5 Numerical geometry of non-rigid shapes In the Rigid Kingdom How to get rid of Euclidean isometries? n How to remove translation and rotation ambiguity? n Find some “canonical” placement of the shape in . n Extrinsic centroid (a. k. a. center of mass, or center of gravity): n Set to resolve translation ambiguity. n Three degrees of freedom remaining…

Numerical geometry of non-rigid shapes In the Rigid Kingdom How to get rid of the rotation ambiguity? n Find the direction in which the surface has maximum extent. n Maximize variance of projection of n onto is the covariance matrix n Second-order geometric moments of n is the first principal direction : 6

Numerical geometry of non-rigid shapes In the Rigid Kingdom 7 How to get rid of the rotation ambiguity? n Project on the plane orthogonal to . n Repeat the process to find second and third principal directions .

8 Numerical geometry of non-rigid shapes In the Rigid Kingdom Canonical basis n span a canonical orthogonal basis for in .

9 Numerical geometry of non-rigid shapes In the Rigid Kingdom How to get rid of the rotation ambiguity? n Direction maximizing n and n = largest eigenvector of correspond to the second and third eigenvectors of admits unitary diagonalization n Setting axes . aligns . with the standard basis . n Principal component analysis (PCA), a. k. a. Karhunen-Loéve transform (KLT), or Hotelling transform. n Bottom line: the transformation brings the shape into a canonical configuration in . .

10 Numerical geometry of non-rigid shapes In the Rigid Kingdom Second-order geometric moments n Eigenvalues of are second-order moments n In the canonical basis, mixed moments n Ratio n Magnitudes of vanish. describe eccentricity of express shape scale. of. .

Numerical geometry of non-rigid shapes In the Rigid Kingdom Higher-order geometric moments n Second-order moments allow some discrimination. n Use higher-order moments gives more discrimination. n -th order moment n Computed in the canonical basis. n Invariant to rigid motion. n Signature of moments n A fingerprint of the extrinsic geometry of . 11

12 Numerical geometry of non-rigid shapes In the Rigid Kingdom A signal decomposition intuition n Moments are decomposition coefficients in the monomial basis n is a Dirac delta function elsewhere. n span . for and

Numerical geometry of non-rigid shapes In the Rigid Kingdom 13 A signal decomposition intuition n uniquely identify a shape (up to a rigid motion). can be reconstructed exactly from is the bi-orthonormal basis, i. e. n The monomial basis is not orthogonal. n The bi-orthonormal basis is ugly, but we do not need to reconstruct .

14 Numerical geometry of non-rigid shapes In the Rigid Kingdom Truncated signatures of moments n Compute the truncated moment signature n Construct a moments distance function, e. g. n A distance function on the shape of spaces. n Quantifies the extrinsic dissimilarity of and .

15 Numerical geometry of non-rigid shapes In the Rigid Kingdom Moments distance n is small for nearly congruent n is large for strongly non-congruent n If and are truly congruent, n However, congruent (unless does not imply that . and are ). n Which shapes are indistinguishable by ? n Ideally, congruent at a coarse resolution (“low frequency”) and differing in fine details (“high frequency”). n Degree of coarseness is controlled by the moments order n Geometric moments do not satisfy this requirement. .

Numerical geometry of non-rigid shapes In the Rigid Kingdom Other moments n Instead of the monomial basis, other bases can be chosen n Fourier basis n Spherical harmonics, Zernike polynomials, wavelets, etc. 16

Numerical geometry of non-rigid shapes In the Rigid Kingdom 17 Moments of joy, moments of sorrow Joy: n Shape similarity is translated to similarity of moment signatures. n Comparison of moments signatures is fast (e. g. Euclidean distance). Sorrow: n Do not allow for partial similarity!

18 Numerical geometry of non-rigid shapes In the Rigid Kingdom Iterative closest point (ICP) algorithms n Given two shapes bringing n and , find the best rigid motion as close as possible to : is some shape-to-shape distance. n Minimum = extrinsic dissimilarity of and n Minimizer = best rigid alignment between . and . n ICP is a family of algorithms differing in n The choice of the shape-to-shape distance. n The choice of the numerical minimization algorithm.

19 Numerical geometry of non-rigid shapes In the Rigid Kingdom Shape-to-shape distance n The Hausdorff distance n the shape and is the distance between a point and . n the shape is the distance between a point .

Numerical geometry of non-rigid shapes In the Rigid Kingdom Shape-to-shape distance n A non-symmetric version is preferred to allow for partial similarity n The n Use the n (max-min) formulation is sensitive to outliers. variant is a point-to-shape distance. n Different possibilities to define . 20

21 Numerical geometry of non-rigid shapes In the Rigid Kingdom Point-to-point distance n Treat as a cloud of points. n Find the closest point n Define the distance as to on .

Numerical geometry of non-rigid shapes In the Rigid Kingdom Point-to-plane distance n Treat n as a plane, and define the point-to-plane distance is the normal to the surface at point n Can be approximated as . 22

Numerical geometry of non-rigid shapes In the Rigid Kingdom Second-order point-to-shape distance n Point-to-plane distance is a first-order approximation of the true point-to-shape distance. n Construct a second-order approximation are the principal curvature radii at . are the principal directions. is the signed distance to the closest point. 23

Numerical geometry of non-rigid shapes In the Rigid Kingdom Second-order point-to-shape distance n The second-order distance approximant may become negative for some values of . n Use a non-negative quadratic approximant 24

Numerical geometry of non-rigid shapes In the Rigid Kingdom Second-order point-to-shape distance n “Near-field” case – point-to-plane distance n “Far-field” case – point-to-point distance 25

Numerical geometry of non-rigid shapes In the Rigid Kingdom 26 Second-order point-to-shape distance n Second-order distance generalizes the point-to-point and the point-toplane distances. n Gives more accurate alignment between shapes. n Requires principal curvatures and directions (second-order quantities).

Numerical geometry of non-rigid shapes In the Rigid Kingdom Iterative closest point algorithm n Initialize n Find the closest point correspondence n Minimize the misalignment between corresponding points n Update n Iterate until convergence… 27

28 Numerical geometry of non-rigid shapes In the Rigid Kingdom Closest points n How to find closest points efficiently? n Straightforward complexity: number of points on n , number of points on . divides the space into Voronoi cells n Given a query point , determine to which cell it belongs.

Numerical geometry of non-rigid shapes In the Rigid Kingdom Closest points 29

30 Numerical geometry of non-rigid shapes In the Rigid Kingdom Approximate nearest neighbors n To reduce search complexity, approximate Voronoi cells. n Use binary space partition trees (e. g. kd-trees or octrees). n Approximate nearest neighbor search complexity: .

Numerical geometry of non-rigid shapes In the Rigid Kingdom Best alignment n Given two sets and of corresponding points. n Find best alignment n A numerical minimization algorithm can be used. n For some point-to-shape distances, a closed-form solution exists. 31

Numerical geometry of non-rigid shapes In the Rigid Kingdom ® MATLAB intermezzo Iterative closest point algorithm 32

33 Numerical geometry of non-rigid shapes In the Rigid Kingdom Until convergence… n ICP should find the solution of n Instead, it solves n Correspondence fixed to instead of . n Not guaranteed to produce a monotonically decreasing sequence of values of . n Not guaranteed to converge!

Numerical geometry of non-rigid shapes In the Rigid Kingdom Enter numerical optimization n Treat as a numerical minimization problem. n Express the distance terms as a quadratic function is a 3× 3 symmetric positive definite matrix, is 3× 1 vector, and is a scalar. 34

Numerical geometry of non-rigid shapes In the Rigid Kingdom Local quadratic approximant n Point-to-point distance: n Point-to-plane distance: 35

Numerical geometry of non-rigid shapes In the Rigid Kingdom Local quadratic approximant n Minimize over . n Dependence of n For small motion and on might be complicated. , hence 36

Numerical geometry of non-rigid shapes In the Rigid Kingdom Minimization variables n is required to be unitary (orthonormal). n Enforcing orthonormality is cumbersome. n Minimization w. r. t. to the rotation angles involves nonlinear functions. n Under small motion assumption, n Linearize rotation matrix 37

38 Numerical geometry of non-rigid shapes In the Rigid Kingdom Let Newton be! n Linearized rotation yields a quadratic objective w. r. t n Use a Newton step to find the steepest descent direction. n Approximation is valid only for small steps. n Use Armijo rule to find a fractional step ensuring sufficient decrease of objective function. n What is a fractional step? .

39 Numerical geometry of non-rigid shapes In the Rigid Kingdom Fractional step n Let n n Hence be a small transformation, which applied is a rotation by . times gives .

Numerical geometry of non-rigid shapes In the Rigid Kingdom Iterative closest point algorithm revisited n Initialize n Find closest point correspondence n Construct local quadratic approximant of n Find Newton direction n Use Armijo rule to find such that n Update n Iterate until convergence… 40

41 Numerical geometry of non-rigid shapes In the Rigid Kingdom Iterative closest point algorithm revisited n Coefficients of the quadratic approximant can be computed on demand using efficient nearest neighbor search. n Alternative: approximate the values of using a space partition tree. in the space