Rotation Invariant Spherical Harmonic Representation of 3 D

Rotation Invariant Spherical Harmonic Representation of 3 D Shape Descriptors Michael Kazhdan Thomas Funkhouser Szymon Rusinkiewicz Princeton University

Motivation Large databases of 3 D models Computer Graphics (Princeton 3 D Search Engine) Mechanical CAD (National Design Repository) Molecular Biology (Audrey Sanderson)

Retrieval Approach 3 D Model Shape Descriptor Nearest Neighbor Model Database

Shape Unchanged by Rotation =

Problem Many shape descriptors are functions that rotate with the shape Extended Gaussian Image [Horn ’ 84] Spherical Attribute Image [Ikeuchi ’ 95] Shape Histogram [Ankerst ’ 99] Spherical Extent Function [Vranic ’ 00] Reflective Symmetry Descriptor [Kazhdan ’ 02] Gaussian EDT [Funkhouser ’ 03]

Goal Compute similarity of shape descriptors independent of rotation - = ?

Brute Force Approach Impractical for databases - - min =(rotation) -

Normalization Use PCA to place models into a canonical coordinate frame Covariance Matrix Computation Principal Axis Alignment

Normalization Doesn’t always work • Only second order information

Our Approach Eliminate rotation dependence in spherical and 3 D descriptors Shape Descriptor EGI [Horn ’ 84] SAI [Ikeuchi ’ 95] EXT [Vranic ’ 00] RSD [Kazhdan ’ 02] EDT [Funkhouser ’ 03] etc.

Our Approach Eliminate rotation dependence in spherical and 3 D descriptors Shape Descriptor Rotation Invariant Representation

Outline Introduction Background Harmonic Representation Properties Experimental Results Conclusion and Future Work

Key Idea Obtain rotation invariant representation by storing amplitude and eliminating phase = [Lo 1989] [Burel 1995] + + …

Fourier Descriptors Circular Function

Fourier Descriptors = + + + Circular Function Cosine/Sine Decomposition + …

Fourier Descriptors = + + + Circular Function = Constant Frequency Decomposition + …

Fourier Descriptors = + + + + Circular Function Constant 1 st Order Frequency Decomposition + …

Fourier Descriptors = + + + + Circular Function Constant 1 st Order 2 nd Order Frequency Decomposition + …

Fourier Descriptors = + + + + + … Circular Function Constant 1 st Order 2 nd Order 3 rd Order Frequency Decomposition

Fourier Descriptors = Amplitudes invariant + + + to rotation + + … Circular Function + = Constant + 1 st Order + 2 nd Order 3 rd Order Frequency Decomposition

Harmonic Representation Spherical Function

Harmonic Representation = + + + Spherical Function Harmonic Decomposition + …

Harmonic Representation = + + … Spherical Function Constant = 1 st Order + 2 nd Order + 3 rd Order + + …

Harmonic Representation Store “how much” (L 2 -norm) of the shape resides in each frequency Norms Invariant to Rotation = + + …

3 D Function (Voxel Grid) Restrict to concentric spheres

3 D Function (Voxel Grid) Compute harmonic representation of each sphere independently = + + + + = + +

3 D Function (Voxel Grid) Combine harmonic representations cy en u eq Fr Ra s u i d

Matching Harmonic Representation 2 L 2 -difference of harmonic representations…

Matching min - (rotations) 2 - … bounds proximity of descriptors over all rotations 2

Outline Introduction Background Harmonic Representation Properties Experimental Results Conclusion and Future Work

Advantages The harmonic representations is: • Rotation invariant • Multi-resolution • Compact • Discriminating

Compact … … …

Compact … … …

Compact … … …

Compact … … …

Compact … … …

Information Loss • Intra-frequency information loss • Cross-radial information loss

Information Loss (Spherical Descriptor) • Intra-frequency information loss • Cross-frequency information loss

Information Loss (Spherical Descriptor) • Intra-frequency information loss • Cross-frequency information loss + = 22. 5 o 90 o = +

Information Loss (3 D Descriptor) • Cross-radial information loss

Outline Introduction Background Harmonic Representation Properties Experimental Results Conclusion and Future Work

Shape Descriptors Extended Gaussian Image Horn 1984 Spherical Extent Function Vranic 2000 Shape Histogram Ankerst 1999 Gaussian EDT Funkhouser 2003

Experimental Database Viewpoint “household” database 1, 890 models, 85 classes 153 dining chairs 25 livingroom chairs 16 beds 12 dining tables 8 chests 28 bottles 39 vases 36 end tables

Gaussian EDT Results PCA-Normalized Results 1 2 3 4 5 6 7 8 9 10 Harmonic Representation Results 1 2 3 4 5 6 7 8 9 10 Query

Gaussian EDT Results Precision vs. Recall Precision 100% Harmonics PCA 50% 0% 0% 50% Recall 100%

Retrieval Results Precision Harmonics PCA 50% SECT 100% Precision EGI 100% Harmonics PCA • EGI: Extended Gaussian Image • SECT: Shape 50% Histogram (Sectors) • EXT: Spherical 0% Recall 100% EXT 100% Precision 50% Harmonics PCA 50% 0% 0% 50% Recall 0% 100% 0% 50% Recall 100% Harmonics PCA 50% 0% 0% 50% Recall Extent Function • EDT: Gaussian EDT 100% Precision 0% 100% Euclidean Distance Transform

Retrieval Results Precision Harmonics PCA 50% SECT 100% Precision EGI 100% Harmonics PCA • EGI: Extended Gaussian Image • SECT: Shape 50% Histogram (Sectors) • EXT: Spherical 0% Recall 100% EXT 100% Precision 50% Harmonics PCA 50% 0% 0% 50% Recall 0% 100% 0% 50% Recall 100% Harmonics PCA 50% 0% 0% 50% Recall Extent Function • EDT: Gaussian EDT 100% Precision 0% 100% Euclidean Distance Transform

Exhaustive Gaussian EDT Results Precision 100% Gaussian EDT - min L 2 Harmonic PCA min (rotation) 50% - 0% 0% 50% Recall 100%

Summary and Conclusion Provide a rotation invariant representation of shape descriptors that: • Eliminates PCA dependence • Gives better matching performance • Is more compact • Is a multi-resolution representation

Future Work Managing Information Loss • Obtain cross radial information for 3 D descriptors • Obtain cross frequency information • Get finer resolution of rotation invariance within frequencies More Generally • Consider new shape descriptors

Thank You Funding National Science Foundation Sloan Foundation Spherical Harmonics Dan Rockmore and Peter Kostelec Spharmonic. Kit: http: //www. cs. dartmouth. edu/~geelong/sphere 3 D Shape Matching Patrick Min, Alex Halderman, Phil Shilane, David Jacobs, Joyce Chen Princeton 3 D Model Search Engine: http: //shape. cs. princeton. edu