Learning Decompositional Shape Models from Examples Alex Levinshtein
- Slides: 30
Learning Decompositional Shape Models from Examples Alex Levinshtein Cristian Sminchisescu Sven Dickinson University of Toronto
Hierarchical Models Manually built hierarchical model proposed by Marr And Nishihara (“Representation and recognition of the spatial organization of three dimensional shapes”, Proc. of Royal Soc. of London, 1978)
Our goal Automatically construct a generic hierarchical shape model from exemplars Challenges: § Cannot assume similar appearance among different exemplars § Generic features are highly ambiguous § Generic features may not be in one-to-one correspondence
Layered Motion Segmentations Kumar, Torr and Zisserman, ICCV 2005 n Models image projection, lighting and motion blur n Models spatial continuity, occlusions, and works over multiple frames ( cf. earlier work by Jojic & Frey, CVPR 2001) n Estimates the number of segments, their mattes, layer assignment, appearance, lighting and transformation parameters for each segment n Initialization using loopy BP, refinement using graph cuts
Constellation models Fergus, R. , Perona, P. , and Zisserman, A. , “Object Class Recognition by Unsupervised Scale-Invariant Learning”, CVPR 2003
Categorical features Match
Input: Automatically constructed Hierarchical Models Question: What is it? Output:
Stages of the system Exemplar images Extract Blob Graphs Blob graphs Match Blob Graphs (many-to-many) Many-to-many correspondences Extract Parts Extract Decomposition Relations Model decomposition relations Model parts Assemble Final Model Extract Attachment Relations Model attachment relations
Blob Graph Construction Exemplar images Extract Blob Graphs Blob graphs Match Blob Graphs (many-to-many) Many-to-many correspondences Extract Parts Extract Decomposition Relations Model decomposition relations Model parts Assemble Final Model Extract Attachment Relations Model attachment relations
Blob Graph Construction the Representation and of Qualitative Shape at Multiple Scales §On. Edges are invariant to Matching articulation § A. Shokoufandeh, S. Dickinson, C. Jonsson, L. Bretzner, and T. Lindeberg, Choose the largest connected component. ECCV 2002
Blob Graph Construction Perceptual grouping of blobs: Connectivity measure: max{d 1/major(A), d 2/major(B)}
Feature matching Exemplar images Extract Blob Graphs Blob graphs Match Blob Graphs (many-to-many) Many-to-many correspondences Extract Parts Extract Decomposition Relations Model decomposition relations Model parts Assemble Final Model Extract Attachment Relations Model attachment relations
Feature matching One-to-one matching. Rely on shape and context, not appearance! Many-to-many matching
A Many-to-Many Graph Matching Framework 1. Embed graphs with low distortion to yield weighted point distributions. 2. Compute many-to-many correspondences between the two distributions using EMD. 3. The computed flows yield a many-to-many node correspondence between the two graphs. Demirci, Shokoufandeh, Dickinson, Keselman, and Bretzner (ECCV 2004)
Feature embedding and EMD Spectral embedding
Returning to our set of inputs § Many-to-many matching of every pair of exemplars.
Part Extraction Exemplar images Extract Blob Graphs Blob graphs Match Blob Graphs (many-to-many) Many-to-many correspondences Extract Parts Extract Decomposition Relations Model decomposition relations Model parts Assemble Final Model Extract Attachment Relations Model attachment relations
Many-to-many matching results 100% 50%
Results of the part extraction stage
Extracting attachment relations Exemplar images Extract Blob Graphs Blob graphs Match Blob Graphs (many-to-many) Many-to-many correspondences Extract Parts Extract Decomposition Relations Model decomposition relations Model parts Assemble Final Model Extract Attachment Relations Model attachment relations
Extracting attachment relations Number of times blobs drawn from the two clusters were attached is high Rightofarm isblobs typically to torso in exemplar images Number times from connected the two clusters co-appeared in an image. Right Arm Torso !
Extracting decomposition relations Exemplar images Extract Blob Graphs Blob graphs Match Blob Graphs (many-to-many) Many-to-many correspondences Extract Parts Extract Decomposition Relations Model decomposition relations Model parts Assemble Final Model Extract Attachment Relations Model attachment relations
Extracting decomposition relations Left Arm Upper Lower
Assemble Final Model Exemplar images Extract Blob Graphs Blob graphs Match Blob Graphs (many-to-many) Many-to-many correspondences Extract Parts Extract Decomposition Relations Model decomposition relations Model parts Assemble Final Model Extract Attachment Relations Model attachment relations
Results
Conclusions n n n Generic models must be defined at multiple levels of abstraction, as Marr proposed. Coarse shape features, such as blobs, are highly ambiguous and cannot be matched without contextual constraints. Moreover, features that exist at different levels of abstraction must be matched many-to-many in the presence of noise. The many-to-many matching results can be analyzed to yield both the parts and relations of a decompositional model. Preliminary results indicate that a limited decompositional model can be learned from a set of noisy examples.
Future work n n n Construct models for objects other than humans – objects with richer decompositional hierarchies. Automatically learn perceptual grouping relations between blobs from labeled examples. Develop indexing and matching frameworks for decompositional models.
- Alex levinshtein
- Modals and semimodals
- Cuadro comparativo e-learning y b-learning
- Drag divergence mach number
- Shape matching and object recognition using shape contexts
- Template matching
- Self-paced learning for latent variable models
- Computer vision models learning and inference
- Gerard schuster
- Computer vision models learning and inference
- Computer vision: models, learning, and inference pdf
- Deploying deep learning models with docker and kubernetes
- Geometric models in machine learning
- Using inaccurate models in reinforcement learning
- Inversion attacks
- Education styles
- Stealing machine learning models via prediction apis
- Computer vision: models, learning, and inference
- Examples of physical models
- Identify the key components of e-commerce business models
- Using algebra to solve problems
- Aristotle model of communication pros and cons
- Example of graphical model
- "deep reinforcement learning"
- Concept learning task in machine learning
- Analytical learning in machine learning
- Non associative learning
- Eager learner and lazy learner
- Define conceptual learning
- Analytical learning vs inductive learning
- Apprenticeship learning via inverse reinforcement learning