ShapeRepresentation and Shape Similarity Part 1 Shapes Dr
- Slides: 102
Shape-Representation and Shape Similarity Part 1: Shapes Dr. Rolf Lakaemper
May I introduce myself… • Rolf Lakaemper • Ph. D (Doctorate Degree) 2000 Hamburg University, Germany • Currently Assist. Professor at Department of Computer and Information Sciences, Temple University, Philadelphia, USA • Main Research Area: Computer Vision
Motivation WHY SHAPE ?
Motivation These objects are recognized by…
Motivation These objects are recognized by… Texture Color X X Context X Shape X X X
Why Shape ? Several applications in computer vision use shape processing: • Object recognition • Image retrieval • Processing of pictorial information • Video compression (eg. MPEG-7) … This presentation focuses on object recognition and image retrieval.
Motivation Typical Application: Multimedia: Image Database Query by Shape / Texture / …(Color / Keyword)
Blobworld Example 1: Blobworld http: //elib. cs. berkeley. edu/photos/blobworld/start. html BLOB = “Binary Large Object”, “an indistinct shapeless (really ? ) form”
Blobworld: Query by Shape / Texture / Location / Color Selected Blob Query: by Color and Texture of Blob Result: Blobs with similar Color and Texture Satisfying ?
Blobworld: Query by Shape / Texture / Location / Color Selected Blob Query: by Shape of Blob Result: …are these shapes similar ? Satisfying ?
Blobworld Result: SHAPE recognition seems to be necessary but not easy !
ISS Database Example 2: ISS-Database http: //knight. cis. temple. edu/~shape
The Interface (JAVA – Applet)
The Sketchpad: Query by Shape
The First Guess: Different Shape - Classes
Selected shape defines query by shape – class
Result
ISS Database ISS: Query by Shape / Texture Sketch of Shape Query: by Shape only Result: Satisfying ?
ISS Database SHAPE recognition seems to be possible and leads to satisfying results !
ISS Database The ISS-Database will be topic of part IV of this tutorial …so stay alert !
Overview Part 1 • • Why shape ? What is shape ? Shape similarity Metrices Classes of similarity measures Feature Based Coding Examples for global similarity
Why Shape ? • Shape is probably the most important property that is perceived about objects. It allows to predict more facts about an object than other features, e. g. color (Palmer 1999) • Thus, recognizing shape is crucial for object recognition. In some applications it may be the only feature present, e. g. logo recognition
Why Shape ? Shape is not only perceived by visual means: • • tactical sensors can also provide shape information that are processed in a similar way. robots’ range sensor provide shape information, too.
Shape Typical problems: • How to describe shape ? • What is the matching transformation? • No one-to-one correspondence • Occlusion • Noise
Shape • Partial match: only part of query appears in part of database shape
What is Shape ? Plato, "Meno", 380 BC: • "figure is the only existing that is found always following color“ • "figure is limit of solid"
What is Shape ? … let’s start with some properties easier to agree on: • Shape describes a spatial region Shape is a (the ? ) specific part of spatial cognition • Typically addresses 2 D space why ?
What is Shape ? • 3 D => 2 D projection
What is Shape ? • the original 3 D (? ) object
What is Shape ? Moving on from the naive understanding, some questions arise: • Is there a maximum size for a shape to be a shape? • Can a shape have holes? • Does shape always describe a connected region? • How to deal with/represent partial shapes (occlusion / partial match) ?
What is Shape ? Shape or Not ? Continuous transformation from shape to no shape: Is there a point when it stops being a shape?
What is Shape ? Shape or Not ? Continuous transformation from shape to two shapes: Is there a point when it stops being a single shape?
What is Shape ? But there’s no doubt that a single, connected region is a shape. Right ?
What is Shape ? A single, connected region. But a shape ? A question of scale !
What is Shape ? • There’s no easy, single definition of shape • In difference to geometry, arbitrary shape is not covered by an axiomatic system • Different applications in object recognition focus on different shape related features • Special shapes can be handled • Typically, applications in object recognition employ a similarity measure to determine a plausibility that two shapes correspond to each other
Similarity So the new question is: What is Shape Similarity ? or How to Define a Similarity Measure
Similarity Again: it’s not so simple (sorry). There’s nothing like THE similarity measure
Similarity which similarity measure, depends on which required properties, depends on which particular matching problem, depends on which application
Similarity …which application Simple Recognition (yes / no) . . . robustness Common Rating (best of. . . ) Analytical Rating (best of, but. . . ) . . . invariance to basic transformations
Similarity …which problem • computation problem: d(A, B) • decision problem: d(A, B) <e ? • decision problem: is there g: d(g(A), B) <e ? • optimization problem: find g: min d(g(A), B)
Similarity …which properties: We concentrate here on the computational problem d(A, B)
Similarity Measure Requirements to a similarity measure • Should not incorporate context knowledge (no AI), thus computes generic shape similarity
Similarity Measure Requirements to a similarity measure • Must be able to deal with noise • Must be invariant with respect to basic transformations Scaling (or resolution) Next: Strategy Rotation Rigid / non-rigid deformation
Similarity Measure Requirements to a similarity measure • Must be able to deal with noise • Must be invariant with respect to basic transformations • Must be in accord with human perception
Similarity Measure Some other aspects worth consideration: • Similarity of structure • Similarity of area Can all these aspects be expressed by a single number?
Similarity Measure Desired Properties of a Similarity Function C (Basri et al. 1998) • C should be a metric • C should be continous • C should be invariant (to…)
Properties Metric Properties S set of patterns Metric: d: S ´ S ® R satisfying 1. Self-identity: " xÎS, d(x, x)=0 2. Positivity: " x ¹yÎS, d(x, y)>0 3. Symmetry: " x, yÎS, d(x, y)= d(y, x) 4. Triangle inequality: " x, y, zÎS, d(x, z)£d(x, y)+d(y, z) • Semi-metric: 1, 2, 3 • Pseudo-metric: 1, 3, 4 • S with fixed metric d is called metric space
Properties 1. Self-identity: " xÎS, d(x, x)=0 2. Positivity: " x ¹yÎS, d(x, y)>0 1. …surely makes sense
Properties
Properties
Properties In general: • a similarity measure in accordance with human perception is NOT a metric. This leads to deep problems in further processing, e. g. clustering, since most of these algorithms need metric spaces !
Properties Continuity: “usually useful”, although sometimes not in accordance with principles of Gestalt properties, e. g. symmetry, collinearity.
Properties
Properties
Properties Some more properties: • One major difference should cause a greater dissimilarity than some minor ones. • S must not diverge for curves that are not smooth (e. g. polygons). However, these demands are contradictory (proof is left as an exercise)
Similarity Measures Classes of Similarity Measures: Similarity Measure depends on • Shape Representation • Boundary • Area (discrete: = point set) • Structural (e. g. Skeleton) • Comparison Model • feature vector • direct
Similarity Measures direct feature based Boundary Spring model, Cum. Angular Function, Chaincode, Arc Decomposition (ASRAlgorithm) Central Dist. Fourier Distance histogram … Area (point set) Hausdorff … Moments Zernike Moments … Structure Skeleton … ---
Feature Based Coding This category defines all approaches that determine a feature-vector for a given shape. Two operations need to be defined: a mapping of shape into the feature space and a similarity of feature vectors. Representation Feature Extraction Vector Comparison
Vector Comparison ,
Vector Comparison
Vector Comparison
Vector Comparison
Vector Comparison More Vector Distances: • Quadratic Form Distance • Earth Movers Distance • Proportional Transportation Distance • …
Vector Comparison Histogram Comparison • Vector Comparison • Histogram Intersection • …
Vector Comparison
Feature Based Coding Again: Two operations need to be defined: a mapping of shape into the feature space and a similarity of feature vectors. We hence have TWO TIMES an information reduction of the basic representation, which by itself is already a mapping of the ‘reality’. Representation Feature Extraction Vector Comparison
Vector Comparison Example 1 Elementary Descriptors Shape A, B given as • Area (continous) or • Point Sets (discrete) (Elementary Descriptors are 1 dimensional feature vectors)
Vector Comparison
Vector Comparison Boundary Box (area): These shapes are equal…
Vector Comparison Boundary Box (area): …these shapes differ
Vector Comparison Example 2 (Discrete) Moments Shape A, B given as • Area (continous) or • Point Sets (discrete)
Moments Discrete Point Sets
Moments
Moments
Moments
Discrete Moments Exercise: Please compute all 7 moments for the following shapes, compare the vectors using different comparison techniques
Discrete Moments Result: each shape is transformed to a 7 dimensional vector. To compare the shapes, compare the vectors (how ? ).
Vector Comparison Example 3 Central Distance Fourier Shape A, B given as Contour (Boundary)
Vector Comparison Contour is given as list of euclidean coordinates: 0, 0; 1, 0; 2, 1; 2, 2; 3, 3; 4, 3; 5, 2; … 0 1 2 3 4 7. . . 5 6 . . .
Central Distance Fourier (MATLAB DEMO)
3 D Distance Histogram Example 4 3 D Distance Histogram Shape A, B given as 3 D point set
3 D Distance Histogram (MATLAB DEMO)
Vector Comparison All Feature Vector approaches have similar properties: • Provide a compact representation • this is especially interesting for database indexing ! • Works for any shape • Requires complete shapes (global comparison) • Sensible to noise (except Zernike moments which are computationally demanding) • Map dissimilar shapes to similar feature vectors (!) • They can be used as a prefilter for database applications ! • Make the choice of a similarity function difficult
Direct Comparison End of Feature Based Coding ! Next: Direct Comparison
Vector Direct Comparison Example 1 Hausdorff Distance Shape A, B given as point sets A={a 1, a 2, …} B={b 1, b 2, …}
Feature Vector Based Comparison Coding
Vector Hausdorff Comparison Distance
Vector Comparison Boundary Representation Hausdorff: Unstable with respect to noise (This is easy to fix ! How ? ) Problem: Invariance ! Nevertheless: Hausdorff is the motor behind many applications in specific fields (e. g. character recognition)
Vector Comparison Boundary Representation Example 2 Chaincode Comparison Shape A, B given as chaincode
Vector Comparison Boundary Representation
Vector Comparison Boundary Representation A binary image can be converted into a ‘chain code’ representing the boundary. The boundary is traversed and a string representing the curvature is constructed. 3 2 1 4 C 0 5 6 7 5, 6, 6, 3, 3, 4, 3, 2, 3, 4, 5, 3, …
Vector Comparison Boundary Representation For curvature classes, a similarity can be defined:
Vector Comparison Boundary Representation To extend this measure to strings, two steps are carried out. 1. Extend the measure to character against string, for example by summing up individual similarity measures. 2. Employing a matching to compute a correspondence of sub-strings. Hereby, the matching constitutes from 1 -to-1, 1 to-many, and many-to-1 -matchings. It is computed as string matching by means of dynamic programming.
Vector Comparison Boundary Representation Digital curves suffer from effects caused by digitalization, e. g. rotation:
Vector Comparison Boundary Representation Compare chaincodes by string matching As string-matching is not able to model a matching of digital curves adequately, more sophisticated matching algorithms are employed in “real applications” using chain codes: Weighted Levensthein Distance Defines an edit distance for transforming one string into another. Costs are defined for altering, deleting, or inserting a character. Extended Distance Formal translation system with costs assigned to individual production rules.
Structural Vector Comparison Representation Example 3 Skeletons Shape A, B primarily given as area or boundary, structure is derived from representation
Structural Vector Comparison Representation Structural approaches capture the structure of a shape, typically by representing shape as a graph. Typical example: skeletons
Structural Vector Comparison Representation The computation can be described as a medial axis transform, a kind of discrete generalized voronoi.
Structural Vector Comparison Representation The graph is constructed mirroring the adjacency of the skeleton’s parts. Edges are labeled according to the qualitative classes. Matching two shapes requires matching two usually different graphs against each other.
Structural Vector Comparison Representation Problems of skeletons: - Pruning
Structural Vector Comparison Representation -Robustness (MATLAB Demo)
Vector Shape Comparison similarity All similarity measures shown can not deal with occlusions or partial matching (except skeletons ? ) ! They are useful (and used) for specific applications, but are not sufficient to deal with arbitrary shapes Solution: Part – based similarity !
- Shapes that seem to follow no rules.
- Shape matching and object recognition using shape contexts
- Bolongie
- How to tell if shapes are similar
- Similar shapes and proportions lesson 4-1
- Cessna airfoil
- Each shape is 1 whole
- Part whole model subtraction
- Part to part ratio definition
- Brainpop ratios
- What is a technical description
- Parts of cocktail bar
- The part of a shadow surrounding the darkest part
- 미니탭 gage r&r 해석
- Sas warm up
- Similarities of parents and offspring
- Dilations
- Similarity and proportions
- Competitive dynamics examples
- Dilations and similarity
- Triangle similarity: aa assignment
- 7-2 similarity transformations
- Similarity and dissimilarity measures in data mining
- Proportions and similarity
- Vedf
- Similarity between prokaryotic and eukaryotic cells
- What is one similarity between ghana and mali
- Triangle similarity aa quiz
- 7 3 proving triangles similar
- 7-2 properties of proportions answers
- Vector quantities measure
- Fly-in
- Market commonality and resource similarity examples
- Tact extensions aba
- Smc vs jaccard
- Uncircumcised penis
- Dilations and similarity
- Standard cycle market example
- Dilations and similarity in the coordinate plane
- Five elements of hair design
- Contrasting lines in hair
- Comparison and contrast paragraph examples
- Unit 2 lesson 3 proving triangles similar
- Similarity heuristic
- Maze isosceles and equilateral triangles answer key
- Postulate
- Similarity flooding
- Similarity theorem examples
- Aa similarity theorem
- Similarity transformation
- Similarity of ending sounds existing between two words
- Earth similarity index
- Sequence identity vs similarity
- Similarity in right triangles
- Measure of software similarity
- Phonemic transcription
- Dot summary
- David laibson
- Law of similarity
- 7-3 similar triangles aa similarity
- Leg rule right triangle
- Corr sides similar triangle
- Sas similarity theorem examples
- Similarity theorem examples
- Aa similarity postulate
- Aa postulate
- Side side side similarity
- Laplacian matrix
- Sequence identity vs similarity
- Ratio and proportion for use after section 7-2
- Projected cognitive similarity is the tendency to
- Extended ratio geometry definition
- 9-7 similarity transformations
- Similarity in right triangles
- 8-1 similarity in right triangles
- 8-1 similarity in right triangles
- 7-4 similarity in right triangles
- 7-3 triangle similarity aa sss sas
- Ways to prove triangles similar
- Phonetic similarity example
- Triangle similarity aa quiz
- 9-4 similarity in right triangles
- Lesson 12 modeling using similarity
- What is the range of similarity measure
- Mining user similarity based on location history
- Lesson 7-5 proportions in triangles
- Limitations of hydraulic similitude
- Aa similarity
- Vedf
- Similarity cartoon
- If abe acd by the sas similarity theorem what is ad
- 8-1 similarity in right triangles
- 7-3 triangle similarity
- Similar polygons definition geometry
- Geometric similarity search
- Visual similarity
- Image similarity measure
- Cohesive devices used to signal consequences
- Proportion of similarity
- Meaning
- Image similarity search
- Image similarity search
- 1:4 scale