Shapebased Image Retrieval 4 Image Retrieval by Shape

Shape-based Image Retrieval

4 Image Retrieval by Shape Similarity

Shape Here: shape is geometry

QBIC – Search by shape ** Images courtesy : Yong Rao

Region-based similar Contour based similar

Region Finding（segmentation) • Instead of (or in addition to) detecting edges in an image, we can also search for homogeneous regions as the basis for scene analysis. • In such regions, the intensity, texture, or other features do not change abruptly (突然地). • Often, both edge detection and region finding are used to complement each other.

Region Finding • Definition of a region: • A region is a set of connected pixels meeting two requirements: • A region is homogeneous. This could be defined as, for example, the maximum difference in intensity values between two pixels being no greater than some threshold . （内部特性） • The union of the pixels of two adjacent regions never satisfies the homogeneity criterion. （不可再扩 大性）

The Split-and-Merge Algorithm • First, we perform splitting: • At the start of the algorithm, the entire image is considered as the candidate region. • If the candidate region does not meet the homogeneity criterion, we split it into four smaller candidate regions. • This is repeated until there are no candidate regions to be split any more. • Then, we perform merging: • Check all pairs of neighboring regions and merge them if it does not violate the homogeneity criterion.

The Split-and-Merge Algorithm 1 1 3 1 1 1 1 4 8 1 1 9 8 1 1 8 4 1 1 2 0 0 0 1 1 1 1 6 5 5 1 6 6 6 1 3 3 2 1 1 0 0 0 • Sample image 阈值 定义为 2 即差值<2

The Split-and-Merge Algorithm 1 1 3 1 1 1 1 4 8 1 1 9 8 1 1 8 4 1 1 2 0 0 0 1 1 1 1 6 5 5 1 6 6 6 1 3 3 2 1 1 0 0 0 • First split

The Split-and-Merge Algorithm 1 1 3 1 1 1 1 4 8 1 1 9 8 1 1 8 4 1 1 2 0 0 0 1 1 1 1 6 5 5 1 6 6 6 1 3 3 2 1 1 0 0 0 • Second split

The Split-and-Merge Algorithm 1 1 3 1 1 1 1 4 8 1 1 9 8 1 1 8 4 1 1 2 0 0 0 1 1 1 1 6 5 5 1 6 6 6 1 3 3 2 1 1 0 0 0 • Third split

The Split-and-Merge Algorithm 1 1 3 1 1 1 1 4 8 1 1 9 8 1 1 8 4 1 1 2 0 0 0 1 1 1 1 6 5 5 1 6 6 6 1 3 3 2 1 1 0 0 0 • Merge

The Split-and-Merge Algorithm 1 1 3 1 1 1 1 4 8 1 1 9 8 1 1 8 4 1 1 2 0 0 0 1 1 1 1 6 5 5 1 6 6 6 1 3 3 2 1 1 0 0 0 • Final result

Region Finding • There a variety of algorithms for region finding. • Another example is “Smooth Region Analysis” (AT&T Cambridge): • Design a filter that detects smoothness of color and texture, • apply the filter to create a smoothness image, • find closed regions of high smoothness.

Region Finding • Sample image

Region Finding • Smoothness image

Region Finding • Resulting regions shaded to their mean color

Chain Codes（Contour description) • An interesting way of describing a contour is using chain codes. • A chain code is an ordered list of local orientations (directions) of a contour. • These local directions are given through the locations of neighboring pixels, so there are only eight different possibilities. • We assign each of these directions a code: 2 1 8 3. 7 4 5 6

Chain Codes • Then we start at the first edge in the list and go clockwise around the contour. • We add the code for each edge to a list, which becomes our chain code.

Chain Codes • What happens if in our chain code for a given contour we replace every code n with (n mod 8) + 1 ? • The contour will be rotated clockwise by 45 degrees. • We can also compute the derivative of a chain code, also referred to as difference code. • Difference codes are rotation-invariant descriptions of contours. • Some features of regions, such as their corners or areas, can be directly computed from chain or difference codes.

Similarity computing • Given two strings A=a 1 a 2…an, B=b 1 b 2…bm how to measure the dissimilarity between A and B Given two strings A=a 1 a 2…an, B=b 1 b 2…bm how to change A to B character by character using three types of edit operation (1) insert a character (2) delete a character (3) replace one character with a different character such that the number of edit steps is minimum Dissimilarity = minimum number of edit steps

Distance computing for two sets

The shape through transformation approach • Shapes can be distinguished by measuring the effort needed to transform one shape into another • Similarity is measured as a transformational distance

Use of global features • All shapes are represented as binary images. A set of 22 features is used for their representation: – Area , computed as the number of pixels, set in the binary image – Circularity , expressed as the ratio between the squared perimeter and the area – Eccentricity (离心率), computed as the ratio between the smallest and the largest eigenvalues

Based on digital moments • If we consider binary images , where region R (whose shape we want to describe) is the set of points so that , the digital i, j-th moment of the R is represented by is the count of represent the area of R and

Based on digital moments • The centroid of R is expressed as :

Based on digital moments • Powerful descriptors based on digital moments are functions of moments that are invariant under scaling , translation , rotation and squeezing. • Normalized moments , are invariant under translation , scaling , stretching and squeezing transformations

Based on digital moments • Normalized moments are defined by – Considering the central i , j -th moments

Based on digital moments – Normalizing coordinates by their standard deviations and

Based on digital moments • And then normalized by the area : (4. 1)

Based on digital moments normalized central moments are obtained from central moments according to the following transformation: (4. 2) (4. 3)

Based on digital moments • A set of functions are defined from these moments. Six of them are rotation invariant , and one is both skew and rotation invariant:

Based on digital moments

Based on edge image correlation

SHAPE-BASED RETRIEVAL • Shape transformation approaches model shape similarity more closely to human perception • These methods are typically more robust in order to cope with shape distortions , support a higher precision in retrieving similar images and allow comparison with partially occluded shapes • Indexing with these approaches is impossible • 具体检索，可以根据基于chain code的相似性计 算，面积计算等。更主要的应用是对象发现。

MPEG-7 Visual Shape Descriptors • Shape Spectrum – 3 -D shape descriptor • Angular Radial Transformation(ART)— Region-based shape descriptor • Contour-based Shape Descriptor • 2 -D/3 -D Descriptor