ContentBased Image Retrieval Feature Extraction Algorithms EE381 K14

Content-Based Image Retrieval: Feature Extraction Algorithms EE-381 K-14: Multi-Dimensional Digital Signal Processing BY: Michele Saad EMAIL: michele. saad@mail. utexas. edu PROF: Brian L. Evans 11/28/2020

Motivation • Increased use of image and video – Education – Entertainment – Commercial purpose • Need for efficient and effective browsing into image databases • Need for reduction of semantic gap between low-level features and high-level user semantics 11/28/2020

Objectives and Contributions • Objective: – Implementation and comparison of texture and color feature extraction algorithms • Contribution: – An up-to-date comparison of state-of-the-art texture and color feature extraction methods 11/28/2020

1 11/28/2020

Color Features Color Feature Pros Conventional • Fast computation Color histogram • Simple Fuzzy Color Histogram • Fast Computation • Color similarity • Robust to quantization noise • Robust to contrast Cons Color Space • High dimensionality • No color similarity • No spatial info HSV • High dimensionality • More computation • Appropriate choice of membership weights needed HSV 11/28/2020 J. Huang, S. R. Kumar, M. Mitra, W. J. Zhu and R. Zabih, “Time Indexing Using Color Correlograms”, Proc. IEEE Conf. on Computer Vision and Pattern Recognition, pp. 762 – 768, June 1997

Color Features Cont’d Color Feature Correlogram Color/Shape Method Pros • Spatial Info • Spatial info • Area • Shape Cons • Very slow • High dimensionality • No color similarity • More computation • Sensitive to clutter • Choice of appropriate color quantization thresholds needed Color Space HSV 11/28/2020 N. R. Howe, D. P. Huttenlocher, “Integrating Color, Texture and Geometry for Image Retrieval”, Proc. IEEE Conf. on Computer Vision and Pattern Recognition, vol. II, pp. 239 -246, June 2000.

Color Image Database: The Corel Database • 10 classes of 100 images each 11/28/2020 http: //wang. ist. psu. edu/IMAGE

Color Feature Extraction: Retrieval Results CCH Average Retrieval Score 80. 12 % FCH 82. 05 % NB: Euclidean distance measure used 11/28/2020 Correlogra Color/Shap m e 69. 48% 70. 03%

2 11/28/2020

Texture Features Texture Feature Steerable Pyramid Pros Cons Frequency Domain Partition • Supports any • Sub-bands number of undecimated orientation • Lower sub. Contourlet bands Transform decimated • Number of orientations is a power of 2 S. Oraintara, T. T. Nguyen, “Using Phase and Magnitude Information of the Complex directional Filter Bank 11/28/2020 for Texture Image Retrieval”, Proc. IEEE Int. Conf. on Image Processing, vol. 4, pp. 61 -64, Oct. 2007

Texture Features Cont’d Texture Feature Gabor Wavelet Pros • Highest retrieval results Cons Frequency Domain Partition • Over-complete representation • Computationally intensive Complex • Competitive Directional retrieval • More computation Filter Bank results 11/28/2020 S. Oraintara, T. T. Nguyen, “Using Phase and Magnitude Information of the Complex directional Filter Bank for Texture Image Retrieval”, Proc. IEEE Int. Conf. on Image Processing, vol. 4, pp. 61 -64, Oct. 2007

Texture Database: The Brodatz Database • 13 different textures: – Bark, brick, bubbles, grass, leather, pigskin, raffia, sand, straw, water, weave, wood and wool – Rotated at different angles • Examples: 11/28/2020 http: //www. ux. uis. no/~tranden/brodatz. html

Texture Feature Extraction: Retrieval Results Average Retrieval Score Steerab le Pyrami d Contourl Complex et Direction Gabor Transfor al Filter m Bank 63. 02% 81. 48 % NB: L 1 Norm used in the distance measure 11/28/2020 63. 67% 76%

Conclusion and Future Work • Highest retrieval results obtained by: – Fuzzy color histogram – Gabor wavelet transform • Keeping in mind some trade offs • Appropriate distance measures need to be considered further – May improve results further 11/28/2020