Multiscale Combinatorial Grouping for Image Segmentation and Object

Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation Student: YEH, HAO-WEI Advisor: Sheng-Jyh, Wang

References • P. Arbelaez, M. Maire, C. Fowlkes, J. Malik. “Contour Detection and Hierarchical image Segmentation, ” IEEE Trans. on PAMI , 2011. • Also slides from Hsin-Min, Cheng • X. Ren and L. Bo, “Discriminatively trained sparse code gradients for contour detection, ” in NIPS, 2012. • P. Doll´ar and C. Zitnick, “Structured forests for fast edge detection, ” ICCV, 2013. • Also Slides by Piotr Dollár • P. Arbelaez, J. Pont-Tuset, Jonathan T. Barron, F. Marques, and J. Malik. “Multiscale combinatorial grouping, ” In CVPR, 2014. • Jordi Pont-Tuset, Pablo Arbelaez, Jonathan T. Barron, Ferran Marques, Jitendra Malik, “Multiscale Combinatorial Grouping for Image Segmentation and Object Proposal Generation, ” ar. Xiv 2015

Outline • Introduction to Image Segmentation & Object Generation • Workflow of the paper • Single-scale Segmentation • UCM in PAMI 2010 • Modifications in CVPR 2014, ar. Xiv 2015 • Rescaling & Alignment • Combinatorial grouping • Experimental Results

Introduction

Image Segmentation vs. Contour Detection Contour Original Image Segmentation

Hierarchical Image Segmentation

Object Generation

Workflow

Workflow (CVPR 2014, ar. Xiv 2015)

g. Pb-owt-ucm (PAMI 2011) Original Image Local cues Global cues Oriented Gradient of histograms Contour (g. Pb) • Oriented Watershed Transform (OWT) • Iteratively Merging Hierarchical Image Segmentation (UCM)

Local Pb (m. Pb) Example • Local Cues • Brightness, Color • L*a*b* color space • Texture • Textons with 17 filters Filters for creating textons Gradient magnitude G at location(x, y) • Method • Oriented gradient of histograms • Three scales of r

ure

Spectral Pb (s. Pb) • Global Cues • Weighted graph G=(V, E) from image • V: image pixels • E: connections between pairs of nearby pixels => • Method • Normalized Cuts (Spectral Clustering) U. v. Luxburg, “A tutorial on spectral clustering, ” Statistics and Computing, vol. 17, no. 4, pp. 395– 416, 2007. http: //www. informatik. uni-hamburg. de/ML/contents/people/luxburg/publications/Luxburg 07_tutorial. pdf

Spectral Pb (s. Pb)

Globalized Pb (g. Pb) Local Cues Global cues

g. Pb-owt-ucm (PAMI 2011) Original Image Local cues Global cues Oriented Gradient of histograms Contour (g. Pb) • Oriented Watershed Transform (OWT) • Iteratively Merging Hierarchical Image Segmentation (UCM)

Hierarchical Image Segmentation

Hierarchical Segmentation • From contour to segments (OWT) • Hierarchical Segmentation by iteratively merging the regions (UCM)

Hierarchical Segmentation • Oriented Watershed transform • Approximate the watershed arcs with line segments • Use the orientation of the arcs o(x, y) OWT WT

Hierarchical Segmentation • Hierarchical Segmentation by iteratively merging the regions (UCM)

g. Pb-owt-ucm (PAMI 2011) Original Image Local cues Global cues Oriented Gradient of histograms Contour (g. Pb) • Oriented Watershed Transform (OWT) • Iteratively Merging Hierarchical Image Segmentation (UCM)

UCM vs. Region-Tree

Experimental Results(PAMI 2011)

Single-scale Segmentation • Modifications • Different local cues • Brightness, color, texture differences as in PAMI 2010 • Sparse coding on patches • Structured forest contour • Faster eigenvector computation • Similarity of the affinity matrix in different scales

Sparse Coding Patches X. Ren and L. Bo, “Discriminatively trained sparse code gradients for contour detection, ” in NIPS, 2012.

Structured Forest Detector P. Doll´ar and C. Zitnick, “Structured forests for fast edge detection, ” ICCV, 2013.

Workflow (CVPR 2014, ar. Xiv 2015)

Rescaling & Alignment • Align every UCM to the original scale • Problem : How to upsample/downsample the contour? • Solution : Projection of the corresponding segmentations

Workflow (CVPR 2014, ar. Xiv 2015)

Object Generation

Combinatorial Grouping • • Choose object proposals(combination of regions) from UCM Method: Supervised Learning Problem: Exhaustive search is intractable!! Solution: • Pareto Optimization • Regressed Ranking

Experimental Results(ar. Xiv 2015)

Workflow (CVPR 2014, ar. Xiv 2015)

Thank you for your attentions!!