Segmentation and Boundary Detection Using Multiscale Intensity Measurements
Segmentation and Boundary Detection Using Multiscale Intensity Measurements Eitan Sharon, Meirav Galun, Ronen Basri, Achi Brandt Dept. of Computer Science and Applied Mathematics The Weizmann Institute of Science Eitan Sharon - Weizmann Institute
Image Segmentation Eitan Sharon - Weizmann Institute
Local Uncertainty Eitan Sharon - Weizmann Institute
Global Certainty Eitan Sharon - Weizmann Institute
Local Uncertainty Eitan Sharon - Weizmann Institute
Global Certainty Eitan Sharon - Weizmann Institute
Coarse Measurements for Texture Eitan Sharon - Weizmann Institute
A Chicken and Egg Problem: Coarse measurements mix neighboring statistics Solution: support of measurements is determined as the segmentation process proceeds Eitan Sharon - Weizmann Institute
Segmentation by Weighted Aggregation q Normalized-cuts measure in graphs q Complete hierarchy in linear time q Use multiscale measures of intensity, texture, shape, and boundary integrity Eitan Sharon - Weizmann Institute
Segmentation by Weighted Aggregation q Normalized-cuts measure in graphs q Complete hierarchy in linear time q Use multiscale measures of intensity, texture, shape, and boundary integrity Eitan Sharon - Weizmann Institute
Segmentation by Weighted Aggregation q Normalized-cuts measure in graphs q Complete hierarchy in linear time q Use multiscale measures of intensity, texture, shape and boundary integrity Eitan Sharon - Weizmann Institute
The Pixel Graph Couplings Reflect intensity similarity Low contrast – strong coupling High contrast – weak coupling Eitan Sharon - Weizmann Institute
Hierarchical Graph Eitan Sharon - Weizmann Institute
Hierarchy in SWA Eitan Sharon - Weizmann Institute
Normalized-Cut Measure Eitan Sharon - Weizmann Institute
Normalized-Cut Measure Eitan Sharon - Weizmann Institute
Normalized-Cut Measure Eitan Sharon - Weizmann Institute
Normalized-Cut Measure Minimize: Eitan Sharon - Weizmann Institute
Normalized-Cut Measure High-energy cut Minimize: Eitan Sharon - Weizmann Institute
Normalized-Cut Measure Low-energy cut Minimize: Eitan Sharon - Weizmann Institute
Recursive Coarsening Eitan Sharon - Weizmann Institute
Recursive Coarsening Representative subset Eitan Sharon - Weizmann Institute
Recursive Coarsening For a salient segment : , sparse interpolation matrix Eitan Sharon - Weizmann Institute
Weighted Aggregation aggregate Eitan Sharon - Weizmann Institute
Segment Detection Eitan Sharon - Weizmann Institute
SWA Detects the salient segments Hierarchical structure Linear in # of points (a few dozen operations per point) Eitan Sharon - Weizmann Institute
Coarse-Scale Measurements • Average intensities of aggregates • Multiscale intensity-variances of aggregates • Multiscale shape-moments of aggregates • Boundary alignment between aggregates Eitan Sharon - Weizmann Institute
Adaptive vs. Rigid Measurements Original Our algorithm - SWA Averaging Geometric Eitan Sharon - Weizmann Institute
Adaptive vs. Rigid Measurements Original Our algorithm - SWA Interpolation Geometric Eitan Sharon - Weizmann Institute
Recursive Measurements: Intensity aggregate intensity of pixel i average intensity of aggregate Eitan Sharon - Weizmann Institute
Use Averages to Modify the Graph Eitan Sharon - Weizmann Institute
Use Averages to Modify the Graph Eitan Sharon - Weizmann Institute
Texture Examples Eitan Sharon - Weizmann Institute
Isotropic and Oriented Filters A brief tutorial Textons by K-Means Malik et al IJCV 2001 Eitan Sharon - Weizmann Institute
Isotropic Texture in SWA Intensity Variance Isotropic Texture of aggregate – average of variances in all scales Eitan Sharon - Weizmann Institute
Isotropic Texture in SWA Intensity Variance Isotropic Texture of aggregate – average of variances in all scales Eitan Sharon - Weizmann Institute
Isotropic Texture in SWA Intensity Variance Isotropic Texture of aggregate – average of variances in all scales Eitan Sharon - Weizmann Institute
Oriented Texture in SWA with Meirav Galun Shape Moments • center of mass • width • length • orientation Oriented Texture of aggregate – orientation, width and length in all scales Eitan Sharon - Weizmann Institute
Gestalt – Perceptual Grouping Shashua and Ullman ICCV 1988: A brief Tutorial Group curves by: • Proximity • Co-linearity Sharon, Brandt, Basri PAMI 2000: Eitan Sharon - Weizmann Institute
Boundary Integrity in SWA Eitan Sharon - Weizmann Institute
Sharpen the Aggregates Top-down Sharpening: • Expand core • Sharpen boundaries Eitan Sharon - Weizmann Institute
Hierarchy in SWA Eitan Sharon - Weizmann Institute
Experiments images on a pentium III 1000 MHz PC: • Our SWA algorithm (CVPR’ 00 + CVPR’ 01) run-time: 5 -10 seconds. • Normalized cuts (Shi and Malik, PAMI’ 00; Malik et al. , IJCV’ 01) run-time: about 10 -15 minutes. Software courtesy of Doron Tal, UC Berkeley. Eitan Sharon - Weizmann Institute
Isotropic Texture - Horse I Our Algorithm (SWA) Normalized Cuts Eitan Sharon - Weizmann Institute
Isotropic Texture - Horse II Our Algorithm (SWA) Normalized Cuts Eitan Sharon - Weizmann Institute
Isotropic Texture - Tiger Our Algorithm (SWA) Normalized Cuts Eitan Sharon - Weizmann Institute
Isotropic Texture - Butterfly Our Algorithm (SWA) Normalized Cuts Eitan Sharon - Weizmann Institute
Isotropic Texture - Leopard Our Algorithm (SWA) Eitan Sharon - Weizmann Institute
Isotropic Texture - Dalmatian Dog Our Algorithm (SWA) Eitan Sharon - Weizmann Institute
Isotropic Texture - Squirrel Our Algorithm (SWA) Normalized Cuts Eitan Sharon - Weizmann Institute
Full Texture - Squirrel Our Algorithm (SWA) Normalized Cuts with Meirav Galun Eitan Sharon - Weizmann Institute
Full Texture - Composition Our Algorithm (SWA) with Meirav Galun Eitan Sharon - Weizmann Institute
Full Texture – Lion Cub Our Algorithm (SWA) with Meirav Galun Eitan Sharon - Weizmann Institute
Full Texture – Polar Bear Our Algorithm (SWA) with Meirav Galun Eitan Sharon - Weizmann Institute
Full Texture – Penguin Our Algorithm (SWA) with Meirav Galun Eitan Sharon - Weizmann Institute
Full Texture –Leopard Our Algorithm (SWA) with Meirav Galun Eitan Sharon - Weizmann Institute
Full Texture - Zebra Our Algorithm (SWA) with Meirav Galun Eitan Sharon - Weizmann Institute
Segmentation by Weighted Aggregation • Efficient approximation to Ncut-like measures • Recursive computation of multiscale measurements • Novel adaptive pyramid representing the image Eitan Sharon - Weizmann Institute
Matching Experiment (Chen Brestel) Eitan Sharon - Weizmann Institute
Matching Experiment (Chen Brestel) Eitan Sharon - Weizmann Institute
Matching Experiment (Chen Brestel) Eitan Sharon - Weizmann Institute
Experiments – Clustering Silhouettes Data from: Gdalyahu, Weinshall, Werman – CVPR 99 (Also: Domany, Blatt, Gdalyahu, Weinshall) Eitan Sharon - Weizmann Institute
Descending order of prominence – objects are fully categorized most prominent less prominent Eitan Sharon - Weizmann Institute
Bottom-Up and Top-Down Eitan Sharon - Weizmann Institute
- Slides: 64