Segmentation Through Optimization Pyry Matikainen He who fights

“He who fights with monsters should look to it that he himself does not

Formulate Problem ly s e tiv sion c a i ro dec t Re

What is wrong with this? Formulate Problem ely ns v i t o ac

Z. Tu and S. C. Zhu (2002) to the rescue! and also Ren and

Z. Tu and S. C. Zhu. Image Segmentation by Data-Driven Markov Chain Monte Carlo.

“What is a good segment? ” Ren and Malik (2003)

How do we model a segment? Texture Contours Raw pixel values

Number of regions Region perimeter length (smoothness) Notably absent: the data Region area Region

Superpixels (normalized cuts) Oriented energy Brightness Texture (textons)

MCMC is a technique for sampling from distributions.

Ren and Malik Merge Split Boundary competition The ‘data driven’ part revealed! Model adaptation

Data driven = do some clustering to make the MCMC faster.

Tu & Zhu New paradigm? Combines and generalizes other techniques? Principled? Good results? Ren

(gaussian) (mixture of gaussians) (3 x Bezier spline)

(g 1) (gaussian) (g 2) (histogram) (g 3) (gabor filter) (g 4) (Bezier spline)

Number of regions Region appearance model parameters Region appearance model Pixels in region

Xiaofeng Ren and Jitendra Malik. Learning a Classification Model for Segmentation. ICCV 2003.

Classification certainty Tu and Zhu 2002 Sampling P(W|I) Generative models Pixels Ren and Malik

Slides: 49

Download presentation

Segmentation Through Optimization Pyry Matikainen

“He who fights with monsters should look to it that he himself does not become a monster. ” -Friedrich Nietzsche, Beyond Good and Evil

Formulate Problem ly s e tiv sion c a i ro dec t Re ify st ju Force problem into favorite algorithm “Refine” Publish Gradient ascent via parameter tweaking

What is wrong with this? Formulate Problem ely ns v i t o ac cisi o tr de Re ify st ju Force problem into favorite algorithm “Refine” Publish • Difficult to use • Difficult to extend • Difficult to study Gradient ascent via parameter tweaking

Z. Tu and S. C. Zhu (2002) to the rescue! and also Ren and Malik (2003)…

Z. Tu and S. C. Zhu. Image Segmentation by Data-Driven Markov Chain Monte Carlo. PAMI, vol. 24, no. 5, pp. 657 -673, May, 2002: The DDMCMC paradigm combines and generalizes these [all other] segmentation methods in a principled way.

Segmenter Evaluator Optimizer

Everything is search.

Evaluator Optimizer

“What is a good segment? ” Ren and Malik (2003)

How do we model a segment? Texture Contours Raw pixel values

x 2 G(x) h(f(x)) G(b(x) - x)

(gaussian) (histogram) (gabor) (Bezier)

Number of regions Region perimeter length (smoothness) Notably absent: the data Region area Region appearance model complexity

Superpixels (normalized cuts) Oriented energy Brightness Texture (textons)

* Classifier G(W|I)

Evaluator Optimizer

MCMC is a technique for sampling from distributions.

Number of regions Region? ?

Ren and Malik Merge Split Boundary competition The ‘data driven’ part revealed! Model adaptation Switching image models

Data driven = do some clustering to make the MCMC faster.

Evaluator Optimizer

Tu & Zhu

Ren & Malik

Tu & Zhu New paradigm? Combines and generalizes other techniques? Principled? Good results? Ren & Malik 1/2 0 0 1 1/3

Evaluator Optimizer Evaluator

(gaussian) (mixture of gaussians) (3 x Bezier spline)

(g 1) (gaussian) (g 2) (histogram) (g 3) (gabor filter) (g 4) (Bezier spline)

Number of regions Region appearance model parameters Region appearance model Pixels in region

MCMC

Xiaofeng Ren and Jitendra Malik. Learning a Classification Model for Segmentation. ICCV 2003.

Boundary between i and j

Classification certainty Tu and Zhu 2002 Sampling P(W|I) Generative models Pixels Ren and Malik 2003 Maximizing G(W|I) Discriminative models Superpixels