Segmentation Through Optimization Pyry Matikainen He who fights

Segmentation Through Optimization Pyry Matikainen

“He who fights with monsters should look to it that he himself does not become a monster. ” -Friedrich Nietzsche, Beyond Good and Evil

Formulate Problem ly s e tiv sion c a i ro dec t Re ify st ju Force problem into favorite algorithm “Refine” Publish Gradient ascent via parameter tweaking

What is wrong with this? Formulate Problem ely ns v i t o ac cisi o tr de Re ify st ju Force problem into favorite algorithm “Refine” Publish • Difficult to use • Difficult to extend • Difficult to study Gradient ascent via parameter tweaking

Z. Tu and S. C. Zhu (2002) to the rescue! and also Ren and Malik (2003)…

Z. Tu and S. C. Zhu. Image Segmentation by Data-Driven Markov Chain Monte Carlo. PAMI, vol. 24, no. 5, pp. 657 -673, May, 2002: The DDMCMC paradigm combines and generalizes these [all other] segmentation methods in a principled way.

Segmenter Evaluator Optimizer

Everything is search.

Evaluator Optimizer

“What is a good segment? ” Ren and Malik (2003)

How do we model a segment? Texture Contours Raw pixel values

x 2 G(x) h(f(x)) G(b(x) - x)

(gaussian) (histogram) (gabor) (Bezier)

Number of regions Region perimeter length (smoothness) Notably absent: the data Region area Region appearance model complexity

Superpixels (normalized cuts) Oriented energy Brightness Texture (textons)

* Classifier G(W|I)

Evaluator Optimizer

MCMC is a technique for sampling from distributions.

Number of regions Region? ?

Ren and Malik Merge Split Boundary competition The ‘data driven’ part revealed! Model adaptation Switching image models

Data driven = do some clustering to make the MCMC faster.

Evaluator Optimizer

Tu & Zhu

Ren & Malik

Tu & Zhu New paradigm? Combines and generalizes other techniques? Principled? Good results? Ren & Malik 1/2 0 0 1 1/3

Evaluator Optimizer Evaluator

(gaussian) (mixture of gaussians) (3 x Bezier spline)

(g 1) (gaussian) (g 2) (histogram) (g 3) (gabor filter) (g 4) (Bezier spline)

Number of regions Region appearance model parameters Region appearance model Pixels in region

MCMC

Xiaofeng Ren and Jitendra Malik. Learning a Classification Model for Segmentation. ICCV 2003.

Boundary between i and j

Classification certainty Tu and Zhu 2002 Sampling P(W|I) Generative models Pixels Ren and Malik 2003 Maximizing G(W|I) Discriminative models Superpixels