Grab Cut Interactive Image and Stereo Segmentation Joon

Grab. Cut Interactive Image (and Stereo) Segmentation Joon Jae Lee Keimyung University

Characteristics Improved from graph cut – Use Gaussian Mixture Model (GMM) for clustering in color space – Iterative optimization n User interaction greatly reduced compared with other methods (Magic Wand, Intelligent Scissors) n

Photomontage Grab. Cut – Interactive Foreground Extraction 1

Problem Fast & Accurate ? Grab. Cut – Interactive Foreground Extraction 3

What Grab. Cut does Magic Wand (198? ) Intelligent Scissors Mortensen and Barrett (1995) Grab. Cut User Input Result Regions Boundary Regions & Boundary Grab. Cut – Interactive Foreground Extraction 4

Gaussian Mixture Model n The probability of a pixel vector is represented as:

Gaussian Mixture Model n Two sets of GMM, one for background, one for unknown region

Algorithm For each pixel assign a value α α = 0 for background and α = 1 foreground n Use graph cut to minimize a Gibbs energy: n α: background / foreground label θ: GMM parameters k: GMM labeling

Iterative minimization Initialize background pixel to α = 0 and unknown region (draw box) to α = 1. n Initialize two sets of GMMs. 1. Assign GMM labels to each pixel. (Which set of GMM is determined by α) 2. Graph cut minimize (α optimized, GMM label changed to corresponding set of GMM) 3. Update GMM parameters n

Graph Cuts - Boykov and Jolly (2001) Image Foreground (source) Min Cut Background (sink) Cut: separating source and sink; Energy: collection of edges Min Cut: Global minimal enegry in polynomial time Grab. Cut – Interactive Foreground Extraction 7

Iterated Graph Cuts Gu ar co ante nv ed er ge to 1 Result 2 3 4 Energy after each Iteration Grab. Cut – Interactive Foreground Extraction 9

Colour Model R Iterated graph cut Foregroun d& Backgroun d Backgroun G d R Foreground Backgroun d G Gaussian Mixture Model (typically 5 -8 components) Grab. Cut – Interactive Foreground Extraction 10

Coherence Model An object is a coherent set of pixels: 25 Error (%) over training set: How do we choose ? 25

Parameter Learning Problems A Gaussian MRF is not a realistic texture model Gaussian? Real Image synthetic GMRF Grab. Cut – Interactive Foreground Extraction Gaussian! 13

Moderately simple examples … Grab. Cut completes automatically Grab. Cut – Interactive Foreground Extraction 14

Difficult Examples Camouflage & Low Contrast Fine structure No telepathy Initial Rectangle Initial Result Grab. Cut – Interactive Foreground Extraction 15

Evaluation – Labelled Database Available online: http: //research. microsoft. com/vision/cambridge/segmentation/ Grab. Cut – Interactive Foreground Extraction 16

Comparison Boykov and Jolly (2001) Grab. Cut User Input Result Error Rate: 1. 87% 1. 81% 1. 32% 1. 25% 0. 72% Error Rate: 0. 72% Grab. Cut – Interactive Foreground Extraction 17

Comparison Input Image Ground Truth Trimap Boykov and Jolly Error rate - modestly increase Error Rate: 1. 36% Bimap Grab. Cut Error Rate: 2. 13% User Interactions - considerable reduced Grab. Cut – Interactive Foreground Extraction 18

Results Parameter Learning Grab. Cut – Interactive Foreground Extraction 19

Comparison Magic Wand (198? ) Intelligent Lazy. Snappin Graph Scissors g Cuts Mortensen and Li et al. Boykov Barrett (1995) and (2004) Grab. Cut – Interactive Foreground Extraction Grab. Cut Rother et al. (2004) 20