Interactive Image Segmentation using Graph Cuts PRASA 2009

Outline • • • Image Segmentation Problem Our Approach Graph cuts and Gaussian Mixture

Our Approach Image properties eg. colour, texture Difference between adjacent pixels Region information Boundary

Graph Cuts Source (foreground) Pixel connectivity (boundaries) Inter-pixel weights (boundaries) Source and Sink weights

Gaussian Mixture Models Background GMM Foreground GMM

Gaussian Mixture Models Foreground GMM pf pb Background GMM Log Likelihood Ratio = log(K

GMM components • Greyscale images – Intensity values and MR 8 filters • Colour

Boundary information • Inter-pixel weights – Edge detection – Difference between adjacent pixels –

Results Original Image RGB, Luv and MR 8 (Fscore = 0. 916) Luv and

Results Original Image Luv (Fscore = 0. 945) RGB, Luv and MR 8 (Fscore

Analysis of Results • Accurate segmentation achieved • Components in the GMM depend on

Future Research • • Different grid (non-pixel grid) Ratio cuts Exploring other statistical models

References • • • Y. Boykov and M. P. Jolly. Interactive graph cuts for

Slides: 16

Download presentation

Interactive Image Segmentation using Graph Cuts PRASA 2009 Mayuresh Kulkarni and Fred Nicolls Digital Image Processing Group University of Cape Town

Outline • • • Image Segmentation Problem Our Approach Graph cuts and Gaussian Mixture Models Results and Discussion Future Research

What is foreground?

Image Segmentation

Our Approach Image properties eg. colour, texture Difference between adjacent pixels Region information Boundary information Graph Cuts Segmentation Cost Function : E(A) = λ R(A) + B(A) 8 – pixel neighbourhood Pixel connectivity

Graph Cuts Source (foreground) Pixel connectivity (boundaries) Inter-pixel weights (boundaries) Source and Sink weights (regions) Cost Function : E(A) = λ R(A) + B(A) Sink (background)

Gaussian Mixture Models Background GMM Foreground GMM

Gaussian Mixture Models Foreground GMM pf pb Background GMM Log Likelihood Ratio = log(K *pf/pb)

GMM components • Greyscale images – Intensity values and MR 8 filters • Colour images – – RGB values G, (G-R), (G-B) values Luv values MR 8 filters

Boundary information • Inter-pixel weights – Edge detection – Difference between adjacent pixels – Gradient • Pixel connectivity

Results Κ = 0. 01 Κ = 0. 1 Κ=1

Results Original Image RGB, Luv and MR 8 (Fscore = 0. 916) Luv and MR 8 (Fscore = 0. 921) Luv (Fscore = 0. 934)

Results Original Image Luv (Fscore = 0. 945) RGB, Luv and MR 8 (Fscore = 0. 906) RGB (Fscore = 0. 951)

Analysis of Results • Accurate segmentation achieved • Components in the GMM depend on image • Segmentation can be controlled using K and λ

Future Research • • Different grid (non-pixel grid) Ratio cuts Exploring other statistical models Obj. Cut – segmenting particular objects

References • • • Y. Boykov and M. P. Jolly. Interactive graph cuts for optimal boundary and region segmentation of objects in N-D images. In ICCV, volume 1, pages 105– 112, July 2001. Yuri Boykov and Vladimir Kolmogorov. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. , 26(9): 1124– 1137, 2004. Pushmeet Kohli, Jonathan Rihan, Matthieu Bray, and Philip H. S. Torr. Simultaneous segmentation and pose estimation of humans using dynamic graph cuts. International Journal of Computer Vision, 79(3): 285– 298, 2008. H. Permuter, J. Francos, and I. Jermyn. Gaussian mixture models of texture and colour for image database. In ICASSP, pages 25– 88, 2003. D. Martin, C. Fowlkes, D. Tal, and J. Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proc. 8 th Int’l Conf. Computer Vision, volume 2, pages 416– 423, July 2001. Carsten Rother, Vladimir Kolmogorov, and Andrew Blake. “Grab. Cut”: interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. , 23(3): 309– 314, August 2004.