Markov Random Fields with Efficient Approximations Yuri Boykov
Markov Random Fields with Efficient Approximations Yuri Boykov, Olga Veksler, Ramin Zabih Computer Science Department CORNELL UNIVERSITY 1
Introduction MAP-MRF approach (Maximum Aposteriori Probability estimation of MRF) • Bayesian framework suitable for problems in Computer Vision (Geman and Geman, 1984) • Problem: High computational cost. Standard methods (simulated annealing) are very slow. 2
Outline of the talk n Models where MAP-MRF estimation is equivalent to min-cut problem on a graph • generalized Potts model • linear clique potential model n Efficient methods for solving the corresponding graph problems n Experimental results • stereo, image restoration 3
MRF framework in the context of stereo • image pixels (vertices) • neighborhood relationships (n-links ) - disparity at pixel p - configuration MRF defining property: Hammersley-Clifford Theorem: 4
MAP estimation of MRF configuration Observed data Bayes rule Likelihood function (sensor noise) Prior (MRF model) 5
Energy minimization Find that minimizes the Posterior Energy Function : Data term (sensor noise) Smoothness term (MRF prior) 6
Generalized Potts model Clique potential Penalty for discontinuity at (p, q) Energy function 7
Static clues - selecting Stereo Image: White Rectangle in front of the black background Disparity configurations minimizing energy E( f ): 8
Minimization of E(f) via graph cuts Terminals (possible disparity labels) Cost of n-link Cost of t-link p-vertices (pixels) 9
Multiway cut vertices edges V = pixels + terminals E = n-links + t-links Graph G = <V, E> Remove a subset of edges C Graph G(C) = <V, E-C > • C is a multiway cut if terminals are separated in G(C) • A multiway cut C yields some disparity configuration 10
Main Result (generalized Potts model) n Under some technical conditions on the multiway min-cut C on G gives___ that minimizes E( f ) - the posterior energy function for the generalized Potts model. • Multiway cut Problem: find minimum cost multiway cut C graph G 11
Solving multiway cut problem n Case of two terminals: • max-flow algorithm (Ford, Fulkerson 1964) • polinomial time (almost linear in practice). n NP-complete if the number of labels >2 • (Dahlhaus et al. , 1992) n Efficient approximation algorithms that are optimal within a factor of 2 12
Our algorithm Initialize at arbitrary multiway cut C 1. Choose a pair of terminals 2. Consider connected pixels 13
Our algorithm Initialize at arbitrary multiway cut C 1. Choose a pair of terminals 2. Consider connected pixels 3. Reallocate pixels between two terminals by running max-flow algorithm 14
Our algorithm Initialize at arbitrary multiway cut C 1. Choose a pair of terminals 2. Consider connected pixels 3. Reallocate pixels between two terminals by running max-flow algorithm 4. New multiway cut C’ is obtained Iterate until no pair of terminals improves the cost of the cut 15
Experimental results (generalized Potts model) n Extensive benchmarking on synthetic images and on real imagery with dense ground truth • From University of Tsukuba • Comparisons with other algorithms 16
Synthetic example Image Correlation Multiway cut 17
Real imagery with ground truth Ground truth Our results 18
Comparison with ground truth 19
Gross errors (> 1 disparity) 20
Comparative results: normalized correlation Gross errors Data 21
Statistics 22
Related work (generalized Potts model) Greig et al. , 1986 is a special case of our method (two labels) n Two solutions with sensor noise (function g) highly restricted n • Ferrari et al. , 1995, 1997 23
Linear clique potential model Clique potential Penalty for discontinuity at (p, q) Energy function 24
Minimization of via graph cuts Cost of t-link cut C Cost of n-link {p, q} part of graph a cut C yields some configuration 25
Main Result (linear clique potential model) n Under some technical conditions on the min-cut C on gives that minimizes - the posterior energy function for the linear clique potential model. 26
Related work (linear clique potential model) n Ishikawa and Geiger, 1998 • earlier independently obtained a very similar result on a directed graph n Roy and Cox, 1998 • undirected graph with the same structure • no optimality properties since edge weights are not theoretically justified 27
Experimental results n (linear clique potential model) Benchmarking on real imagery with dense ground truth • From University of Tsukuba n Image restoration of synthetic data 28
Ground truth stereo image ground truth Generalized Potts model Linear clique potential model 29
Image restoration Noisy diamond image Generalized Potts model Linear clique potential model 30
- Slides: 30