STEREO MATCHING USING POPULATIONBASED MCMC Joonyoung Park 1

STEREO MATCHING USING POPULATION-BASED MCMC Joonyoung Park 1 Wonsik Kim 2 Kyoung Mu Lee 2 1 LG Electronics Inc. 2 Computer Vision Lab. School of EECS Seoul National University, Korea

Low-Level Vision as an Optimization Problem ü ü Stereo Photomontage Segmentation Denoising /Inpainting . . . General pairwise MRF model: Computer Vision Lab. SNU

Existing Methods for Optimization Problem • fast • easy to get stuck at local minimum ü Graph cuts • α–Expansion / Swap ü Message passing • Belief Propagation (BP) / Tree-Re. Weighted Message Passing (TRW) ü Sampling based method • Gibbs sampler • Swendsen-Wang Cuts • slow • global minimum Computer Vision Lab. SNU

Our Study We develop a new efficient sampling based optimization method. Population based MCMC Computer Vision Lab. SNU

What is sampling ? Computer Vision Lab. SNU

Sampling To generate samples from a target distribution Given probability distribution Obtained samples Computer Vision Lab. SNU

Optimization previous sampling based methods Two important concepts Annealing + Sample move Computer Vision Lab. SNU

Optimization Annealing previous sampling based methods target distribution: temperature: . . . Computer Vision Lab. SNU

Optimization Annealing previous sampling based methods target distribution: temperature: . . . Computer Vision Lab. SNU

Optimization Annealing previous sampling based methods target distribution: temperature: . . . Most Probable State! Computer Vision Lab. SNU

Optimization previous sampling based methods Sample move Previous methods Gibbs sampler Swendsen–Wang Cuts Computer Vision Lab. SNU

Optimization previous sampling based methods Sample move Prev. Method①: Gibbs Sampler Flip the label of single node Too slow (only local move) Computer Vision Lab. SNU

Optimization Sample move previous sampling based methods Prev. Method②: Swendsen-Wang Cuts Flip the label of a cluster Still too slow (only local move) Computer Vision Lab. SNU

Optimization previous sampling based methods Limitation of previous methods slow convergence rate à Fast cooling: easy to get stuck at local minima Computer Vision Lab. SNU

Optimization Parallel tempering Conventional annealing: . . . time Parallel tempering: time . . . time Computer Vision Lab. SNU

Optimization Parallel tempering Conventional annealing: . . . time Parallel tempering: time interaction! time . . . interaction! time Computer Vision Lab. SNU

Population based MCMC • It was originated from parallel tempering • It uses multiple chains – It allows more active interactions between chains. • We designed 3 sample moves – Mutation : The move of single chain – Exchange : – Crossover : Ways of interaction between two chains Our contribution!! Computer Vision Lab. SNU

Optimization Population based MCMC Pop. MCMC: Move① – Mutation Computer Vision Lab. SNU

Optimization Pop. MCMC: Move① – Mutation Population based MCMC time . . . time Computer Vision Lab. SNU

Optimization Pop. MCMC: Move① – Mutation Population based MCMC time . . . time Computer Vision Lab. SNU

Optimization Population based MCMC Pop. MCMC: Move② – Exchange Computer Vision Lab. SNU

Optimization Pop. MCMC: Move② – Exchange Population based MCMC time . . . time Computer Vision Lab. SNU

Optimization Pop. MCMC: Move② – Exchange Population based MCMC time . . . time Computer Vision Lab. SNU

Optimization Population based MCMC Pop. MCMC: Move③ – Crossover Computer Vision Lab. SNU

Optimization Pop. MCMC: Move③ – Crossover Population based MCMC time . . . time Computer Vision Lab. SNU

Optimization Pop. MCMC: Move③ – Crossover Population based MCMC time . . . time Computer Vision Lab. SNU

Pop. MCMC: Overall mutation mutation crossover /exchange mutation . . . mutation mutation crossover /exchange mutation Computer Vision Lab. SNU

Proposed: Population-based MCMC Advantages ü Global move Unlike Swendsen-Wang Cuts that performs only local move, our exchange and crossover moves allow global move. ü Fast Mixing Global move and parallel tempering enable fast mixing. Computer Vision Lab. SNU

Application to Stereo Problem using Segment-based Energy Model (A. Klaus et. al. , ICPR 2006) Computer Vision Lab. SNU

Application to Stereo Segment-based Energy Model = MRF DESIGN = NODES: segments NEIGHBORS: adjacent segments LABELS: planes of disparities Computer Vision Lab. SNU

Application to Stereo Test Images Tsukuba Venus Teddy Cones From Middlebury web-site Computer Vision Lab. SNU

Experimental Results Disparity maps Bad pixels(%) Tsukuba Venus Teddy Cones 1. 38 1. 21 14. 7 13. 1 Computer Vision Lab. SNU

Experimental Results Convergence Tsukuba Cones Computer Vision Lab. SNU

Conclusion – We develop a new efficient Population-based MCMC for optimization and applied it to stereo problem. – It finds lower energy state compared with other methods. Future works – Other applications / More complex energy models Computer Vision Lab. SNU

Thank You Computer Vision Lab. Seoul National University http: //cv. snu. ac. kr Computer Vision Lab. SNU