act Windows for Visual Correspondence via Minimum Ratio

  • Slides: 20
Download presentation
act Windows for Visual Correspondence via Minimum Ratio Cycle Algorithm Olga Veksler NEC Labs

act Windows for Visual Correspondence via Minimum Ratio Cycle Algorithm Olga Veksler NEC Labs America

Global Approach • Look at the whole image • Solve one large problem •

Global Approach • Look at the whole image • Solve one large problem • Slower, more accurate Local Approach • Look at one image patch at at time • Solve many small problems independently • Fast, sufficient for some problems

Local Approach • Sufficient for some problems • Central problem: window shape selection •

Local Approach • Sufficient for some problems • Central problem: window shape selection • Efficiently solved using Minimum Ratio Cycle algorithm for graphs

Visual Correspondence left image (x 1, y) disparity = x 1 -x 2 right

Visual Correspondence left image (x 1, y) disparity = x 1 -x 2 right image (x 2, y) stereo vertical motion first image (x 1, y 1) second image (x 2, y 2) motion horizontal motion

Local Approach [Levine’ 73] left image right image 3 p 2 1 = i

Local Approach [Levine’ 73] left image right image 3 p 2 1 = i which gives best 2 Common C = + 2 + 2

need different window shapes Fixed Window Shape Problems left image fixed small window true

need different window shapes Fixed Window Shape Problems left image fixed small window true disparities fixed large window

Variable Window: Previous Work • Two inefficient methods proposed previously 1. Local greedy search

Variable Window: Previous Work • Two inefficient methods proposed previously 1. Local greedy search [Levine CGIP’ 73, Kanade’PAMI 94] 2. Direct search [Intille ECCV 94, Geiger IJCV 95] ……. • Need efficient optimization algorithm over sizes and shapes

Minimum Ratio Cycle image pixels • G(V, E) and w(e), t(e): E R •

Minimum Ratio Cycle image pixels • G(V, E) and w(e), t(e): E R • Find cycle C which minimizes: W = t

From Area to Cycle 1 1 1 blue edge 1 red edge + 5

From Area to Cycle 1 1 1 blue edge 1 red edge + 5 sum up terms inside using weights of edges -2

Window Cost OK for any graphs 2 C(W) = +…+ size of W 2

Window Cost OK for any graphs 2 C(W) = +…+ size of W 2 = not OK for any graph positive negative

Compact Windows we construct graph s. t. p only clockwise cycles simple graph cycles

Compact Windows we construct graph s. t. p only clockwise cycles simple graph cycles C one-to-one correspondence compact windows W

Cycle which is not Simple cycle C in this case: m (C ) =

Cycle which is not Simple cycle C in this case: m (C ) = m (C )

Solving MRC find smallest s. t. åw(e) åt (e ) åw(e) - l åt

Solving MRC find smallest s. t. åw(e) åt (e ) åw(e) - l åt (e ) £ for some cycle 0 • search for smallest s. t. there is negative cycle on graph with edge weights: w (e ) - l t (e ) • negative cycle detection takes time due to the special structure of our graphs

examples of compact windows (small perimeter ) area • there are compact windows, if

examples of compact windows (small perimeter ) area • there are compact windows, if the largest allowed window is n by n • Contains all possible rectangles but much more general than just rectangles Find optimal window in in theory, linear ( ) in practice Search over in time • •

Sample Compact Windows

Sample Compact Windows

Speedup for pixel p, the algorithm extends windows over pixels which are likely to

Speedup for pixel p, the algorithm extends windows over pixels which are likely to have the same disparity as p use the optimal window computed for p to approximate for pixels inside that window

Comparison to Fixed Window true disparities fixed small window: 33% errors Compact windows: 16%

Comparison to Fixed Window true disparities fixed small window: 33% errors Compact windows: 16% errors fixed large window: 30% errors

motion

motion

Results all global Algorithm Tsukuba Venus Sawtooth Map Layered 1. 58 1. 52 0.

Results all global Algorithm Tsukuba Venus Sawtooth Map Layered 1. 58 1. 52 0. 34 0. 37 Graph cuts 1. 94 1. 79 1. 30 0. 31 Belief prop 1. 15 1. 00 0. 98 0. 84 GC+occl. 1. 27 2. 79 0. 36 1. 79 Graph cuts 1. 86 1. 69 0. 42 2. 39 Multiw. Cut 8. 08 0. 53 0. 61 0. 26 Comp. win. 3. 36 1. 67 1. 61 0. 33 13 other algorithms, local and global Running time: 8 to 22 seconds

Future Work • Generalize the window class • Generalize objective function – mean? –

Future Work • Generalize the window class • Generalize objective function – mean? – variance?