Multiresolution stereo image matching using complex wavelets Julian

Multiresolution stereo image matching using complex wavelets Julian Magarey Anthony Dick CRC for Sensor Signal and Information Processing Dept of Computer Science University of Adelaide

Stereo Vision Problem

A Stereo Pair AIM: To recover 3 D shape from stereo pair

Stereo Matching b Find a point in each image which represents the same point in the scene • corresponding points disparity

Feature Based Matching b Detect and match distinctive features b Problems • • • featureless areas occluded features same feature may appear different

Multiresolution Matching b Match points at several levels of detail Right Image MATCH FINE COARSE Left Image

Wavelet Transform Original Image Resolution i, j Level 1 Res i/2, j/2 Level 2 Res i/4, j/4

Multiresolution Matching b Now have multiresolution representation b Level m similarity distance measure: where x is a pixel in the level m representation of the left image x’ is a pixel in the level m representation of the right image

Similarity distance surface b Can extrapolate similarity surface about x’ where the surface minimum, the location of the surface minimum, a 2 x 2 curvature matrix, are derived from

Stereo Matching Algorithm b Now have basic matching algorithm • perform wavelet transform on images • minimise SD(x, x’) for all x at top level • use as starting point for finer level matching b What if top level match is wrong? b How do we interpolate matches to finer level?

Coping with Mismatches b Find a field of disparity vectors minimises which where is a directed measure of the difference between {u} and the unsmoothed disparity field is a measure of the uniformity of {u} is a scalar controlling their relative influence

Regularisation Features b Based on Anandan [IJCV, 1989] b Use curvature matrix κ to smooth more in directions of less certainty Smooth less in this direction Similarity surface contours Smooth more in this direction

Coarse-to-fine interpolation b Robust disparity interpolation A C B a b c d D = level m pixel = level m+1 pixel D(a) = choice of {D(A), D(B), D(C), D(D)} which minimises similarity distance

Results b Calibrated camera setup b Projective reconstruction b Form textured VRML surface 3 D Surface C 1 Left Camera C 2 Right Camera

Results

Results

Conclusion b Future • • • work lighting geometric constraint incorporation colour images camera self-calibration more than two images b Already, results are promising!