Discontinuity Preserving Stereo with Small Baseline MultiFlash Illumination
Discontinuity Preserving Stereo with Small Baseline Multi-Flash Illumination Rogerio Feris 1, Ramesh Raskar 2, Longbin Chen 1, Karhan Tan 3 and Matthew Turk 1 1 University of California, Santa Barbara 2 Mitsubishi Electric Research Labs 3 Epson Palo Alto Lab
Introduction § Correspondence Problem § Stereo Near Depth Discontinuities: - Occlusion Problem - Perspective Distortions - Violation of Smoothness Constraints § Passive Versus Active Methods
Introduction § Our Approach: Small Baseline Multi-Flash Illumination - Simple, Inexpensive - Compact, Self-Contained - Discontinuity Preserving
Depth Edges with Multi-Flash Raskar, Tan, Feris, Yu, Turk – ACM SIGGRAPH 2004
Bottom Flash Top Flash Left Flash Right Flash Ratio images and directions of epipolar traversal Shadow-Free Depth Edges
Qualitative Depth Map
Qualitative Depth Sign of Depth Edge - Indicates which side is the foreground and which side is the background Shadow Width - Encodes object relative distances
Sign of Depth Edge Original Ratio Left Ratio Right Signed Edges -++- (+) Foreground (-) Background
Shadow Width Bottom Flash Image Ratio Image Plot Along Scanline
Shadow Width Bottom Flash Image Ratio Image Shadow Width Estimation: Meanshift Segmentation algorithm applied on the ratio image
Imaging Geometry Flash B Camera f d z 1 z 2 Object Shadow Width
Qualitative Depth Working on this Equation … Gradient-Domain Problem! Shadow Width Log Depth Difference
Qualitative Depth 1) Compute Sharp Depth Gradient G = (Gh, Gv) Log Depth Difference Sign of depth edge 2) Compute Q’ by integrating G (Poisson Equation) 3) Qualitative depth map Q = exp(Q’)
Qualitative Depth Useful Prior Information for Stereo !
Occlusion Map
Partial Occlusion Problem Occlusion (Seen by A but not by B) Object A B Camera
Occlusion Bounded by Shadows Occlusion (Seen by A but not by B) Object Flash A B Camera
Occlusion Bounded by Shadows Lower Bound Shadow Object Flash A B Camera
Occlusion Bounded by Shadows Upper Bound Shadow Object Flash A B Camera
Occlusion Bounded by Shadows Occlusion Average of Upper/Lower Shadow widths Object Flash A B Camera
Occlusion Bounded by Shadows Left View Right View Occlusion Map
Discontinuity Preserving Stereo Matching
Local Stereo Problem: Shape and size of correlation window - Small Window Ambiguities / Noise - Large Window Problems at Depth Discontinuities Depth Edge Preserving Local Stereo Correlation Window Object Boundary (Depth Edge)
Local Stereo Problem: Shape and size of correlation window - Small Window Ambiguities / Noise - Large Window Problems at Depth Discontinuities Depth Edge Preserving Local Stereo Correlation Window Smooth Disparity Delimited by depth edges + Occlusions
Local Stereo Challenging Scene: - Ambiguous patterns, textureless regions, geometrically complex object, thin structures Left View Depth Edges + Occlusion Ground Truth
Local Stereo Conventional 9 x 9 Conventional 31 x 31 Our Approach 31 x 31 --- Conventional Stereo Our Approach
Global Stereo Global Optimization – Markov Random Field (MAP-MRF) X = {xs} Disparity of each pixel (Hidden) Y = {ys} Matching cost at each disparity (Observed) y 1 X 3 X 4 X 1 X 5 y 2 X 8 X 6 X 7
Global Stereo Global Optimization – Markov Random Field (MAP-MRF) X = {xs} Disparity of each pixel (Hidden) Y = {ys} Matching cost at each disparity (Observed) Data Term Smoothness Term Inference by Belief Propagation [Jian Sun et al, 2003]
Global Stereo § Qualitative Depth Map as Evidence - Used to set the smoothness term - Information propagation is stopped at depth edges - Encourage disparities for neighboring pixels according to depth difference in qualitative map § Occlusion Penalty
Global Stereo Conventional Belief Propagation RMS: 0. 9589 Our Approach RMS: 0. 4590
Conclusions § Contributions - Stereo with small baseline illumination - Useful Feature Maps (Qualitative Depth + Occlusion Map) - Enhanced Local and Global Stereo Algorithms § Pros / Cons - Robust, Simple, Inexpensive and Compact - Limited to handle outdoor scenes and motion § Website (datasets, source code) - http: //www. cs. ucsb. edu/~rferis/multi-flash-stereo
Thank you ! Multi-Flash Stereo Webpage http: //www. cs. ucsb. edu/~rferis/multi-flash-stereo Four Eyes Lab, UCSB http: //ilab. cs. ucsb. edu
Occlusion Bounded by Shadows Images taken with light sources surrounding the other camera Occlusion Detection by averaging length of shadows
- Slides: 38