A Bayesian Approach For 3 D Reconstruction From

A Bayesian Approach For 3 D Reconstruction From a Single Image Presented By: Erick Delage Supervisor: Prof. Andrew Y. Ng AI Laboratory, Stanford University Erick Delage, Stanford University, 2005

Autonomous Monocular Vision Depth Reconstruction for Indoor Image Can a robot reconstruct 3 D from a single image? Erick Delage, Stanford University, 2005 2

Review of Publications Popular 3 d reconstruction Þ Stereo Vision (Trucco et Verri, 1998) Þ Structure from Motion Single View 3 d reconstruction Þ Shape from Shading (Zhang et al. , 1999) Þ 3 d Metrology (Criminisi et al. , 2000) Our Goal To develop an autonomous algorithm that recovers 3 D information from a single image in a complex environment Erick Delage, Stanford University, 2005 3

Simplification of the Problem Assumptions 1. 2. 3. 4. Image contains flat floor and walls Camera is parallel to the ground plane The camera is at a known height above the ground The image is obtained by perspective projection 5. Our Theory: 6. Given the floor boundary position, the 3 D coordinates in an image of all points can be recovered Erick Delage, Stanford University, 2005 4

General Approach Erick Delage, Stanford University, 2005 5

General Approach (2) Prior Knowledge about Indoor + Machine Learning Þ Image Analysis Þ Floor Boundary detection (Machine Learning) Þ 3 D reconstruction Erick Delage, Stanford University, 2005 6

Floor Boundary Detection Input Image Magnitude of Image gradient Difference in chromatic space Difference from the floor color How can we combine these image features for floor boundary detection ? Erick Delage, Stanford University, 2005 7

Floor Boundary Detection Input Image Using Logistic Regression : (Martin, D. R. , et al. , 2002) Training Mask The model was trained using 25 labeled images of a diverse range of indoor environments on Stanford’s campus Erick Delage, Stanford University, 2005 8

Floor Boundary Detection : Results Erick Delage, Stanford University, 2005 9

Floor Boundary Detection : Results Precision = “true positives” / “all positives” Recall = “true positives” / “all true’s” Erick Delage, Stanford University, 2005 10

Bayesian Inference on Floor Boundary Can we use prior knowledge about the structure of floors and their boundaries? Erick Delage, Stanford University, 2005 11

Bayesian Inference D 1 D 3 … DN Y 1 Y 3 … YN X 1 X 3 … XN … C Di : Direction of the floor boundary in column i Yi : Position of floor boundary in column i Xi : Local image features C : Color of the floor Erick Delage, Stanford University, 2005 12

Bayesian Inference D 1 D 3 … DN Y 1 Y 3 … YN X 1 X 3 … XN … C : initial distribution of variables Erick Delage, Stanford University, 2005 13

Bayesian Inference D 1 D 3 … DN Y 1 Y 3 … YN X 1 X 3 … XN … C Erick Delage, Stanford University, 2005 14

Bayesian Inference D 1 D 3 … DN Y 1 Y 3 … YN X 1 X 3 … XN … C Erick Delage, Stanford University, 2005 15

Bayesian Inference D 1 D 3 … DN Y 1 Y 3 … YN X 1 X 3 … XN … C from the detection algorithm Erick Delage, Stanford University, 2005 16

Bayesian Inference D 1 D 3 … DN Y 1 Y 3 … YN X 1 X 3 … XN … C Erick Delage, Stanford University, 2005 17

Training / Bayesian Inference § 60 images of indoor environment in 8 different buildings of Stanford’s campus § Leave-one-out cross-validation: train on 7 buildings, test on 1 § Parameters for density models estimated from training data using Maximum Likelihood § Exact inference on graph done using Viterbi-like algorithm Erick Delage, Stanford University, 2005 18

Results – Floor Boundary Detection Erick Delage, Stanford University, 2005 19

Results – Floor Boundary Detection Erick Delage, Stanford University, 2005 20

3 D Reconstruction Erick Delage, Stanford University, 2005 21

3 D Reconstruction Erick Delage, Stanford University, 2005 22

3 D Reconstruction Extra Material: Exemples #1, #2, #3 Or at: http: //www. stanford. edu/~edelage/indoor 3 drecon Erick Delage, Stanford University, 2005 23

Performance Precision of floor boundary in segmentation Precision of floor boundary in 3 d localization Erick Delage, Stanford University, 2005 24

Conclusion Monocular 3 d reconstruction is a good example of an ambiguous problem that can be resolved using prior knowledge about the domain n The presented Bayesian network proves high efficiency in learning prior knowledge necessary for this application n This is the first autonomous algorithm for depth recovery in a rich, textured indoor scene. n Erick Delage, Stanford University, 2005 25

Future Work Apply graphical modeling for more complex geometry. n Formulate the problem in a form that scales precision performance with depth of objects. n Embed this approach in real robot navigation problem (ex. RC car, night indoor navigation) n Erick Delage, Stanford University, 2005 26

Questions ? Erick Delage, Stanford University, 2005 27

References Criminisi, A. , Reid, I. , & Zisserman, A. (2000). Single View Metrology. IJCV, 40, 123 -148. Martin, D. R. , Fowlkes, C. C. , & Malik, J. (2002). Learning to Detect Natural Image Boundaries using Brightness and Texture. NIPS. Trucco, E. , & Verri, A. (1998). Introductory techniques for 3 d computer vision. Prentice Hall. Zhang, R. , Tsai, P. -S. , Cryer, J. E. , & M. Shah (1999). Shape from shading: a survey. IEEE Trans. On PAMI, 21, 690 -706. Erick Delage, Stanford University, 2005 28