Unfolding an Indoor Origami World David Fouhey Abhinav

Unfolding an Indoor Origami World David Fouhey, Abhinav Gupta, Martial Hebert 1

2

3

Local Evidence Hoiem et al. 2005, Saxena et al. 2005, Fouhey et al. 2013, etc. 4

Constraints 5

Constraints for Single Image 3 D Local Smoothness Low Level, Generic Hoiem et al. 2005, Saxena et al. 2005, 2008, Munoz et al. , 2009, etc. 6

Constraints for Single Image 3 D Local Smoothness Low Level, Generic Hoiem et al. 2005, Saxena et al. 2005, 2008, Munoz et al. , 2009, etc. 7

Constraints for Single Image 3 D Low Level, Generic High Level, Physical 8

Constraints for Single Image 3 D Local Smoothness Low Level, Generic High Level, Physical Coughlan and Yuille 2000, etc. 9

Constraints for Single Image 3 D Local Smoothness Low Level, Generic High Level, Physical Hedau et al. 2009, Del Pero et al. , 2011, Wang et al. , 2012, Schwing et al. 2012, etc. 10

Constraints for Single Image 3 D Local Smoothness Low Level, Generic High Level, Physical Lee et al. 2010, Xiao et al. 2012, Zhao et al. 2013, Schwing et al. , 2013, etc. 11

Constraints for Single Image 3 D Low Level, Generic High Level, Physical 12

Mid-level in the Past Huffman 71, Clowes 71, Kanade 80, 81 Sugihara 86, Malik 87, etc. 13

Our Mid-Level Constraints 14

This Work Input: Single Image Output: Discrete Scene Parse 15

Overview Parameterization Formulation Experimental Results 16

Overview Parameterization Formulation Experimental Results 17

Parameterization 18

Parameterization vp 2 vp 1 vp 3 VP Estimator from Hedau et al. , 2009 19

Parameterization Two VPs give grid cell 20

Encoding Surface Normals 21

Encoding Surface Normals 22

Encoding Surface Normals 23

Encoding Surface Normals x 1, …, x 400 x 401, …, x 800 x 801, …, x 1200 24

Related Parameterizations vp 2 vp 3 x 4 vp 1 x 2 x 1 x 3 Hedau et al. , 2009; Wang et al. 2010, Schwing et al. , 2012, 2013 25

Overview Parameterization Formulation Experimental Results 26

Parameterization 27

Formulation 28

Unaries 29

Unaries Any 3 D Evidence High c Low c 30

Unaries Local: Data-Driven 3 D Primitives Input 3 D Primitive Bank … Fouhey, Gupta, Hebert, 2013 31

Unaries Global: Cuboid Fit + Clutter Mask Input Predicted Walls Clutter Mask Hedau, Hoiem, Forsyth, 2009 32

Binaries 33

Convex/Concave Constraints Convex (+) Concave (-) 34

Convex/Concave Constraints Detected Concave (-) 35

Convex/Concave Constraints Detected Concave (-) 36

Convex/Concave Constraints Detected Concave (-) 37

Convex/Concave Constraints Detected Concave (-) 38

Convex/Concave Constraints Detected Concave (-) 39

Detecting Convex/Concave Use 3 DP to Transfer Discontinuities Input 3 D Primitive Bank … Ground-Truth Discontinuities similar to Gupta, Arbelaez, Malik, 2013 3 DP from Fouhey, Gupta, Hebert, 2013 40

Smoothness 41

Constraints 42

Solving the Model 43

Overview Parameterization Formulation Experimental Results 44

Dataset NYU Depth v 2: 795 Train, 654 Test 45

Qualitative Results 46

Qualitative Results 47

Qualitative Results 48

Qualitative Results 49

Surface Connection Graphs Convex Concave 50

Baseline Primary Baseline: 3 D Primitives (3 DP) Input Output Fouhey, Gupta, Hebert, 2013 51

Quantitative Results Summary Stats (⁰) (Lower Better) Proposed 3 DP Hedau et al. Lee et al. % Good Pixels (Higher Better) Mean Median RMSE 11. 25⁰ 22. 5⁰ 35. 1 36. 0 40. 0 43. 3 37. 6 35. 9 34. 2 18. 6 19. 2 20. 5 23. 5 36. 3 48. 7 49. 4 54. 1 54. 6 53. 3 52. 0 49. 3 38. 6 30⁰ 58. 9 57. 8 54. 4 49. 9 52

Quantitative Results 53

Failure Modes Mistaken but Confident Evidence 54

Failure Modes Missing High-Level Modeling 55

Conclusions Parameterization Discrete Parse Single Image Formulation 56

Thank You 57