A Bayesian Estimation of Building Shape using MCMC

A Bayesian Estimation of Building Shape using MCMC Anthony Dick 1 1 Phil Torr 2 Department of Engineering University of Cambridge Roberto Cipolla 1 2 Microsoft Research, Cambridge Bayesian Estimation of Building Shape using MCMC - ECCV'02

Our goal • Reconstruction and recognition of architecture Bayesian Estimation of Building Shape using MCMC - ECCV'02 2

Our approach • Generic method that uses domain-specific prior information to interpret ambiguous or misleading image data • Bayesian framework for incorporating prior knowledge with image data • Given this framework, we have 3 main problems: – How to represent our model – How to formulate our prior knowledge – How to estimate the best model for our data Bayesian Estimation of Building Shape using MCMC - ECCV'02 3

Shape representation • Model is a collection of “wall” planes • Each wall plane may contain primitives defined by 4 – 8 parameters Eg: Window Door Pediment Pedestal Entablature Column Buttress Drainpipe c b Example shape (window) (x, y) a Front view Bayesian Estimation of Building Shape using MCMC - ECCV'02 d r a Overhead view 4

The prior • There is a prior on shape and texture • A texture prior is learnt for each type of primitive – Show examples of the primitive from a suite of approx 100 architectural images – Details in [ICCV 01] • A shape prior is more problematic – Shape + scale of individual primitives – Layout of multiple primitives (e. g. alignment, symmetry) • We use MCMC to simulate sampling from this complex distribution – Requires a decent starting point Bayesian Estimation of Building Shape using MCMC - ECCV'02 5

Sampling the shape prior • Requires a scoring function and jumping distributions • Scoring function is a function of the model parameters – Combination of scale, shape, alignment and symmetry terms • Jumping distribution is a mixture of several types of jump: – – – Add/Remove/Modify shape Add/Remove/Modify wall Add/Remove/Modify row/column of shapes Regularise row/column of shapes Symmetrise row/column of shapes Perturb row/column of shapes Bayesian Estimation of Building Shape using MCMC - ECCV'02 6

Verifying the shape prior • “Seed” buildings: • Samples on a city grid: Bayesian Estimation of Building Shape using MCMC - ECCV'02 7

Model estimation • Initial shape estimate obtained via existing structure and motion algorithms – Extract and match corners and lines – Self-calibrate cameras – Plane fitting post-process to estimate walls • Search for likely primitives on each wall [ICCV 01] – This produces seed points for the MCMC process – Likelihood measure is based on sum squared error of reprojected pixels • Assumes Lambertian model • Now run Reversible Jump MCMC on seed models – At least 2000 iterations Bayesian Estimation of Building Shape using MCMC - ECCV'02 8

Reconstructed model Bayesian Estimation of Building Shape using MCMC - ECCV'02 9

Completed model Bayesian Estimation of Building Shape using MCMC - ECCV'02 10

Other likely models Sills Included: Split windows: Door Not included: Extra Columns: Bayesian Estimation of Building Shape using MCMC - ECCV'02 11

Effect of prior • Wall of Downing College library: Without alignment prior With alignment prior Bayesian Estimation of Building Shape using MCMC - ECCV'02 12

Effect of prior Classical shape prior Gothic shape prior Bayesian Estimation of Building Shape using MCMC - ECCV'02 13

Gothic model Bayesian Estimation of Building Shape using MCMC - ECCV'02 14

Completed model Bayesian Estimation of Building Shape using MCMC - ECCV'02 15

Ground truth Bayesian Estimation of Building Shape using MCMC - ECCV'02 16

Ground truth comparison Ratio Ground truth Lower Upper Model Lower Upper Height / width 1. 48 1. 52 1. 50 1. 64 Width / depth 5. 67 6. 37 5. 00 7. 40 Wall-col / width 2. 22 2. 24 2. 18 2. 28 Col circ / width 2. 56 2. 77 2. 74 3. 00 • All values within bounds, but the model is less precise than hand measurement • Need for more accurate image measurements e. g. super-resolution Bayesian Estimation of Building Shape using MCMC - ECCV'02 17

Conclusion • Combination of structure from motion and recognition • Use of high level prior information is crucial Bayesian Estimation of Building Shape using MCMC - ECCV'02 18