Single View Metrology Class 3 3 D photography

Single View Metrology Class 3

3 D photography course schedule (tentative) Lecture Exercise Sept 17 Introduction - Sept 24 Geometry & Camera model Camera calibration Oct. 1 Single View Metrology Measuring in images Oct. 8 Feature Tracking/matching Correspondence computation (Jan-Michael Frahm) Oct. 15 Epipolar Geometry F-matrix computation (David Gallup) Oct. 22 Shape-from-Silhouettes Visual-hull computation Oct. 29 Stereo matching Project proposals Nov. 5 Structured light and active range sensing Papers (TBD) Nov. 12 Structure from motion Papers Nov. 19 Multi-view geometry and self-calibration Papers Nov. 26 Shape-from-X Papers Dec. 3 3 D modeling and registration Papers Dec. 10 Appearance modeling and image-based rendering Papers (TBD) Dec. 17 Final project presentations

Single View Metrology

Measuring in a plane Need to compute H as well as uncertainty

Direct Linear Transformation (DLT)

• Direct Linear Transformation (DLT) Equations are linear in h • Only 2 out of 3 are linearly independent (indeed, 2 eq/pt) (only drop third row if wi’≠ 0) • Holds for any homogeneous representation, e. g. (xi’, yi’, 1)

Direct Linear Transformation (DLT) • Solving for H size A is 8 x 9 or 12 x 9, but rank 8 Trivial solution is h=09 T is not interesting 1 -D null-space yields solution of interest pick for example the one with

Direct Linear Transformation (DLT) • Over-determined solution No exact solution because of inexact measurement i. e. “noise” Find approximate solution - Additional constraint needed to avoid 0, e. g. not possible, so minimize

DLT algorithm Objective Given n≥ 4 2 D to 2 D point correspondences {xi↔xi’}, determine the 2 D homography matrix H such that xi’=Hxi Algorithm (i) For each correspondence xi ↔xi’ compute Ai. Usually only two first rows needed. (ii) Assemble n 2 x 9 matrices Ai into a single 2 nx 9 matrix A (iii) Obtain SVD of A. Solution for h is last column of V (iv) Determine H from h

Importance of normalization ~102 1 ~104 ~102 orders of magnitude difference! Monte Carlo simulation for identity computation based on 5 points (not normalized ↔ normalized)

Normalized DLT algorithm Objective Given n≥ 4 2 D to 2 D point correspondences {xi↔xi’}, determine the 2 D homography matrix H such that xi’=Hxi Algorithm (i) Normalize points (ii) Apply DLT algorithm to (iii) Denormalize solution

Geometric distance measured coordinates estimated coordinates true coordinates d(. , . ) Euclidean distance (in image) Error in one image e. g. calibration pattern Symmetric transfer error Reprojection error

Reprojection error

Statistical cost function and Maximum Likelihood Estimation • Optimal cost function related to noise model • Assume zero-mean isotropic Gaussian noise (assume outliers removed) Error in one image Maximum Likelihood Estimate

Statistical cost function and Maximum Likelihood Estimation • Optimal cost function related to noise model • Assume zero-mean isotropic Gaussian noise (assume outliers removed) Error in both images Maximum Likelihood Estimate

Gold Standard algorithm Objective Given n≥ 4 2 D to 2 D point correspondences {xi↔xi’}, determine the Maximum Likelyhood Estimation of H (this also implies computing optimal xi’=Hxi) Algorithm (i) Initialization: compute an initial estimate using normalized DLT or RANSAC (ii) Geometric minimization of reprojection error: ● Minimize using Levenberg-Marquardt over 9 entries of h or Gold Standard error: ● compute initial estimate for optimal {xi} ● minimize cost over {H, x 1, x 2, …, xn} ● if many points, use sparse method

Uncertainty: error in one image (i) Estimate the transformation from the data (ii) Compute Jacobian , evaluated at (iii) The covariance matrix of the estimated is given by

Uncertainty: error in both images separate in homography and point parameters

Using covariance matrix in point transfer Error in one image Error in two images (if h and x independent, i. e. new points)

Example: s=1 pixel S=0. 5 cm (Criminisi’ 97)

Example: s=1 pixel S=0. 5 cm (Criminisi’ 97)

Example: (Criminisi’ 97)

Monte Carlo estimation of covariance • To be used when previous assumptions do not hold (e. g. non-flat within variance) or to complicate to compute. • Simple and general, but expensive • Generate samples according to assumed noise distribution, carry out computations, observe distribution of result

Single view measurements: 3 D scene

Background: Projective geometry of 1 D 3 DOF (2 x 2 -1) The cross ratio Invariant under projective transformations

Vanishing points • Under perspective projection points at infinity can have a finite image • The projection of 3 D parallel lines intersect at vanishing points in the image

Basic geometry

Basic geometry • Allows to relate height of point to height of camera

Homology mapping between parallel planes • Allows to transfer point from one plane to another

Single view measurements

Single view measurements

Forensic applications 190. 6± 4. 1 cm 190. 6± 2. 9 cm A. Criminisi, I. Reid, and A. Zisserman. Computing 3 D euclidean distance from a single view. Technical Report OUEL 2158/98, Dept. Eng. Science, University of Oxford, 1998.

Example courtesy of Antonio Criminisi

La Flagellazione di Cristo (1460) Galleria Nazionale delle Marche by Piero della Francesca (1416 -1492) http: //www. robots. ox. ac. uk/~vgg/projects/Single. View/

More interesting stuff • Criminisi demo http: //www. robots. ox. ac. uk/~vgg/presentations/ spie 98/criminis/index. html • work by Derek Hoiem on learning single view 3 D structure and apps http: //www. cs. cmu. edu/~dhoiem/ • similar work by Ashutosh Saxena on learning single view depth http: //ai. stanford. edu/~asaxena/learningdepth/

Next class • Feature tracking and matching