Multiple View Geometry THE GEOMETRY OF MULTIPLE VIEWS

Multiple View Geometry

THE GEOMETRY OF MULTIPLE VIEWS • Epipolar Geometry • The Essential Matrix • The Fundamental Matrix • The Trifocal Tensor • The Quadrifocal Tensor Reading: Chapter 10.

Epipolar Geometry • Epipolar Plane • Epipoles • Epipolar Lines • Baseline

Epipolar Constraint • Potential matches for p have to lie on the corresponding epipolar line l’. • Potential matches for p’ have to lie on the corresponding epipolar line l.

Epipolar Constraint: Calibrated Case Essential Matrix (Longuet-Higgins, 1981)

Properties of the Essential Matrix T • E p’ is the epipolar line associated with p’. T • ETp is the epipolar line associated with p. • E e’=0 and ETe=0. • E is singular. • E has two equal non-zero singular values (Huang and Faugeras, 1989).

Epipolar Constraint: Small Motions To First-Order: Pure translation: Focus of Expansion

Epipolar Constraint: Uncalibrated Case Fundamental Matrix (Faugeras and Luong, 1992)

Properties of the Fundamental Matrix • F p’ T is the epipolar line associated with p’. • FT p is the epipolar line associated with p. T • F e’=0 • F is singular. and FT e=0.

The Eight-Point Algorithm (Longuet-Higgins, 1981) Minimize: under the constraint |F |2 =1.

Non-Linear Least-Squares Approach (Luong et al. , 1993) Minimize with respect to the coefficients of F , using an appropriate rank-2 parameterization.

Problem with eight-point algorithm linear least-squares: unit norm vector F yielding smallest residual What happens when there is noise?

The Normalized Eight-Point Algorithm (Hartley, 1995) • Center the image data at the origin, and scale it so the mean squared distance between the origin and the data points is sqrt(2) pixels: i q =i T p i, q’ =i T’ p’. • Use the eight-point algorithm to compute F from the points q and q’. i i • Enforce the rank-2 constraint. • Output T F T’. T

Weak-calibration Experiments

Epipolar geometry example

Example: converging cameras courtesy of Andrew Zisserman

Trinocular Epipolar Constraints These constraints are not independent!

Trinocular Epipolar Constraints: Transfer Given p and p , p can be computed 1 2 3 as the solution of linear equations.

Trinocular Epipolar Constraints: Transfer l problem for epipolar transfer in trifocal plane! There must be more to trifocal geometry… image from Hartley and Zisserman

Trifocal Constraints

Trifocal Constraints Calibrated Case All 3 x 3 minors must be zero! Trifocal Tensor

Trifocal Constraints Uncalibrated Case Trifocal Tensor

Trifocal Constraints: 3 Points Pick any two lines l 2 and l 3 through p 2 and p 3. Do it again. T( p 1 , p )=0 2 3

Properties of the Trifocal Tensor • T i For any matching epipolar lines, l 2 G 1 l 3 = 0. • The matrices G i are singular. 1 • They satisfy 8 independent constraints in the uncalibrated case (Faugeras and Mourrain, 1995). Estimating the Trifocal Tensor • Ignore the non-linear constraints and use linear least -squares • Impose the constraints a posteriori.

For any matching epipolar lines, l TG i l 2 1 3 = 0. The backprojections of the two lines do not define a line!

Trifocal Tensor Example 108 putative matches (26 samples) 88 inliers (0. 43) (0. 23) 18 outliers 95 final inliers (0. 19) courtesy of Andrew Zisserman

Trifocal Tensor Example additional line matches images courtesy of Andrew Zisserman

Transfer: trifocal transfer (using tensor notation) doesn’t work if l’=epipolar line image courtesy of Hartley and Zisserman

Image warping using T(1, 2, N) (Avidan and Shashua `97)

Multiple Views (Faugeras and Mourrain, 1995)

Two Views Epipolar Constraint

Three Views Trifocal Constraint

Four Views Quadrifocal Constraint (Triggs, 1995)

Geometrically, the four rays must intersect in P. .

Quadrifocal Tensor and Lines

Quadrifocal tensor l determinant is multilinear thus linear in coefficients of lines l ! There must exist a tensor with 81 coefficients containing all possible combination of x, y, w coefficients for all 4 images: the quadrifocal tensor