TwoView Geometry Jana Kosecka George Mason University Geyer

Two-View Geometry Jana Kosecka George Mason University Geyer, Daniilidis, UPenn ICRA 2003

General Formulation Two view geometry ICRA 2003 2

Pinhole Camera Imaging Model • Image points First frame is the reference Second frame • Moving camera ICRA 2003 3

Rigid Body Motion – Two Views Camera motion is represented as the special Euclidean group ICRA 2003 4

3 D Structure and Motion Recovery Euclidean transformation measurements unknowns Find such Rotation and Translation and Depth that the reprojection error is minimized Optimization ICRA 2003 5

Two views ~ 200 points 6 unknowns – Motion 3 Rotation 3 Translation - Structure 200 x 3 coordinates - (-) universal scale Difficult optimization problem ICRA 2003 6

Epipolar Geometry Image correspondences Algebraic Elimination of Depth [Longuet-Higgins ’ 81]: Essential matrix

Epipolar Geometry • Epipolar lines • Epipoles Image correspondences ICRA 2003 8

Characterization of Essential Matrix Essential matrix Special 3 x 3 matrix Theorem 1 a (Essential Matrix Characterization) [Huang and Faugeras] A non-zero matrix is an essential matrix iff its SVD: satisfies: with and ICRA 2003 9

Pose Recovery from Essential Matrix Essential matrix Theorem 1 a (Pose Recovery) There are two relative poses with corresponding to a non-zero matrix essential matrix. and • Twisted pair ambiguity ICRA 2003 10

Estimating Essential Matrix Special 3 x 3 matrix Essential matrix Given n pairs of image correspondences: Find such Rotation and Translation that the epipolar error is minimized Space of all Essential Matrices is 5 dimensional 3 Degrees of Freedom – Rotation 2 Degrees of Freedom – Translation (up to scale !) ICRA 2003 11

Estimating Essential Matrix denote rewrite collect constraints from all points

Estimating Essential Matrix Solution • eigenvector associated with the smallest eigenvalue of • if degenerate configuration E

Projection on to Essential Space Theorem 2 a (Project to Essential Manifold) [Toscani and Faugeras ’ 86] If the SVD of a matrix is given by then the essential matrix which minimizes the Frobenius distance is given by with E

Two view linear algorithm (8 -point) • Solve the LLSE problem: followed by projection • Project onto the essential manifold: SVD: E is 5 diml. sub. mnfld. in • 8 -point linear algorithm • Recover the unknown pose: ICRA 2003 15

Pose Recovery There are exactly two pairs matrix. There also two pairs matrix. corresponding to each essential Positive depth constraint - used to disambiguate the physically impossible solutions • Translation has to be non-zero • Points have to be in general position - degenerate configurations – planar points - quadratic surface Prerequisite – we need to have the correspondence of at least 8 points • Nonlinear 5 -point algorithms yield up to 10 solutions ICRA 2003 16

3 D Structure Recovery • Eliminate one of the scale’s • Solve LLSE problem ICRA 2003 17

Summary n If the configuration is non-critical, the Euclidean structure of then points and motion of the camera can be reconstructed up to a universal scale. ICRA 2003 18

Rigid Body Motion – Continuous Case Image velocities Perspective Projection: • Image points First frame is the reference ICRA 2003 19

Two views differential case Algebraic elimination of depth Only symmetric component of can be recovered Differential epipolar constraint Continuous Essential Matrix ICRA 2003 20

Continuous Essential Matrix Special Symmetric Component Theorem 1 b (Special Symmetric Matrix Characterization) A matrix is a special symmetric matrix iff it can be diagonalized as with and. Theorem 2 b (Project to Special Symmetric Space) If a symmetric matrix is diagonalized as with and then the special symmetric matrix which minimizes the Frobenius distance is given by with ICRA 2003 21

Two view Linear Algorithm (continuous case) Sllse • Solve the LLSE problem: , S is 5 diml. sub. mnfld. in • Project onto the special sym. space: • Recover from