Perspective Projection Direct Linear Transformation Traiangulation PERSPECTIVE PROJECTION

  • Slides: 25
Download presentation

 • Perspective Projection • Direct Linear Transformation • Traiangulation

• Perspective Projection • Direct Linear Transformation • Traiangulation

PERSPECTIVE PROJECTION

PERSPECTIVE PROJECTION

Perspective projection Image plane Focal length Camera frame Scene point Optical axis Image coordinates

Perspective projection Image plane Focal length Camera frame Scene point Optical axis Image coordinates Thus far, in camera’s reference frame only.

Camera parameters • Extrinsic: location and orientation of camera frame with respect to reference

Camera parameters • Extrinsic: location and orientation of camera frame with respect to reference frame • Intrinsic: how to map pixel coordinates to image plane coordinates Reference frame Camera 1 frame

Extrinsic camera parameters Camera reference frame World reference frame

Extrinsic camera parameters Camera reference frame World reference frame

Camera parameters • Extrinsic: location and orientation of camera frame with respect to reference

Camera parameters • Extrinsic: location and orientation of camera frame with respect to reference frame • Intrinsic: how to map pixel coordinates to image plane coordinates Reference frame Camera 1 frame

Intrinsic camera parameters Ignoring any geometric distortions from • optics, we can describe them

Intrinsic camera parameters Ignoring any geometric distortions from • optics, we can describe them by: Coordinates of projected point in camera reference frame Coordinates of image point in pixel units Coordinates of image center in pixel units Effective size of a pixel (mm)

Camera parameters • We know that in terms of camera reference frame: and •

Camera parameters • We know that in terms of camera reference frame: and • Substituting previous eqns describing intrinsic and extrinsic parameters, can relate pixels coordinates to world points: Ri = Row i of rotation matrix

Projection matrix point in camera coordinates • This can be rewritten as a matrix

Projection matrix point in camera coordinates • This can be rewritten as a matrix product using homogeneous coordinates: • The motion of the camera is equivalent to the inverse motion of the scene Internal parameters K= wxim wyim w = K[R t] External parameters Xw Yw Zw 1

Calibrating a camera • Compute intrinsic and extrinsic parameters using observed camera data •

Calibrating a camera • Compute intrinsic and extrinsic parameters using observed camera data • Main idea • Place “calibration object” with known geometry in the scene • Get correspondences • Solve for mapping from scene to image: estimate M=K[R t]

Projection matrix Pw in homog. • This can be rewritten as a matrix product

Projection matrix Pw in homog. • This can be rewritten as a matrix product using homogeneous coordinates: wxim wyim w = K[R t] Xw Yw Zw 1 product M is single projection matrix encoding both extrinsic and intrinsic parameters Let Mi be row i of matrix M

Projection matrix Pw in homog. • This can be rewritten as a matrix product

Projection matrix Pw in homog. • This can be rewritten as a matrix product using homogeneous coordinates: wxim wyim w = K[R t] Xw Yw Zw 1 product M is single projection matrix encoding both extrinsic and intrinsic parameters Let Mi be row i of matrix M

Estimating the projection matrix For a given feature point

Estimating the projection matrix For a given feature point

Estimating the projection matrix Expanding this first equation, we have:

Estimating the projection matrix Expanding this first equation, we have:

Estimating the projection matrix

Estimating the projection matrix

Estimating the projection matrix This is true for every feature point, so we can

Estimating the projection matrix This is true for every feature point, so we can stack up n observed image features and their associated 3 d points Pm = 0 in single equation: P … … … m … … Solve for mij’s (the calibration information) [F&P Section 3. 1]

Summary: camera calibration • Associate image points with scene points on object with known

Summary: camera calibration • Associate image points with scene points on object with known geometry • Use together with perspective projection relationship to estimate projection matrix • (Can also solve for explicit parameters themselves)

DIRECT LINEAR TRANSFORMATION

DIRECT LINEAR TRANSFORMATION

Direct Linear Transformation from 3 D to 2 D • Given two cameras (a

Direct Linear Transformation from 3 D to 2 D • Given two cameras (a stereo pair) and a 3 D object with known 3 D coordinates we can: 1) For each camera compute the direct linear transformation from 3 D to 2 D • Now we can move the stereo pair to a different place, take a pair of pictures and recover 3 D information: 1) Given two camera matrices and a pair of matching points, we can triangulate to get its depth. 2) We’ll talk about triangulation later in class

When would we calibrate this way? • Makes sense when geometry of system is

When would we calibrate this way? • Makes sense when geometry of system is not going to change over time • …When would it change?

TRIANGULATION

TRIANGULATION

Triangulation • Problem: Given some points in correspondence across two or more images (taken

Triangulation • Problem: Given some points in correspondence across two or more images (taken from calibrated cameras), {(uj, vj)}, compute the 3 D location X 24

Triangulation • Method I: intersect viewing rays in 3 D, minimize: X – X

Triangulation • Method I: intersect viewing rays in 3 D, minimize: X – X is the unknown 3 D point – Cj is the optical center of camera j – Vj is the viewing ray for pixel (uj, vj) – sj is unknown distance along Vj • Advantage: geometrically intuitive 25 Vj Cj

Triangulation • Method II: solve linear equations in X – advantage: very simple •

Triangulation • Method II: solve linear equations in X – advantage: very simple • Method III: non-linear minimization – advantage: most accurate (image plane error) 26