Stanford CS 223 B Computer Vision Winter 2006
Stanford CS 223 B Computer Vision, Winter 2006 Lecture 4 Camera Calibration Professor Sebastian Thrun CAs: Dan Maynes-Aminzade and Mitul Saha [with slides by D Forsyth, D. Lowe, M. Polleyfeys, C. Rasmussen, G. Loy, D. Jacobs, J. Rehg, A, Hanson, G. Bradski, …] Sebastian Thrun CS 223 B Computer Vision, Winter 2005 1
Today’s Goals • • Calibration: Problem definition Solution via Singular Value Decomposition Solution by nonlinear Least Squares Distortion Sebastian Thrun CS 223 B Computer Vision, Winter 2005 2
Intrinsic Camera Parameters • Determine the intrinsic parameters of a camera (with lens) • What are Intrinsic Parameters? Sebastian Thrun CS 223 B Computer Vision, Winter 2005 3
Perspective Projection, Remember? O X Z Sebastian Thrun -x f CS 223 B Computer Vision, Winter 2005 4
Intrinsic Camera Parameters • Determine the intrinsic parameters of a camera (with lens) • Intrinsic Parameters: – Focal Length f – Pixel size sx , sy – Distortion coefficients k 1 , k 2… – Image center ox , oy Sebastian Thrun CS 223 B Computer Vision, Winter 2005 5
A Quiz • Can we determine all intrinsic parameters by … exposing the camera to many known objects? Sebastian Thrun CS 223 B Computer Vision, Winter 2005 6
Example Calibration Pattern Sebastian Thrun CS 223 B Computer Vision, Winter 2005 7
Our Calibration target Sebastian Thrun CS 223 B Computer Vision, Winter 2005 8
Harris Corner Detector Sebastian Thrun CS 223 B Computer Vision, Winter 2005 9
Another Quiz (the last today) • How Many Flat Calibration Targets are Needed for Calibration? 1: 2: 3: 4: 5: 10 • How Many Corner Points do we need in Total? 1: 2: 3: 4: 10: 20 Sebastian Thrun CS 223 B Computer Vision, Winter 2005 10
Experiment 1: Parallel Board Sebastian Thrun CS 223 B Computer Vision, Winter 2005 11
Projective Perspective of Parallel Board 10 cm Sebastian Thrun 20 cm 30 cm CS 223 B Computer Vision, Winter 2005 12
Experiment 2: Tilted Board Sebastian Thrun CS 223 B Computer Vision, Winter 2005 13
Projective Perspective of Tilted Board 10 cm 20 cm 50 cm 100 cm Sebastian Thrun 30 cm 500 cm CS 223 B Computer Vision, Winter 2005 14
Perspective Camera Model Object Space Sebastian Thrun CS 223 B Computer Vision, Winter 2005 15
Calibration: 2 steps • Step 1: Transform into camera coordinates • Step 2: Transform into image coordinates Sebastian Thrun CS 223 B Computer Vision, Winter 2005 16
Calibration Model (extrinsic) Homogeneous Coordinates Sebastian Thrun CS 223 B Computer Vision, Winter 2005 17
Homogeneous Coordinates • Idea: Most Operations Become Linear! • Extract Image Coordinates by Znormalization Sebastian Thrun CS 223 B Computer Vision, Winter 2005 18
Advantage of Homogeneous C’s i-th data point Sebastian Thrun CS 223 B Computer Vision, Winter 2005 19
Calibration Model (intrinsic) Focal length Pixel size Image center Sebastian Thrun CS 223 B Computer Vision, Winter 2005 20
Intrinsic Transformation Sebastian Thrun CS 223 B Computer Vision, Winter 2005 21
Plugging the Model Together! Sebastian Thrun CS 223 B Computer Vision, Winter 2005 22
Summary Parameters • Extrinsic – Rotation – Translation • Intrinsic – Focal length – Pixel size – Image center coordinates – (Distortion coefficients) Sebastian Thrun CS 223 B Computer Vision, Winter 2005 23
Q: Can We recover all Intrinsic Params? • No Sebastian Thrun CS 223 B Computer Vision, Winter 2005 24
Summary Parameters, Revisited • Extrinsic – Rotation – Translation • Intrinsic – Focal length, in pixel units – Pixel size Aspect ratio – Image center coordinates – (Distortion coefficients) Sebastian Thrun CS 223 B Computer Vision, Winter 2005 25
Today’s Goals • • Calibration: Problem definition Solution via Singular Value Decomposition Solution by nonlinear Least Squares Distortion Sebastian Thrun CS 223 B Computer Vision, Winter 2005 26
Calibration via SVD Sebastian Thrun CS 223 B Computer Vision, Winter 2005 27
Calibration via SVD N>=7 points, not coplanar Sebastian Thrun CS 223 B Computer Vision, Winter 2005 28
Calibration via SVD Sebastian Thrun CS 223 B Computer Vision, Winter 2005 29
Calibration via SVD A has rank 7 (without proof) Sebastian Thrun CS 223 B Computer Vision, Winter 2005 30
Calibration via SVD • Remaining Problem: • See book Sebastian Thrun CS 223 B Computer Vision, Winter 2005 31
Summary, SVD Solution • Replace rotation matrix by arbitrary matrix • Transform into linear set of equations • Solve via SVD • Enforce rotation matrix (see book) • Solve for remaining parameters (see book) SVD solution: algebraic minimization, assume Gaussian noise in parameter space Sebastian Thrun CS 223 B Computer Vision, Winter 2005 32
Today’s Goals • • Calibration: Problem definition Solution via Singular Value Decomposition Solution by nonlinear Least Squares Distortion Sebastian Thrun CS 223 B Computer Vision, Winter 2005 33
Calibration by nonlinear Least Squares … • Calibration Examples: Sebastian Thrun CS 223 B Computer Vision, Winter 2005 34
Calibration by nonlinear Least Squares • Least Squares Sebastian Thrun CS 223 B Computer Vision, Winter 2005 35
Calibration by nonlinear Least Squares • Least Mean Square • Gradient descent: Sebastian Thrun CS 223 B Computer Vision, Winter 2005 36
Summary Non-Linear Least Squares • Solve nonlinear equations via gradient descent • Assume Gaussian noise in image space, not parameter space Sebastian Thrun CS 223 B Computer Vision, Winter 2005 37
SVD Versus LQ SVD • Minimization of squared distance in parameter space • Globally optimal Sebastian Thrun Nonlin Least Squares • Minimization of squared distance in Image space • Locally optimal CS 223 B Computer Vision, Winter 2005 38
Q: How Many Images Do We Need? • • • Assumption: K images with M corners each 4+6 K parameters 2 KM constraints 2 KM 4+6 K M>3 and K 2/(M-3) 2 images with 4 points, but will 1 images with 5 points work? • No, since points cannot be co-planar! Sebastian Thrun CS 223 B Computer Vision, Winter 2005 39
Today’s Goals • • Calibration: Problem definition Solution via Singular Value Decomposition Solution by nonlinear Least Squares Distortion Sebastian Thrun CS 223 B Computer Vision, Winter 2005 40
Advanced Calibration: Nonlinear Distortions • Barrel and Pincushion • Tangential Sebastian Thrun CS 223 B Computer Vision, Winter 2005 41
Barrel and Pincushion Distortion wideangle Sebastian Thrun tele CS 223 B Computer Vision, Winter 2005 42
Models of Radial Distortion distance from center Sebastian Thrun CS 223 B Computer Vision, Winter 2005 43
Tangential Distortion cheap CMOS chip cheap lense image cheap glue cheap camera Sebastian Thrun CS 223 B Computer Vision, Winter 2005 44
Image Rectification (to be continued) Sebastian Thrun CS 223 B Computer Vision, Winter 2005 45
Summary • • Calibration: Problem definition Solution via Singular Value Decomposition Solution by nonlinear Least Squares Distortion Sebastian Thrun CS 223 B Computer Vision, Winter 2005 46
- Slides: 46