Stanford CS 223 B Computer Vision Winter 2007
Stanford CS 223 B Computer Vision, Winter 2007 Lecture 4 Camera Calibration Professors Sebastian Thrun and Jana Kosecka CAs: Vaibhav Vaish and David Stavens Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Today’s Goals • • • Calibration: Problem definition Solution by nonlinear Least Squares Solution via Singular Value Decomposition Homogeneous Coordinates Distortion Calibration Software Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Camera Calibration Perspective Equations Sebastian Thrun and Jana Kosecha Feature Extraction CS 223 B Computer Vision, Winter 2007
Perspective Projection, Remember? O X Z Sebastian Thrun and Jana Kosecha x f CS 223 B Computer Vision, Winter 2007
Intrinsic Camera Parameters • Determine the intrinsic parameters of a camera (with lens) • What are Intrinsic Parameters? (can you name 7? ) Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Intrinsic Parameters O X Z Sebastian Thrun and Jana Kosecha f CS 223 B Computer Vision, Winter 2007
Intrinsic Camera Parameters • Intrinsic Parameters: – Focal Length f – Pixel size sx , sy – Image center ox , oy – (Nonlinear radial distortion coefficients k 1 , k 2…) • Calibration = Determine the intrinsic parameters of a camera Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Why Intrinsic Parameters Matter Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Questions • Can we determine the intrinsic parameters by exposing the camera to many known objects? • If so, – How often do we have to see the object? – How many features on the object do we need? Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Example Calibration Pattern: Object with features of known size/geometry Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Harris Corner Detector (see Assignment 2) Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Why Tilt the Board? Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Experiment 1: Parallel Board Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Projective Perspective of Parallel Board 10 cm Sebastian Thrun and Jana Kosecha 20 cm 30 cm CS 223 B Computer Vision, Winter 2007
Experiment 2: Tilted Board Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Projective Perspective of Tilted Board 10 cm 20 cm 50 cm 100 cm Sebastian Thrun and Jana Kosecha 30 cm 500 cm CS 223 B Computer Vision, Winter 2007
Intrinsics and Extrinsics • Intrinsics: – Focal Length f – Pixel size sx , sy – Image center ox , oy • Extrinsics: – Location and orientation of k-th calib. pattern: Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Perspective Camera Model • Step 1: Transform into camera coordinates • Step 2: Transform into image coordinates Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Perspective Camera Model • Step 1: Transform into camera coordinates • Step 2: Transform into image coordinates Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
The Full Perspective Camera Model Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
The Calibration Problem • Given – Calibration pattern with N corners – K views of this calibration pattern • Recover the intrinsic parameters – We’ll also recover the extrinsics, but we won’t care about them Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Calibration Questions • Can we determine the intrinsic parameters by exposing the camera to many known objects? • If so, – How often do we have to see the object? – How many features on the object do we need? – Do we need to see object at angle? Yes. Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Today’s Goals • • • Calibration: Problem definition Solution by nonlinear Least Squares Solution via Singular Value Decomposition Homogeneous Coordinates Distortion Calibration Software Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Calibration constraints • Step 1: Transform into camera coordinates • Step 2: Transform into image coordinates Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Camera Calibration Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Calibration by nonlinear Least Squares • Least Mean Square • Gradient descent: Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
The Calibration Problem Quiz • Given – Calibration pattern with N corners – K views of this calibration pattern • How large would N and K have to be? • Can we recover all intrinsic parameters? Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Intrinsic Parameters, Degeneracy O X Z Sebastian Thrun and Jana Kosecha f CS 223 B Computer Vision, Winter 2007
Summary Parameters, Revisited • Extrinsic – Rotation – Translation • Intrinsic – Focal length, in pixel units – Pixel size Aspect ratio – Image center coordinates Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
The Calibration Problem Quiz • Given – Calibration pattern with N corners – K views of this calibration pattern • How large would N and K have to be? • Can we recover all intrinsic parameters? N 1 3 4 4 6 K 1 1 3 3 3 4 6 Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Constraints • N points • K images 2 NK constraints • 4 intrinsics (distortion: +2) • 6 K extrinsics need 2 NK ≥ 6 K+4 (N-3)K ≥ 2 Hint: may not be co-linear Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
The Calibration Problem Quiz N 1 3 4 4 6 K 1 1 3 3 3 4 6 No No Yes Yes need (N-3)K ≥ 2 Hint: may not be co-linear Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Problem with Least Squares • Many parameters (=slow) • Many local minima! (=slower) Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Today’s Goals • • • Calibration: Problem definition Solution by nonlinear Least Squares Solution via Singular Value Decomposition Homogeneous Coordinates Distortion Calibration Software Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Perspective Camera Model • Step 1: Transform into camera coordinates • Step 2: Transform into image coordinates Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Calibration Model (extrinsic) (Homogeneous Coordinates) (nonlinear perspective projection) Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Affine Problem Relaxation Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Affine Problem Relaxation Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Calibration via SVD [see Trucco/Verri] Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Calibration via SVD N>=7 points, not coplanar Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Calibration via SVD Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Calibration via SVD A has rank 7 (without proof) Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Calibration via SVD • Remaining Problem: • See book Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Summary, SVD Solution • Replace rotation matrix by arbitrary matrix • Transform into linear set of equations • Solve via SVD • Enforce rotation matrix (see book) • Solve for remaining parameters (see book) Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Comparison Nonlinear least squares • Gaussian image noise • Many local minima • Iterative • Can incorporate nonlinear distortion Sebastian Thrun and Jana Kosecha Singular Value Decomp. • Gaussian parameter noise (algebraic) • No local minima • “Closed” form • No distortion CS 223 B Computer Vision, Winter 2007
Today’s Goals • • • Calibration: Problem definition Solution by nonlinear Least Squares Solution via Singular Value Decomposition Homogeneous Coordinates Distortion Calibration Software Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Homogeneous Coordinates • Idea: In homogeneous coordinates most operations become linear! • Extract Image Coordinates by Znormalization Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Today’s Goals • • • Calibration: Problem definition Solution by nonlinear Least Squares Solution via Singular Value Decomposition Homogeneous Coordinates Distortion Calibration Software Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Advanced Calibration: Nonlinear Distortions • Barrel and Pincushion • Tangential Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Barrel and Pincushion Distortion wideangle Sebastian Thrun and Jana Kosecha tele CS 223 B Computer Vision, Winter 2007
Models of Radial Distortion distance from center Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Tangential Distortion cheap CMOS chip cheap lens image cheap glue cheap camera Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Image Rectification (to be continued) Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Distorted Camera Calibration • Set k 1=k 2=0, solve for undistorted case • Find optimal k 1, k 2 via nonlinear least squares • Iterate Tends to generate good calibrations Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Today’s Goals • • • Calibration: Problem definition Solution by nonlinear Least Squares Solution via Singular Value Decomposition Homogeneous Coordinates Distortion Calibration Software Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Calibration Software: Matlab Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Calibration Software: Open. CV Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
Summary • • • Calibration: Problem definition Solution by nonlinear Least Squares Solution via Singular Value Decomposition Homogeneous Coordinates Distortion Calibration Software Sebastian Thrun and Jana Kosecha CS 223 B Computer Vision, Winter 2007
- Slides: 58