Calibration ECE 847 Digital Image Processing Stan Birchfield
Calibration ECE 847: Digital Image Processing Stan Birchfield Clemson University
Image projection • Project world point (x, y, z) to image point (u, v) • Assume pinhole camera model
Image projection (u, v) image point internal calibration (x, y, z) world point external calibration (pose) Total parameters: 11 (5 internal, 6 external)
Internal calibration parameters where • fx: horizontal focal length • fy: vertical focal length • q: skew (angle b/w horizontal and vertical axes – p/2 for most real cameras • (u 0, v 0): principal point (intersection of optical axis with image plane)
Two approaches to calibration • Calibrate to meaningful fixed world coord system – Solve for P directly – Good for fixed camera, specific application (e. g. , tracking vehicles on highway, mobile robot on ground plane) – But gives no insight into internal calibration parameters – If camera-world relationship changes, calibration must start from scratch • Compute internal and external parameters separately – P = A[R t] – Internal parameters turn camera into a metric device – Can now be used for computing 3 D rays (and points) in Euclidean space – Necessary for SFM
Homography mapping from world plane
Bird’s eye view • Use normalized DLT to find H • Now can unwarp image to provide bird’s eye (top-down) view of scene • This upgrades projective to Euclidean
Stratification of geometries single known length identify circular points identify l∞ • • Euclidean Similarity Affine Projective more transformations, fewer invariants
Calibration algorithms • Tsai 1987 – Uses some known parameters from camera specs – Requires 3 D calibration target with orthogonal planes • Zhang 2000 – Uses projective geometry – Only requires planar calibration target – Much simpler
Zhang’s algorithm • Based on the image of the absolute conic (IAC): • Algorithm at a glance: – Find w∞ – Decompose w∞ to get K • What is the IAC?
Celestial sphere • Celestial sphere is imaginary sphere with infinite radius • Points on sphere (stars) unaffected by translation http: //www. oneminuteastronomer. com/wp-content/uploads/2009/06/celestial-sphere. jpg
Plane at infinity • • p∞ is idealized celestial sphere points at infinity lie on p∞ =(0, 0, 0, 1)T last coordinate is zero invariant to translation
Line at infinity • l∞ is analogous quantity in 2 D • points at infinity lie on l∞ , capture direction Euclidean plane l∞ =(0, 0, 1)T
Absolute points (2 D) • Two special points on l∞ : (1, ±i, 0) • These are the absolute points, or circular points • All circles intersect at absolute points • absolute points satisfy x 2+y 2=0, w=0 • absolute points Euclidean
Ellipses and circles • Q 1: Two ellipses intersect at 4 points, two circles at only 2. Why? • Q 2: An ellipse is defined by 5 points, circle by only 3. Why?
All circles intersect at absolute points Answer: if Therefore, the absolute points lie on every circle
Absolute points are invariant to similarity transformations
Absolute points encode Euclidean (similarity) geometry in a single compact representation
Absolute conic (3 D) • • • All spheres intersect at absolute conic plane at infinity: P∞=(0, 0, 0, 1) points at infinity: (x, y, z, 0) absolute conic satisfies x 2+y 2+z 2=0, w=0 Therefore, W∞=I 3 x 3 (identity matrix) in a metric frame • W∞ contains purely imaginary points • absolute conic Euclidean
IAC • Projection of the absolute conic on the image plane • Image of the absolute conic (IAC) • IAC calibrated camera (internal parameters) • Projection of P∞ onto image plane is homography – Translate camera, no change to this projection (cf. seeing stars on celestial sphere) – Rotate camera, projection does change – But IAC, which is the projection of W∞ , does not change with translation or rotation – H = KR for plane at infinity, but w∞ = P W∞ = K-T K-1
Algorithm overview • Take images of multiple planes • Each plane contains absolute points • Projection of absolute points lie on IAC • Use multiple planes to perform least squares fit to IAC • Decompose IAC to get K (internal parameters)
Zhang’s algorithm • Recall: • Euclidean constraints on rotation matrix: • Substitute to yield constraints on K:
Cholesky decomposition • Recall: 0 0 KT : 0 K: K-1: K-T: • Cholesky factorizes any symmetric and positive definite matrix into product of lower and upper triangular matrices: w∞ = LU 0
Zhang’s algorithm (bundle adjustment)
Geometric interpretation • Draw projection of IAC
Image of absolute points yields same constraints • Projection of plane is homography H • Leads to constraint: • Splitting into real and imaginary:
Tsai’s algorithm
What can you do with a calibrated camera
Radial lens distortion
Radial lens distortion http: //toothwalker. org/optics/distortion. html
Radial lens distortion ru = rd + k 1 rd 3 where k 1 > 0 for barrel distortion and k 1 < 0 for pincushion. http: //www. imatest. com/docs/distortion. html
Lens distortion It is frequently asserted that negative values of D correspond to barrel distortion, and positive values to pincushion distortion. For s Barrel distortion typically will have a positive term for K 1 where as pincushion distortion will have a negative value. -- http: //en. wikipe http: //toothwalker. org/optics/distortion. html
- Slides: 32