Computer Vision Calibration Marc Pollefeys COMP 256 Read
Computer Vision Calibration Marc Pollefeys COMP 256 Read F&P Chapter 2 Some slides/illustrations from Ponce, Hartley & Zisserman
Tentative class schedule Computer Vision Jan 16/18 - Introduction Jan 23/25 Cameras Radiometry Sources & Shadows Color Feb 6/8 Linear filters & edges Texture Feb 13/15 Multi-View Geometry Stereo Feb 20/22 Optical flow Project proposals Affine Sf. M Projective Sf. M Camera Calibration Segmentation Mar 13/15 Springbreak Mar 20/22 Fitting Prob. Segmentation Mar 27/29 Silhouettes and Photoconsistency Linear tracking Apr 3/5 Project Update Non-linear Tracking Apr 10/12 Object Recognition Apr 17/19 Range data Final project Jan 30/Feb 1 Feb 27/Mar 1 Mar 6/8 2 Apr 24/26
Computer Vision Previously Hierarchy of 3 D transformations Projective 15 dof Affine 12 dof Similarity 7 dof Euclidean 6 dof 3 Intersection and tangency Parallellism of planes, Volume ratios, centroids, The plane at infinity π∞ Angles, ratios of length The absolute conic Ω∞ Volume
Computer Vision Camera calibration Compute relation between pixels and rays in space ? 4
Computer Vision 5 Pinhole camera
Computer Vision Pinhole camera model non-homogeneous coordinates 6 linear projection in homogeneous coordinates!
Computer Vision 7 Pinhole camera model
Computer Vision Principal point offset principal point 8
Computer Vision Principal point offset calibration matrix 9
Computer Vision 11 Object motion
Computer Vision 12 Camera motion
Computer Vision 13 CCD camera
Computer Vision General projective camera 11 dof (5+3+3) non-singular intrinsic camera parameters extrinsic camera parameters 14
Computer Vision Camera matrix decomposition Finding the camera center (use SVD to find null-space) (for all X and λ C must be camera center) Finding the camera orientation and internal parameters (use RQ decomposition ~QR) (if only QR, invert) =( 15 Q ) = R -1 Q -1
Computer Vision 16 Affine cameras
Computer Vision Radial distortion • Due to spherical lenses (cheap) • Model: R R barrel dist. 17 straight lines are not straight anymore pincushion dist. http: //foto. hut. fi/opetus/260/luennot/11/atkinson_6 -11_radial_distortion_zoom_lenses. jpg
Computer Vision 18 Radial distortion example
Computer Vision Camera model Relation between pixels and rays in space ? 19
Computer Vision Projector model Relation between pixels and rays in space (dual of camera) ? (main geometric difference is vertical principal point offset to reduce keystone effect) 20
Computer Vision 21 Meydenbauer camera vertical lens shift to allow direct ortho-photographs
Computer Vision Action of projective camera on points and lines projection of point forward projection of line back-projection of line 22
Computer Vision Action of projective camera on conics and quadrics back-projection to cone projection of quadric 23
Computer Vision 24 Resectioning
Computer Vision Direct Linear Transform (DLT) rank-2 matrix 25
Computer Vision Direct Linear Transform (DLT) Minimal solution P has 11 dof, 2 independent eq. /points 5½ correspondences needed (say 6) Over-determined solution n 6 points minimize subject to constraint use SVD 26
Computer Vision Singular Value Decomposition Homogeneous least-squares 27
Computer Vision Degenerate configurations (i) Points lie on plane or single line passing through projection center (ii) Camera and points on a twisted cubic 28
Computer Vision Data normalization Scale data to values of order 1 1. 2. 29 move center of mass to origin scale to yield order 1 values
Computer Vision Line correspondences Extend DLT to lines (back-project line) (2 independent eq. ) 30
Computer Vision 31 Geometric error
Computer Vision Gold Standard algorithm Objective Given n≥ 6 2 D to 2 D point correspondences {Xi↔xi’}, determine the Maximum Likelyhood Estimation of P Algorithm (i) Linear solution: (a) Normalization: (b) DLT (ii) Minimization of geometric error: using the linear estimate as a starting point minimize the geometric error: ~ ~~ (iii) Denormalization: 32
Computer Vision Calibration example (i) Canny edge detection (ii) Straight line fitting to the detected edges (iii) Intersecting the lines to obtain the images corners (iv) typically precision <1/10 (v) (H&Z rule of thumb: 5 n constraints for n unknowns) 33
Computer Vision Errors in the image (standard case) Errors in the world Errors in the image and in the world 34
Computer Vision Restricted camera estimation Find best fit that satisfies • skew s is zero • pixels are square • principal point is known • complete camera matrix K is known Minimize geometric error impose constraint through parametrization Minimize algebraic error assume map from param q P=K[R|-RC], i. e. p=g(q) minimize ||Ag(q)|| 35
Computer Vision Restricted camera estimation Initialization • Use general DLT • Clamp values to desired values, e. g. s=0, x= y Note: can sometimes cause big jump in error Alternative initialization • Use general DLT • Impose soft constraints • 36 gradually increase weights
Computer Vision 37
Computer Vision 38 Image of absolute conic
Computer Vision A simple calibration device (i) (ii) (iii) (iv) 39 compute H for each square (corners (0, 0), (1, 0), (0, 1), (1, 1)) compute the imaged circular points H(1, ±i, 0)T fit a conic to 6 circular points compute K from w through cholesky factorization (≈ Zhang’s calibration method)
Computer Vision Some typical calibration algorithms Tsai calibration Zhangs calibration http: //research. microsoft. com/~zhang/calib/ Z. Zhang. A flexible new technique for camera calibration. IEEE Transactions on Pattern Analysis and Machine Intelligence, 22(11): 1330 -1334, 2000. Z. Zhang. Flexible Camera Calibration By Viewing a Plane From Unknown Orientations. International Conference on Computer Vision (ICCV'99), Corfu, Greece, pages 666 -673, September 1999. 40 http: //www. vision. caltech. edu/bouguetj/calib_doc/
Computer Vision Sequential Sf. M • Initialize motion from two images • Initialize structure • For each additional view – Determine pose of camera – Refine and extend structure • Refine structure and motion 41
Computer Vision Initial projective camera motion • Choose P and P´compatible with F (reference plane; arbitrary) Reconstruction up to projective ambiguity Same for more views? different projective basis 42 • Initialize motion • Initialize structure • For each additional view • Determine pose of camera • Refine and extend structure • Refine structure and motion
Computer Vision Initializing projective structure • Reconstruct matches in projective frame by minimizing the reprojection error Non-iterative optimal solution 43 • Initialize motion • Initialize structure • For each additional view • Determine pose of camera • Refine and extend structure • Refine structure and motion
Computer Vision Projective pose estimation • Infere 2 D-3 D matches from 2 D-2 D matches • Compute pose from (RANSAC, 6 pts) X F x Inliers: 44 • Initialize motion • Initialize structure • For each additional view • Determine pose of camera • Refine and extend structure • Refine structure and motion
Computer Vision Refining and extending structure • Refining structure (Iterative linear) • Extending structure 2 -view triangulation • Initialize motion • Initialize structure • For each additional view 45 • Determine pose of camera • Refine and extend structure • Refine structure and motion
Computer Vision Refining structure and motion • use bundle adjustment Also model radial distortion to avoid bias! 46
Computer Vision Metric structure and motion use self-calibration (see next class) Note that a fundamental problem of the uncalibrated approach is that it fails if a purely planar scene is observed (in one or more views) (solution possible based on model selection) 47
Computer Vision 48 Dealing with dominant planes
Computer Vision PPPgric HHgric 49
Computer Vision Farmhouse 3 D models (note: reconstruction much larger than camera field-of-view) 50
Computer Vision 51 Application: video augmentation
Computer Vision 52 Next class: Segmentation Reading: Chapter 14
- Slides: 51