Close Range Photogrammetry Saju John Mathew EE 5358

Close Range Photogrammetry Saju John Mathew EE 5358 Monday, 24 th March 2008 University of Texas at Arlington Monday, March 24, 2008 EE 5358 Computer Vision 1

Overview • • • Definitions Equipment Mathematical Explanations Working Applications Monday, March 24, 2008 EE 5358 Computer Vision 2

Close Range Photogrammetry(CRP) �Photogrammetry is a measurement technique where the coordinates of the points in 3 D of an object are calculated by the measurements made in two photographic images(or more) taken starting from different positions. �CRP is generally used in conjunction with object to camera distances of not more than 300 meters (984 feet). Monday, March 24, 2008 EE 5358 Computer Vision 3

Vertical Aerial Photographs University of Texas at Arlington at approx. 30 meters Monday, March 24, 2008 University of Texas at Arlington at approx. 200 meters EE 5358 Computer Vision 4

CRP It can be broadly divided into two main parts: � Acquiring data from the object to be measured by taking the necessary photographs. � Reducing the photographs (perspective projection) into maps or spatial coordinates (orthogonal projection). Monday, March 24, 2008 EE 5358 Computer Vision 5

Acquisition of Data: Cameras can be broadly classified into two: • Metric • Single Cameras • Stereometric Cameras • Non-metric Monday, March 24, 2008 EE 5358 Computer Vision 6

Metric Cameras Photogrammetric Camera that enables geometrically accurate reconstruction of the optical model of the object scene from its stereo photographs Single Cameras • • • Total depth of field Photographic material Nominal focal length Format of photographic material Tilt range of camera axis and number of intermediate stops Monday, March 24, 2008 EE 5358 Computer Vision 7

Metric Cameras (contd. ) Stereometric Cameras • • • Base Length Nominal Focal Length Operational Range Photographic Material Format of photographic material Tilt range of optical axes and number of intermediate tilt stops Monday, March 24, 2008 EE 5358 Computer Vision 8

Non-metric Cameras that have not been designed especially for photogrammetric purposes: • A camera whose interior orientation is completely or partially unknown and frequently unstable. Monday, March 24, 2008 EE 5358 Computer Vision 9

Non-metric Cameras Advantages • • General availability Flexibility in focusing range Price is considerably less than for metric cameras Can be hand-held and thereby oriented in any direction Disadvantages • Lenses are designed for high resolution at the expense of high distortion • Instability of interior orientation (changes after every exposure) • Lack of fiducial marks • Absence of level bubbles and orientation provisions precludes the determination of exterior orientation before exposure Monday, March 24, 2008 EE 5358 Computer Vision 10

Data Reduction • Analog 1900 to 1960 • Analytical 1960 onwards • Semi-analytical • Digital Monday, March 24, 2008 1980 onwards EE 5358 Computer Vision 11

Analytical Photogrammetry �Based on camera parameters, measured photo coordinates and ground control �Accounts for any tilts that exist in photos �Solves complex systems of redundant equations by implementing least squares method Monday, March 24, 2008 EE 5358 Computer Vision 12

Review Collinearity Condition L Z y f xa a Tilted photo plane The exposure station of a photograph, an object point and its photo image all lie along a straight line. ya x O Y ZL A Za Xa XL Ya YL X Monday, March 24, 2008 EE 5358 Computer Vision 13

Image Coordinate System z’ • Ground Coordinate System - X, Y, Z L Z Object Point (A) Coordinates – Xa, Ya, Za • Rotated coordinate system parallel to ground coordinate system (XYZ) – x’, y’, z’ x’ za’ wrt Ground Coordinate System • Exposure Station Coordinates – XL, YL, ZL • y’ ya’ Xa’ ZL Y A wrt Rotated Coordinate System � Rotated image coordinates – xa’, ya’, za’ � xa’ , ya’ and za’ are related to the measured photo coordinates xa, ya, focal length (f) and the three rotation angles omega, phi and kappa. Za Xa Ya XL YL X Monday, March 24, 2008 EE 5358 Computer Vision 14

Rotation Formulas � Developed in a sequence of three independent two-dimensional rotations. • ω rotation about x’ axis x 1 = x’ y 1 = y’Cosω + z’Sinω z 1 = -y’Sinω + z’Cosω • φ rotation about y’ axis x 2 = -z 1 Sinφ + x 1 Cosφ y 2 = y 1 z 2 = z 1 Cosφ + x 1 Sinφ • κ rotation about z’ axis x = x 2 Cosκ + y 2 Sin κ y = -x 2 Sinκ + y 2 Cosκ z = z 2 Monday, March 24, 2008 EE 5358 Computer Vision 15

Rotation Matrix x = x’(CosφCosκ) + y’(SinωSinφCosκ + CosωSinκ) + z’(-CosωSinφCosκ + SinωSinκ) y = x’(-CosφSinκ) + y’(-SinωSinφSinκ + CosωCosκ) + z’(CosωSinφSinκ + SinωCosκ ) z = x’(Sinφ) + y’(-SinωCosφ) + z’(CosωCosφ) X = MX’ Rotation Matrix x = m 11 x’ + m 12 y’ + m 13 z’ y = m 21 x’ + m 22 y’ + m 23 z’ z = m 31 x’ + m 32 y’ + m 33 z’ The sum of the squares of the three “direction cosines” in any row or in any column is unity. M -1 = MT X’ = MTX Monday, March 24, 2008 EE 5358 Computer Vision 16

Collinearity Condition Equations Collinearity condition equations developed from similar triangles x = m 11 x’ + m 12 y’ + m 13 z’ y = m 21 x’ + m 22 y’ + m 23 z’ z = m 31 x’ + m 32 y’ + m 33 z’ * Dividing xa and ya by za * Substitute –f for za * Correcting the offset of Principal point (xo, yo) Monday, March 24, 2008 EE 5358 Computer Vision 17

Collinearity Equations • Nonlinear • Nine unknowns • ω, φ, κ • XA, YA and ZA • XL, YL and ZL Taylor’s Theorem is used to linearize the nonlinear equations substituting Monday, March 24, 2008 EE 5358 Computer Vision 18

Linearizing Collinearity Equations Rewriting the Collinearity Equations Taylor’s Theorem • F 0 and G 0 are functions F and G evaluated at the initial approximations for the nine unknowns • dω, dφ, dκ are the unknown corrections to be applied to the initial approximations • The rest of the terms are the partial derivatives of F and G wrt to their respective unknowns at the initial approximations Monday, March 24, 2008 EE 5358 Computer Vision 19

Applying LLSM to Collinearity Equations • Residual terms must be included in order to make the equations consistent J = x a – Fo ; K = y a – Go b terms are coefficients equal to the partial derivatives Numerical values for these coefficient terms are obtained by using initial approximations for the unknowns. The terms must be solved iteratively (computed corrections are added to the initial approximations to obtain revised approximations) until the magnitudes of corrections to initial approximations become negligible. Monday, March 24, 2008 EE 5358 Computer Vision 20

Analytical Stereomodel �Mathematical calculation of three-dimensional ground coordinates of points in the stereomodel by analytical photogrammetric techniques �Three steps involved in forming an Analytical Stereomodel: • Interior Orientation • Relative Orientation • Absolute Orientation Monday, March 24, 2008 EE 5358 Computer Vision 21

Analytical Interior Orientation �Requires camera calibration information and quantification of the effects of atmospheric refraction. � 2 D coordinate transformation is used to relate the comparator coordinates to the fiducial coordinate system to correct film distortion. �Lens distortion and principal-point information from camera calibration are used to refine the coordinates so that they are correctly related to the principal point and free from lens distortion. �Atmospheric refraction corrections are applied. Monday, March 24, 2008 EE 5358 Computer Vision 22

Analytical Relative Orientation �Process of determining the elements of exterior orientation �Fix the exterior orientation elements of the left photo of the stereopair to zero values �Common method in use to find these elements is through Space Resection by Collinearity(see slide below) �Each object point in the stereomodel contributes 4 equations � 5 unknown orientation elements + 3 unknowns(X, Y & Z) Monday, March 24, 2008 EE 5358 Computer Vision 23

Space Resection by Collinearity � Formulate the collinearity equations for a number of control points whose X, Y and Z ground coordinates are known and whose images appear in the tilted photo. The equations are then solved for the six unknown elements of exterior orientation which appear in them. Space Resection collinearity equations for a point A A two dimensional conformal coordinate transformation is used X = ax’ – by’ + Tx X, Y – ground control coordinates for the point Y = ay’ + bx’ + Ty x’, y’ – ground coordinates from a vertical photograph a, b, Tx, Ty – transformation parameters Monday, March 24, 2008 EE 5358 Computer Vision 24

Analytical Absolute Orientation �Utilizes a 3 D conformal coordinate transformation �Requires a min. of 2 horizontal and 3 vertical control points �Stereomodel coordinates of control points are related to their 3 D coordinates in a cartesian coordinate system �Coordinates of all stereomodel points in the ground system can be computed by applying the transformation parameters Monday, March 24, 2008 EE 5358 Computer Vision 25

Bundle Adjustment Adjust all photogrammetric measurements to ground control values in a single solution Unknown quantities • X, Y and Z object space coordinates of all object points • Exterior orientation parameters of all photographs Measurements • x and y photo coordinates of images of object points • X, Y and/or Z coordinates of ground control points • Direct observations of the exterior orientation parameters of the photographs Monday, March 24, 2008 EE 5358 Computer Vision 26

Bundle Adjustment-Observations � Photo Coordinates - Fundamental Photogrammetric Measurements made with a comparator or analytical plotter. According to accuracy and precision the coordinates are weighed � Control Points – determined through field survey � Exterior Orientation Parameters – especially helpful in understanding the angular attitude of a photograph Regardless of whether exterior orientation parameters were observed, a least squares solution is possible since the number of observations is always greater than the number of unknowns. x ij, yij – measured photo coordinates of the image of point j on photo i related to fiducial axis system x o, yo – coordinates of principal points in fiducial axis system f - focal length/principal distance m 11 i, m 12 i, ……. , m 33 i – rotation parameters for photo i X j, Yj, Zj – coordinates of point j in object space XLi, YLi, ZLi – coordinates of incident nodal point of camera lens in object space Monday, March 24, 2008 EE 5358 Computer Vision 27

Bundle Adjustment - Weights �Photo coordinates σ02 – reference variance σxij 2 , σyij 2 – variances in xij and yij resp. σxijyij = σyijxij – covariance of xij and yij • Ground Control coordinates • • σXj 2, σYj 2, σZj 2 – variances in Xj 00, Yj 00, Zj 00 resp. σXj. Yj = σYj. Xj – covariance of Xj 00 with Yj 00 σXj. Zj = σZj. Xj – covariance of Xj 00 with Zj 00 σYj. Zj = σZj. Yj – covariance of Yj 00 with Zj 00 • Exterior Orientation Parameters Monday, March 24, 2008 EE 5358 Computer Vision 28

Direct Linear Transformation (DLT) � This method does not require fiducial marks and can be solved without supplying initial approximations for the parameters Collinearity equations along with the correction for lens distortion δx, δy – lens distortion fx – pd in the x direction fy – pd in the y direction Rearranging the above two equations Monday, March 24, 2008 EE 5358 Computer Vision 29

DLT(contd. ) The resulting equations are solved iteratively using LSM � Advantages - No initial approximations are required for the unknowns. � Limitations � - Requirement of atleast six 3 D object space control points - Lower accuracy of the solution as compared with a rigorous bundle adjustment Monday, March 24, 2008 EE 5358 Computer Vision 30

Analytical Self Calibration The equations take into account adjustment of the calibrated focal length, principal-point offsets and symmetric radial and decentering lens distortion. xa, ya – measured photo coordinates related to fiducials xo, yo – coordinates of the principal point = x a – xo where = y a - yo Monday, March 24, 2008 EE 5358 Computer Vision 31

Analytical Self Calibration(contd. ) k 1, k 2, k 3 = symmetric radial lens distortion coefficients p 1, p 2, p 3 = decentering distortion coefficients f = calibrated focal length r, s, q = collinearity equation terms Provides a calibration of the camera under original conditions which existed when the photographs were taken. Geometric Requirements - Numerous redundant photographs from multiple locations are required, with sufficient roll diversity - Many well-distributed image points be measured over the entire format to determine lens distortion parameters The numerical stability of analytical self calibration is of serious concern. Monday, March 24, 2008 EE 5358 Computer Vision 32

Applications �Automobile Construction �Machine Construction, Metalworking, Quality Control �Mining Engineering �Objects in Motion �Shipbuilding �Structures and Buildings �Traffic Engineering �Biostereometrics Monday, March 24, 2008 EE 5358 Computer Vision 33

Biomedical Applications �Linear tape and caliper measurements of inherently irregular three-dimensional biological structures are inadequate for many purposes. �Subtle movements produced by breathing, pulsation of blood, and reflex correction for control of postural stability. �Short patient involvement times, avoids contact with the patient and thereby avoiding risk of deforming the area of interest and spreading infection. �All medical photogrammetric measurements require further interpretation and analysis to allow meaningful information to be given to the end-user. Monday, March 24, 2008 EE 5358 Computer Vision 34

Bibliography • Kamara, H. M. (1979). Handbook of Non-Topographic Photogrammetry, American Society of Photogrammetry. • Wolf , Paul R. , Dewitt, Bon A. (2000). Elements of Photogrammetry, Mc. Graw Hill. • Devarajan, Venkat and Chauhan, Kriti (Spring 2008). Lecture Notes: Mathematical Foundation of Photogrammetry, EE 5358 University of Texas at Arlington. • Karara, H. M. (1989). Non-Topographic Photogrammetry, American Society for Photogrammetry and Remote Sensing. • Mitchell, H. L. and Newton, I. (2002). Medical photogrammetric measurement: overview and prospects. ISPRS Journal of Photogrammetry & Remote Sensing, 56, 286 -294. Monday, March 24, 2008 EE 5358 Computer Vision 35

Acknowledgments �Dr. Venkat Devarajan �Kriti Chauhan Monday, March 24, 2008 EE 5358 Computer Vision 36