Spring 2007 Task Proposal Vitaliy Orekhov January 22

Spring 2007 Task Proposal Vitaliy Orekhov January, 22 2007 Imaging, Robotics, & Intelligent Systems Laboratory The University of Tennessee Slide

Outline • Overall research goals and topic • Research progress • Research goals for this semester 3/10/2021 Slide 2

Research Goals and Topic • Test previously developed camera calibration method • Address some of the issues of previous method • Develop a new improved camera calibration method NIKON 14 MM Ultra Wide Angle AF Nikkor Lens www. amazon. com 3/10/2021 Slide 3

COMPLETE CAMERA CALIBRATION FOR NORMAL WIDEANGLE, AND TELEPHOTO LENSES WITH AUTOMATIC DISTORTION MODEL COMPLEXITY SELECTION 1. Introduction 2. Survey and Background Theory – – Camera Models and Calibration Wide Angle and Omnidirectinal Camera Calibration Telephoto and Zoom Lens Calibration Complete Camera Calibration 3. Experimental Results and Comparison – – Automatic Model Complexity Selection Validation Telephoto and Zoom Lens Calibration 4. Improved Complete Camera Calibration Method 5. Practical Implementations – – Real Time Video Distortion Correction Calibration with C++ and User Friendly GUI 6. Summary and Conclusion 3/10/2021 Slide 4

Research Progress • Survey of current camera calibration methods – Distortion Calibration – Model Complexity Selection – Telephoto and zoom lens calibration • Verify performance of distortion model complexity selection • Tested current method on telephoto lenses • Implemented methods to improve calibration for lenses with large focal length • Real time distortion correction • Camera calibration in C++ • Camera calibration GUI 3/10/2021 Slide 5

Research Progress Survey of current camera calibration methods – Distortion Calibration • • • – Model Complexity Selection • • – Advantages of wide-angle and omnidirectional camera systems. It is important to address distortion in order to accurately describe how 3 D point is mapped to 2 D image plane. Overview of methods used to address distortion in camera calibration. Separating calibration of distortion and other camera parameters provides some advantages. Few methods address complete camera calibration. Few methods work on wide range of camera systems. Only a handful of publications address distortion model selection. Selection is performed only between a couple of distortion models. Telephoto and Zoom Lens Calibration • • 3/10/2021 Slide 6 Limit of calibration examples with large focal lengths. Different definition of image center is used for calibration initialization of principal point.

Research Progress Verify performance of distortion model complexity selection • Tested performance on synthetic data – Complexity selected between 12 competing models • – Performance successful with both LPDD and RDDD distortion models Tested robustness of the model complexity selection • • Added noise to data with zero mean and standard deviation ranging from zero to 1. 2 pixels Simulation was repeated 250 times at various noise Model selection criterions AI MDL BIC SSD CAIC % Accuracy 94. 8% 98. 0% 99. 6% 99. 2% Are two radial coefficients and two tangential coefficients sufficient to model the lens distortion? 3/10/2021 Slide 7 Should tangential distortion be ignored while increasing the number of coefficient used to model radial distortion?

Other Model Selection Methods • Very few methods mention automatic model selection – [1] and [2] explore distortion model selection from Information Theoretic Criterion • Both papers validate the model selection by picking between three predetermined competing models (ex. without distortion, just with radial distortion, with both radial and tangential distortion) – [2] shows results of model selection with varied amounts of noise added to synthetic data – The same authors from [2] in [3] mention that future work will include model complexity selection – No results are shown on real data [1] G. Wei, K. Arbter, G. Hirzinger, “Active self-calibration of robotic eyes and hand-eye relationships with model identification, ” IEEE Transaction Robotics and Automation, vol. 14(1), pp. 158 -166, Feb. 1998. [2] M. T. El-Melegy and A. A. Farag, “Nonmetric lens distortion calibration: closed-form solutions, robust estimation and model selection, ” in Ninth IEEE Int. Conf. on Computer Vision , 2003. vol. 1, pp. 554 -559, 13 -16 Oct. 2003. [3] M, Ahmed and A. Farag, “Nonmetric Calibration of Camera Lens Distortion: Differential Methods and Robust Estimation, ” in IEEE Transactions on Image Processing , vol. 14, Iss. 8, pp. 1215 -1230, August 2005. 3/10/2021 Slide 8

Research Progress • Tested current calibration method on telephoto lenses. • Implemented methods to improve calibration for lenses with large focal length. Nikon Nikkor AF/MF 70210 mm F/4 -5. 6 www. amazon. com 3/10/2021 Slide 9

Research Progress Real time distortion correction 3/10/2021 Slide 10

Research Progress Real time distortion correction • Code written in C++ • Creates a look up table based on the camera matrix and distortion coefficients. • Code works with most commonly used distortion models: • Perspective • Stereographic • Equidistant • Orthogonal • Logarithmic • Radial polynomial • Look up table is used to correct each image captured from the camera in real-time. 3/10/2021 Slide 11 Image size: 620 x 620 185 deg. fov Image size: 620 x 620 80 deg. fov

Research Progress Calibration in C++ and User Friendly Calibration GUI 3/10/2021 Slide 12

Research Goals Task 1 New Camera Calibration Method Development • Finish performing calibration on telephoto lens with center of distortion or image center used as the initialization value. • • – Find the cause of instability in finding the calibration matrix with telephoto lens Implement a distortion calibration method with distortion model complexity selection – Implement Distortion Curve Approximation in MATLAB – Explore methods to extract functions and distortion coefficients from distortion curve – Research statistical model selection and incorporate it into model complexity selection from distortion curve Combine distortion calibration into a unified full-scale calibration method Task 2 C++ Calibration Code • Finish optimization for complete camera calibration • Work on user friendly GUI Task 3 and Task 4 Thesis and Publication 3/10/2021 Slide 13

Computation Times * < 1 s. * 48 min * Computations performed using a Pentium 4 1. 4 GHz processor on 8 800 x 600 images with 64 calibration points on each image. 3/10/2021 Slide 14

Task 1 New Camera Calibration Method Implementation • Finish performing calibration on telephoto lens with center of distortion or image center used as the initialization value. – Find the cause of instability in finding the calibration matrix with telephoto lens • Implement a distortion calibration method with distortion model complexity selection – Implement Distortion Curve Approximation in MATLAB – Explore methods to extract functions and distortion coefficients from distortion curve – Research statistical model selection and incorporate it into model complexity selection from distortion curve • Combine distortion calibration into a unified full-scale calibration method Task 2 C++ Calibration Code • Finish optimization for complete camera calibration • Work on user friendly GUI Task 3 and Task 4 Thesis and Publication 3/10/2021 Slide 15

Thank you Suggestions/Comments/Questions 3/10/2021 Slide 16