Radiometric Self Calibration Tomoo Mitsunaga Shree K Nayar
Radiometric Self Calibration Tomoo Mitsunaga Shree K. Nayar Hashimoto Signal Processing Lab. Sony Corporation Dept. of Computer Science Columbia University CVPR Conference Ft. Collins, Colorado June 1999 June/1999 CVPR 99
Problem Statement How well does the image represent the real world? Image M 2 (Low exposure) Image M 1 (High exposure) Usual imaging systems have : • Limited dynamic range • Non-linear response 2 June/1999 CVPR 99
Scene Radiance and Image Irradiance E L Radiance Irradiance Image irradiance : Ideal camera response : Aperture area Exposure : 3 June/1999 CVPR 99
Scene Radiance and Measured Brightness Video Image Formation Scene radiance L Image Exposure linear CCD Camera Electronics Digitization Measured brightness M Scaled radiance I Photo Image Formation Image Exposure Film Development Scanning 4 June/1999 CVPR 99
Calibration with Reference Objects The scene must be controlled • The reflectance of the objects must be known • The illumination must be controlled 5 June/1999 CVPR 99
Calibration without Reference Objects • Differently exposed images from an arbitrary scene • Recover the response function from the images • Calibrate the images with the response function Input Images Response function High dynamic range radiance image 6 June/1999 CVPR 99
Previous Works • Mann and Picard (95) : – Take two images with known exposure ratio R – Restrictive model for f : – Find parameters a, b, g by regression • Debevec and Malik (97) : – General model for f : only smoothness constraint – Take several (say, 10) high quality images – At precisely measured exposures (shutter speed) 7 June/1999 CVPR 99
Obtaining Exposure Information We have only rough estimates • Mechanical error • Reading error (ex. F-stop number) 8 June/1999 CVPR 99
Radiometric Self-Calibration Works with roughly estimated exposures • Inputs : – Differently exposed images – Rough estimates of exposure values – ex. F-stop reading • Outputs : – Estimated response function – Corrected exposure values 9 June/1999 CVPR 99
A Flexible Parametric Model High order polynomial model : video f (M) posi nega Parameters to be recovered : • Coefficients cn • Order N M f(M) of some popular imaging products 10 June/1999 CVPR 99
Response Function and Exposure Ratio Images: q = 1, 2, …. Q , Pixels: p = 1, 2, …. . P Exposure ratio: Using polynomial model : Thus, we obtain. . . Objective function : 11 June/1999 CVPR 99
An Iterative Scheme for Optimization Rough estimates Rq, q+1(0) Rq, q+1(i) Optimize for f f (i ) Optimize for Rq, q+1 Optimized f and Rq, q+1 12 June/1999 CVPR 99
Evaluation : Noisy Synthetic Images f (M) M Solid : Computed response function Dots : Actual response function 13 June/1999 CVPR 99
Evaluation : Noisy Synthetic Images (cont’d) Percentage Error in Computed Response Function Trial Number Maximum Error : 2. 7 % 14 June/1999 CVPR 99
Computing a High Dynamic Range Image • Calibrating by the response function • Normalizing by corrected exposure values • Averaging with SNR-based weighting 15 June/1999 CVPR 99
Results : Low Library (video) Captured images I Calibration chart M Computed response function June/1999 CVPR 99 16
Results : Low Library (video) Captured images Computed radiance image June/1999 CVPR 99 17
Results : Adobe Room (photograph) Captured images I M Computed response function June/1999 Computed radiance image 18 CVPR 99
Results : Taos Clay Oven (photograph) Captured images I M Computed response function June/1999 Computed radiance image 19 CVPR 99
Conclusions A Practical Radiometric Self-calibration Method • Works with – Arbitrary still scene – Rough estimates of exposure • Recovers – Response function of the imaging system – High dynamic range image of the scene Software and Demo http: //www. cs. columbia. edu/CAVE/ 20 June/1999 CVPR 99
- Slides: 20