What are Good Apertures for Defocus Deblurring Changyin

What are Good Apertures for Defocus Deblurring? Changyin Zhou Shree K. Nayar Columbia University ICCP 2009, San Francisco 1

Geometry of Defocus Object Lens Aperture Pattern Sensor PSF Focus Plane 2

Formulation of Defocus In the Spatial Domain Focused Image PSF Image Noise Captured Image 3

Formulation of Defocus In the Fourier Domain Focused Image PSF Image Noise Captured Image 4

Various Aperture Coding Techniques Annular Pattern [Welford 1960] [Levin et al. 2007] Wave-front Coding [Dowski 1993] MURA Pattern [Gottesman 1989] [Veeraraghavan et al. 2007] 5

What are Good Apertures for Defocus Deblurring? 6

Outline - Background - Evaluation Criterion for Aperture Patterns - Pattern Optimization - Experiments 7

How to Evaluate Aperture Patterns? Natural image Gaussian white noise Captured Image Focused Image Deblurred Image An Optimal Deblurring Algorithm Basic idea: Evaluate K using expected quality of deblurred image 8

How to Evaluate Aperture Patterns? Natural image Gaussian white noise Captured Image Focused Image Deblurred Image An Optimal Deblurring Algorithm Linear system 9

The Optimal Linear Deblurring Algorithm l For white noise and L 2 distance, Weiner filter is optimal l Optimize C by minimizing the expected recovery error 10

The Optimal Linear Deblurring Algorithm l : Gaussian noise level : Expected power spectrum of natural images 1/f law l A variant of Weiner deconvolution 11

How to Evaluate Aperture Patterns? Gaussian white noise Natural image Captured Image Focused Image Deblurred Image An Optimal Deblurring Algorithm Linear system Evaluation Function: 12

Evaluation Criterion for Aperture Pattern Noise Level Frequency R(K | Prior Power Spectrum Expected deblurring quality at frequency ) : Expected deblurring quality at noise level . 13

Evaluate Patterns Using the Criterion R Curve of Circular Pattern R Noise Level 14

Evaluate Patterns Using the Criterion Curves of Relative R: R(K)/R(K 0) Veeraraghavan Levin Noise Level Annular Image(binary) Image(gray) Random • Coded apertures help more when noise level is low. • Circular aperture is better when noise level is high. 15

Outline - Background - Evaluation Criterion for Aperture Patterns - Pattern Optimization - Experiments 16

Pattern Optimization l Difficult to solve analytically l Patterns evaluated in the Fourier domain, but strictly constrained in the spatial domain l Difficult l For . to do brute force search binary patterns of resolution N x N, the number of possible solutions is huge, when N = 13, if evaluating one pattern takes 1 millisecond, the brute force search requires 1045 yrs. 17

Pattern Optimization Pattern Evolution in Genetic Algorithm: 13 x 13 binary patterns; 8 different noise levels σ =0. 0001 σ =0. 002 σ =0. 005 σ =0. 008 σ =0. 01 σ =0. 02 σ =0. 03 2 nd Run: 3 nd Run: 18

Evaluate the Optimized Patterns 19

Evaluate the Optimized Patterns Relative R curves of the Optimized Patterns Circular Noise Level Image Veeraraghavan σ= 0. 001 20

Outline - Background - Evaluation Criterion for Aperture Patterns - Pattern Optimization - Experiments 21

Implementation Precision Laser Photoplot (1 micron) 22

Implementation Canon EF 50 mm f/1. 8 Lens 23

Implementation Image Pattern Veeraraghavan et al. ’s Pattern Levin et al. ’s Pattern Circular Pattern (wide open) Our Optimized Pattern 24

Comparison Experiments on a CZP Chart Captured Images Focused Image Circular Pattern Levin et al. ’s Pattern Veeraraghavan et al’s Pattern Image Pattern Our Optimized Pattern 25

Comparison Experiments on a CZP Chart Deblurred Images Focused Image Veeraraghavan et al’s Pattern Circular Pattern Levin et al. ’s Pattern Image Pattern Our Optimized Pattern 26

Shrek Captured Image using the Optimized Pattern 27

Shrek Captured Image using the Optimized Pattern Deblurring Result 28

On the Street Captured Image Using the Optimized Pattern 29

On the Street Captured Image Using the Optimized Pattern Deblurring Result 30

Traffic Scene Captured Image Using the Optimized Pattern 31

Traffic Scene Captured Image Using the Optimized Pattern Deblurring Result 32

Summary l l Main Contributions l Aperture Evaluation Criterion for Defocus Deblurring l Aperture Pattern Optimization for Defocus Deblurring Future Work l Optimize gray-level patterns l Apply Criterion to other PSF Engineering Problems l Extend Analysis to Account for Diffraction 33

Thank You! 34