Diffusion Coded Photography for Extended Depth of Field

Diffusion Coded Photography for Extended Depth of Field SIGGRAPH 2010 Oliver Cossairt, Changyin Zhou, Shree Nayar Columbia University Supported by ONR and NSF 1

Conventional Camera (F/1. 8)

Conventional Camera (F/18) 3

Frequency domain Spatial domain Camera Blur Model Focused Image PSF MTF Image Noise (Modulated Transfer Function) Captured Image 4

Deblurring Problems Problem 1: Low SNR Low MTF values Captured image Focused image MTF Problem 2: Object Sensor P Variation with depth Q Lens 5

Extending Depth of Field (EDOF): Previous Work Focal Sweep Cameras [Hausler ’ 72] [Nagahara et al. ’ 08] Wavefront Coding Cameras Sensor Lens Focal Plane Cubic Phase Plate [Dowski and Cathey ’ 95] [Chi and George ’ 01] [Garcia-Guerrero et al. ‘ 07] Sensor Lens Other Related Work [Levin et al. ’ 07] [Veeraraghavan et al. ’ 07] [Levin et al. ’ 09] 6

Focal sweep vs. wavefront coding near focus far Conventional Camera depth Focal Sweep Wavefront Coding Note: only a single PSF will be used to deblur the whole image. 7

Focal sweep vs. wavefront coding Deblurring Error vs. Depth noise Depth Wavefront Coding Focal Sweep 8

Achieve the performance of focal sweep without any moving parts? 9

Optical Diffusers Diffuser sheets SEM image x Scatter function Light ray w x Diffuser Sensor [http: //www. luminitco. com] w

Diffuser Kernels Without a diffuser: u x Light Field u A/2 x A -A/2 Lens Sensor

Diffuser Kernels Without a diffuser: u x Light Field u A/2 x A -A/2 With a diffuser: Lens Sensor u x u A/2 x A Lens Sensor -A/2

Diffuser Kernels Without a diffuser: u x Light Field u A/2 x A -A/2 With a diffuser: Lens Sensor u x u A/2 x A Lens Sensor -A/2

Diffusion Kernels Light field Without diffuser Diffuser kernel u u With diffuser u A/2 x -A/2 x x

Diffusion Kernels Light field Without diffuser Diffuser kernel With diffuser u u u A/2 x x x -A/2 project PSF project x Camera PSF x Scatter function x Diffused PSF

Radially Symmetric Diffuser PSF Without a diffuser: Lens Sensor

Radially Symmetric Diffuser PSF Without a diffuser: Lens Sensor With a diffuser: Lens Sensor

Radially Symmetric Diffuser PSF Without a diffuser: Lens Sensor With a diffuser: Lens Sensor

Radially Symmetric Diffuser PSF Without a diffuser: Lens Sensor With a diffuser: Lens Sensor

Radially Symmetric Diffuser PSF Without a diffuser: Lens Sensor With a diffuser: Lens Sensor

Radially Symmetric Diffuser PSF Without a diffuser: Lens Sensor With a diffuser: Lens Sensor

Radially Symmetric Diffuser PSF Without a diffuser: Lens Sensor With a diffuser: Lens Sensor

Radially Symmetric Diffuser Scatter function Camera PSF: PSF (1 D slice) -50 px 50 px MTF (1 D slice) Normalized frequency depth Normalized frequency 23

Diffusion Coding Performance Deblurring Error vs. Depth Deblurring Error Wavefront Coding Focus Sweep Diffusion Coding (light field) Diffusion Coding (wave optics) noise Depth Similar performance to focal sweep without moving parts 24

Diffuser Implementation Diffuser surface profile Diffuser scatter function Thickness (um) 3 2 RPC Photonics [www. rpcphotonics. com] 1 0 3 6 r (mm) 8 11 Diffuser height map r (mm) Fabricated diffuser 25

Comparison with Prior Work Deblurring Error vs. Depth Diffusion Coding Garcia-Guerrero x Depth Diffusion coding significantly outperforms prior work. 26

Diffusion Coding Experiments Experimental Setup Fabricated Diffuser Cannon 50 mm EF lens Measured PSFs Without diffuser With diffuser depth Cannon 450 D Sensor

Examples

Stuffed Toys Conventional Camera f-number = 1. 8, exposure time = 16 ms

Stuffed Toys Conventional Camera f-number = 18 , exposure time = 16 ms

Stuffed Toys Diffusion Coding Camera: Captured f-number = 1. 8, exposure time = 16 ms

Stuffed Toys Diffusion Coding Camera: Deblurred f-number = 1. 8, exposure time = 16 ms

Statues Captured Deblurred f-number = 1. 8; exposure time = 10 ms 33

People and Flowers Conventional Camera f-number = 1. 8, exposure time = 16 ms

People and Flowers Diffusion Coding Camera: Captured f-number = 1. 8, exposure time = 16 ms

People and Flowers Diffusion Coding Camera: Deblurred f-number = 1. 8, exposure time = 16 ms

Limitations Conventional Camera • Loss of image texture • Loss of contrast • Occlusion errors Diffusion Coding

Conclusions Diffusion Coding Theory Diffusion Coding Implementation Radially Symmetric Diffusers Diffusion Coding Examples

Diffusion Coded Photography for Extended Depth of Field SIGGRAPH 2010 Oliver Cossairt, Changyin Zhou, Shree Nayar Columbia University Supported by ONR and NSF 39