Computational Photography Ramesh Raskar Mitsubishi Electric Research Labs

Computational Photography Ramesh Raskar Mitsubishi Electric Research Labs, Cambridge MA http: //www. merl. com/people/raskar/photo

Traditional ‘film-like’ Photography Detector Lens Pixels Image

Computational Photography Novel Cameras Generalized Sensor Processing Generalized Optics

Computational Photography Programmable Lighting Light Sources Novel Cameras Generalized Sensor Processing Modulators Generalized Optics Scene

Computational Photography Programmable Lighting Light Sources Novel Cameras Generalized Sensor Processing a. Multi-flash Illum Generalized Optics c. Image Fusion b. SAMP d. Camera Array e. Gradient Sensing f. Flutter Shutter Scene

Mitsubishi Electric Research Labs Multi. Flash NPR Camera Depth Edge Camera Raskar, Tan, Feris, Yu, Turk

Mitsubishi Electric Research Labs Multi. Flash NPR Camera Raskar, Tan, Feris, Yu, Turk

Mitsubishi Electric Research Labs Multi. Flash NPR Camera Raskar, Tan, Feris, Yu, Turk Depth Discontinuities Internal and external Shape boundaries, Occluding contour, Silhouettes

Mitsubishi Electric Research Labs Multi. Flash NPR Camera Raskar, Tan, Feris, Yu, Turk

Mitsubishi Electric Research Labs Multi. Flash NPR Camera Raskar, Tan, Feris, Yu, Turk Shadows Highlight Shape Edges Clutter Mark moving parts Many Colors Basic colors

Computational Photography Programmable Lighting Light Sources Novel Cameras Generalized Sensor Processing a. Multi-flash Illum Generalized Optics c. Image Fusion b. SAMP d. Camera Array e. Gradient Sensing f. Flutter Shutter Scene

SAMP Camera (Single Axis Multiple Parameters) Parameters vary in focus, exposure, and aperture.

Multiparameter Camera

A Night Time Scene: Objects are Difficult to Understand due to Lack of Context Dark Bldgs Reflections on bldgs Unknown shapes Ramesh Raskar, Comp. Photo Class Northeastern, Fall 2005

Enhanced Context : All features from night scene are preserved, but background in clear ‘Well-lit’ Bldgs Reflections in bldgs windows Tree, Street shapes Ramesh Raskar, Comp. Photo Class Northeastern, Fall 2005

Night Image Background is captured from day-time scene using the same fixed camera Result: Enhanced Image Day Image Ramesh Raskar, Comp. Photo Class Northeastern, Fall 2005

But, Simple Pixel Blending Creates Ugly Artifacts Ramesh Raskar, Comp. Photo Class Northeastern, Fall 2005

Computational Photography Programmable Lighting Light Sources Novel Cameras Generalized Sensor Processing a. Multi-flash Illum Generalized Optics c. Image Fusion b. SAMP d. Camera Array e. Gradient Sensing f. Flutter Shutter Scene

Gradient Camera Sensing Difference between Neighboring Pixels Short Exposure Goal: High Dynamic Range Long Exposure

High Dynamic Range Images Sensing Pixel Difference with Locally Adaptive Gain Scene Intensity camera saturation map Gradient camera saturation map Intensity camera fails to capture range Gradients saturate at very few isolated pixels

Figure. Problem: 2 results. Motion Deblurring Input Image

Blurred Taxi Image Deblurred by solving a linear system. No post-processing

Application: Aerial Imaging Sharpness versus Image Pixel Brightness Time = T Long Exposure: The moving camera creates smear Time = 0 Shutter Open Shutter Closed Time Short Explosure: Avoids blur. But the image is dark Shutter Open Shutter Closed Time Goal: Capture sharp image with sufficient brightness using a camera on a fast moving aircraft Solution: Flutter Shutter Open Shutter Closed Time

Fluttered Shutter Camera Raskar, Agrawal, Tumblin Siggraph 2006 Ferroelectric shutter in front of the lens is turned opaque or transparent in a rapid binary sequence

Coded Exposure Photography: Assisting Motion Deblurring using Fluttered Shutter Raskar, Agrawal, Tumblin (Siggraph 2006) Short Exposure Traditional MURA Coded Shutter Captured Photos Deblurred Results Image is dark and noisy Result has Banding Artifacts and some spatial frequencies are lost Decoded image is as good as image of a static scene

Computational Photography Ramesh Raskar, MERL Novel Cameras Generalized Sensor Programmable Lighting Processing Generalized Optics Light Sources a. Multi-flash Illum c. Image Fusion e. Gradient Sensing f. Flutter Shutter b. SAMP d. Camera Array