The MultiFocus Plenoptic Camera T Georgiev Adobe Systems
The Multi-Focus Plenoptic Camera T. Georgiev, Adobe Systems © 2010 Adobe Systems Incorporated. All Rights Reserved. A. Lumsdaine, Indiana University
Capturing Memorable Moments © 2010 Adobe Systems Incorporated. All Rights Reserved. 2
What’s Missing with this Picture? © 2010 Adobe Systems Incorporated. All Rights Reserved. 3
The Lippmann Sensor § © 2010 Adobe Systems Incorporated. All Rights Reserved. Can we request that Photography renders the full variety offered by the direct observation of objects? Is it possible to create a photographic print in such a manner that it represents the exterior world framed, in appearance, between the boundaries of the print, as if those boundaries were that of a window opened on reality. Gabriel Lippmann, 1908.
Pixels and Cores: What do we do with that power? § § Moore’s Law: Megapixels keep growing § 7. 2 MP = 8 by 10 at 300 dpi § Available on cell phones 60 MP sensors available now § Larger available soon (can a use be found? ) § Use pixels to capture richer information about a scene § Computationally process captured data § GPU power also riding Moore’s Law curve © 2010 Adobe Systems Incorporated. All Rights Reserved. 5
How do we take advantage of these trends? § Plenoptic camera – High resolution; computationally intensive rendering § The pixels are samples of the radiance in the 4 D ray space § Optical elements (lenses, space) transform the ray space § We take a picture by rendering (computationally) § We adjust the picture by transforming the ray space (computationally) © 2010 Adobe Systems Incorporated. All Rights Reserved. 6
The Full Variety: Focusing © 2010 Adobe Systems Incorporated. All Rights Reserved.
Focusing © 2010 Adobe Systems Incorporated. All Rights Reserved. 8
Focusing © 2010 Adobe Systems Incorporated. All Rights Reserved. 9
The Full Variety: Different Views © 2010 Adobe Systems Incorporated. All Rights Reserved. 10
Different Views © 2010 Adobe Systems Incorporated. All Rights Reserved. 11
Different Views © 2010 Adobe Systems Incorporated. All Rights Reserved. 12
Radiance (Plenoptic Function, Lightfield) § Instead of integrating rays coming from all directions into a given point (traditional camera), capture the rays individually (capture the radiance) using the Lippmann Sensor § Record all the information about the scene © 2010 Adobe Systems Incorporated. All Rights Reserved. 13
Plenoptic lens array outside the camera © 2010 Adobe Systems Incorporated. All Rights Reserved. 14
Microlenses inside the camera © 2010 Adobe Systems Incorporated. All Rights Reserved. 15
Problem - Solution § Problem: Limited depth of field § Solution: Microlenses of different focal lengths (This is similar to HDR where we use microlenses of different apertures) © 2010 Adobe Systems Incorporated. All Rights Reserved. 16
Multifocus § Lytro and Raytrix already selling plenoptic cameras § Variable focal lengths: Better focusing. Our work on that started 2009. § Raytrix actually use microlenses of different focal lengths in their camera. § Lytro use single focal length (Plenoptic 1. 0) © 2010 Adobe Systems Incorporated. All Rights Reserved. 17
Variable Focal Lengths Microlenses © 2010 Adobe Systems Incorporated. All Rights Reserved. 18
Variable Focal Lengths Microlenses © 2010 Adobe Systems Incorporated. All Rights Reserved. 19
Variable Focal Lengths Microlenses © 2010 Adobe Systems Incorporated. All Rights Reserved. 20
Variable Focal Lengths Microlenses © 2010 Adobe Systems Incorporated. All Rights Reserved. 21
Variable Focal Lengths Microlenses © 2010 Adobe Systems Incorporated. All Rights Reserved. 22
Variable Focal Lengths Microlenses: Compare Focus Quality © 2010 Adobe Systems Incorporated. All Rights Reserved. 23
Phase Space analysis © 2010 Adobe Systems Incorporated. All Rights Reserved. 24
Phase Space analysis © 2010 Adobe Systems Incorporated. All Rights Reserved. 25
Phase Space analysis: Miocrolenses of different focal lengths © 2010 Adobe Systems Incorporated. All Rights Reserved. 26
To Find Out More § This work was first presented at Asilomar 2010/ © 2010 Adobe Systems Incorporated. All Rights Reserved. 27
Live Demonstrations © 2010 Adobe Systems Incorporated. All Rights Reserved. 28
© 2010 Adobe Systems Incorporated. All Rights Reserved.
- Slides: 29