The MultiFocus Plenoptic Camera T Georgiev Adobe Systems

Capturing Memorable Moments © 2010 Adobe Systems Incorporated. All Rights Reserved. 2

What’s Missing with this Picture? © 2010 Adobe Systems Incorporated. All Rights Reserved. 3

The Lippmann Sensor § © 2010 Adobe Systems Incorporated. All Rights Reserved. Can we request that Photography renders the full variety offered by the direct observation of objects? Is it possible to create a photographic print in such a manner that it represents the exterior world framed, in appearance, between the boundaries of the print, as if those boundaries were that of a window opened on reality. Gabriel Lippmann, 1908.

Pixels and Cores: What do we do with that power? § § Moore’s Law: Megapixels keep growing § 7. 2 MP = 8 by 10 at 300 dpi § Available on cell phones 60 MP sensors available now § Larger available soon (can a use be found? ) § Use pixels to capture richer information about a scene § Computationally process captured data § GPU power also riding Moore’s Law curve © 2010 Adobe Systems Incorporated. All Rights Reserved. 5

How do we take advantage of these trends? § Plenoptic camera – High resolution; computationally intensive rendering § The pixels are samples of the radiance in the 4 D ray space § Optical elements (lenses, space) transform the ray space § We take a picture by rendering (computationally) § We adjust the picture by transforming the ray space (computationally) © 2010 Adobe Systems Incorporated. All Rights Reserved. 6

The Full Variety: Focusing © 2010 Adobe Systems Incorporated. All Rights Reserved.

Focusing © 2010 Adobe Systems Incorporated. All Rights Reserved. 8

Focusing © 2010 Adobe Systems Incorporated. All Rights Reserved. 9

The Full Variety: Different Views © 2010 Adobe Systems Incorporated. All Rights Reserved. 10

Different Views © 2010 Adobe Systems Incorporated. All Rights Reserved. 11

Different Views © 2010 Adobe Systems Incorporated. All Rights Reserved. 12

Radiance (Plenoptic Function, Lightfield) § Instead of integrating rays coming from all directions into a given point (traditional camera), capture the rays individually (capture the radiance) using the Lippmann Sensor § Record all the information about the scene © 2010 Adobe Systems Incorporated. All Rights Reserved. 13

Plenoptic lens array outside the camera © 2010 Adobe Systems Incorporated. All Rights Reserved.

Microlenses inside the camera © 2010 Adobe Systems Incorporated. All Rights Reserved. 15

Problem - Solution § Problem: Limited depth of field § Solution: Microlenses of different focal lengths (This is similar to HDR where we use microlenses of different apertures) © 2010 Adobe Systems Incorporated. All Rights Reserved. 16

Multifocus § Lytro and Raytrix already selling plenoptic cameras § Variable focal lengths: Better focusing. Our work on that started 2009. § Raytrix actually use microlenses of different focal lengths in their camera. § Lytro use single focal length (Plenoptic 1. 0) © 2010 Adobe Systems Incorporated. All Rights Reserved. 17