Rendering Synthetic Objects into Real Scenes Bridging Traditional

Rendering Synthetic Objects into Real Scenes: Bridging Traditional and Image-based Graphics with Global Illumination and High Dynamic Range Photography Paul Debevec, University of California at Berkley

Motivation n Visual Effects – Adding synthetic actors/props n Architectural simulation – Adding buildings to landscapes

Related Work n Previous methods used to insert synthetic objects into real scenes – Drawing – Environment mapping – Approximate geometric model and lighting Global illumination algorithms n Recovery of HDR radiance maps from photographs n

Related Work n n High Dynamic Range refers to the range of brightness values present in a scene Eyes, photos, displays, all capture a limited, non-linear range 1000: 1 100: 1 Everyday brightness range Source: Microsoft Developers Network (about 1012: 1)

Related Work § Why is this a problem? § Limited brightness range results in lost scene detail from saturation and/or underexposure § OK for traditional image use Photos from Open. EXR by Lucas. Film, LTD. http: //www. openexr. com/samples. html

Related Work n What about image-based lighting? – Photo pixel values are not representative of relative Radiance in the scene Images by Paul Debevec, 98

Related Work n Solution: Take multiple photos with varying shutter speeds and ‘combine’ somehow to create accurate relative radiance maps Images by Paul Debevec, 98

Related Work n Is HDR really important?

Method n Divide the scene into three parts

Method n Distant Scene – Light-based model – Required to provide correct incident illumination to local/synthetic objects from desired viewpoint – Assumed that no light from the model will affect the distant scene – Any level of geometric detail wanted

Method n Local Scene – Material-based model (has BRDF) – Must approximate the real scene local to the position of the synthetic objects – ‘Local’ means photometrically local – ‘Approximate’ means geometrically AND photometrically representative of the real scene

Method n Synthetic Objects – Complete materials-based models of elements that are to be inserted into the scene – Geometric and lighting information – Any shape and material supported by the global illumination method that you plan to use

Method n Compositing using a light probe – One method to create the distant scene from a real location – Uses a metal sphere and a camera to gather light from a point

Method n Compositing using a light probe – This light is mapped onto the distant scene geometry and used to calculate the lighting – Lighting is calculated with global illumination

Method n Results

Method n Problem with the current definition of the local scene – We said that it must approximate the material of the real scene – It can be really hard to get the BRDF/material of a real object! – Can we modify the local scene so it doesn’t require this?

Method n Local Scene 2 (Differential Rendering) – Approximate the material of the scene to within reason (not exact) – Render with and without your synthetic objects and calculate the difference – Add that difference back into the original image

Method n Difference between local scene render with and without synthetic objects =

Results

Conclusion n This paper presented a framework for rendering synthetic objects into real world environments with real lighting – High Dynamic Range radiance maps – Global Illumination – Distant and local scenes, synthetic objects – Differential rendering

Future Work Method to generate boundary between distant and local scene n More efficient global illumination n

Comments n Pros – Open framework – Good results n Cons – Hard to decide what local scene should be! – Local scene can be very complicated