Precomputed Radiance Transfer for RealTime Rendering in Dynamic

Precomputed Radiance Transfer for Real-Time Rendering in Dynamic, Low-Frequency Lighting Environments Peter-Pike Sloan, Microsoft Research Jan Kautz, MPI Informatik John Snyder, Microsoft Research

Previous Work – Where We Fit Lighting full env. map [Blinn 76] ? ? our technique [Moeller 02] single area light [Ashikhmin 02] [Crow 77] [Heidrich 00] [Blinn 76] [Moeller 02] Frozen Lighting [Williams 78] [Malzbender 01] [Miller 84] Irradiance Volumes [Greger 96] [Stamminger 02] [Tong 02] [Latta 02] Surface Lightfields [Ramamoorthi 02] [Miller 98, Wood 00] [Chen 02] Non-Interactive [Crow 77] point lights [Matusik 02] [Heidrich 00] simple shadows interreflections Transport Complexity

Motivation • Better light integration and transport § dynamic, area lights § self-shadowing § interreflections • For diffuse and point light area light glossy surfaces • At real-time rates area lighting, no shadows area lighting, shadows

Basic Idea Preprocess for all i

Diffuse Self-Transfer 2 D example, piecewise constant basis, shadows only Preprocess Project Light Rendering light • • = = =

Precomputation. . . Basis 16 Basis 17 Basis 18. . . illuminate result

Previous Work – Scene Relighting • [Dorsey 91] opera lighting design adjusts intensity of fixed light sources • [Nimeroff 94] natural environments uses steerable functions for general skylight illumination • [Teo 97] efficient linear re-rendering generalizes to non-infinite sources, PCA to reduce basis • [Debevec 00] reflectance field of a face uses directional light basis for relighting faces • [Dobashi 95] lighting design uses SH basis for point light intensity distribution

Basis Functions • We use Spherical Harmonics l=0 m=0 • SH have nice properties: § § § simple projection/reconstruction rotationally invariant (no aliasing) simple rotation simple convolution few basis functions low freqs l=1 m=-1 l=1 m=0 l=1 m=1 l=2 m=1 l=3 m=-1 l=3 m=2 l=4 m=-2

Diffuse Transfer Results No Shadows/Inter Shadows+Inter

Glossy Self-Transfer § exiting radiance is view-dependent depends on BRDF (we use Phong)

Glossy Self-Transfer § exiting radiance is view-dependent depends on BRDF (we use Phong) § represent transferred incident radiance, not exiting accounts for shadows, interreflections § allows run-time BRDF changes

Transfer Matrix Precompute how global lighting transferred lighting * p 1 lighting p 1 p 2 * transfer matrices transferred radiance

Glossy Rendering Lookup at R Integrates incident radiance Transfer matrix (maps radiance to against BRDF in direction v. Lighting Env in SH transferred incident Freeze the view Lightradiance) (vector) Phong: Convolve light and evaluate in reflection direction, R

Glossy Transfer Results No Shadows/Inter Shadows+Inter • Glossy object, 50 K mesh • Runs at 3. 6/16/125 fps on 2. 2 Ghz P 4, ATI Radeon 8500

Interreflections and Caustics interreflections none 1 bounce 2 bounces Transport Paths Runtime is independent of transport complexity caustics

Arbitrary BRDFs [Kautz 02] … BRDF Coefficients =

Arbitrary BRDF Results Anisotropic BRDFs Other BRDFs Spatially Varying

Neighborhood Transfer • Allows to cast shadows/caustics onto arbitrary receivers • Store how object scatters/blocks light around itself (transfer matrices on grid) lighting * receiver transfer matrices * receiver transferred radiance

Neighborhood Transfer Results • • • 64 x 8 neighborhood diffuse receiver timings on 2. 2 Ghz P 4, ATI Radeon 8500 • 4 fps if light changes • 120 fps for constant light

Volumes • Diffuse volume: 32 x 32 grid • Runs at 40 fps on 2. 2 Ghz P 4, ATI 8500 • Here: dynamic lighting

Local Lighting using Radiance Sampling single sample (at center = light at ) multi-sample locations multi-sample result • Sample incident radiance at multiple points • Choose sample points over object using ICP from VQ • Correct for shadows but not interreflections

Light Size vs. SH Order 0° 20° 40° n=2 linear n=3 quadratic n=4 cubic n=5 quartic n=6 quintic n=26 windowed RT

Results Live Demo (Radeon 9700)

Conclusions Contributions: • Fast, arbitrary dynamic lighting § on surfaces or in volumes • Includes shadows and interreflections • Works for diffuse and glossy BRDFs Limitations: • Works only for low-frequency lighting • Rigid objects only, no deformation

Future Work • Practical glossy transfer § Eliminate frozen view/light constraints § Compress matrices/vectors • Enhanced preprocessing § Subsurface scattering, dispersion § Simulator optimization § Adaptive sampling of transfer over surface • Deformable objects

Acknowledgements • Thanks to: § Jason Mitchell & Michael Doggett (ATI) § Matthew Papakipos (NVidia) § Paul Debevec for light probes § Stanford Graphics Lab for Buddha model § Michael Cohen, Chas Boyd, Hans-Peter Seidel for early discussions and support Questions?

Performance Model # Verts Max GF 4 4600 FPS 50, 060 215 R 300 FPS 304 Precompute Buddha 49, 990 191 269 2. 5 h Tweety 48, 668 240 326 1. 2 h Tyra 100, 000 118 179 2. 4 h Teapot 152, 413 93 154 4. 4 h 1. 1 h

Matrix Formulation

Results – Preprocessing Model Type Sampling Preproc. FPS head ring buddha tyra diffuse glossy diffuse 50 K vert. 256 x 256 t. 50 K vert. 100 K vert. 1. 1 h 129 8 m 94 2. 5 h 125 2. 5 h. . 125 2. 4 h 83 tyra glossy 100 K vert. 2. 4 h . . 83 teapot glossy 150 K vert. 4. 4 h . . 49 cloud diffuse 32 x 32 15 m 40 glider neighb. 64 x 8 3 h. . 120

Previous Work – Precomputed Transport • [Greger 96] irradiance volumes move diffuse object through precomputed lighting • [Miller 98, Wood 00, Chen 02] surface lightfields frozen lighting environments • [Ashikmin 02] steerable illumination textures steers small light source over diffuse object • [Matusik 02] image-based 3 D photography surface lightfield + reflectance field – not interactive

Dynamic Lighting • Sample incident lighting on-the-fly § precompute textures for SH basis functions § use cube map parameterization § render into 6 cube map faces around p § read images back § projection: dot-product between cube maps • Results § low-resolution cube maps sufficient: 6 x 16 § average error: 0. 2%, worst-case: 0. 5% § takes 1. 16 ms on P 3 -933 Mhz, ATI 8500