RealTime Rendering COMS 6998 3 Lecture 9 RealTime

Real-Time Rendering COMS 6998 -3, Lecture 9

Real-Time Rendering Demo Motivation: Interactive rendering with complex natural illumination and realistic, measured BRDFs

Complex Light Transport

Challenges • Illumination complexity • Material (BRDF)/view complexity • Transport complexity (shadows, interreflection)

Reflection Maps Blinn and Newell, 1976

Environment Maps Miller and Hoffman, 1984

Environment Maps Interface, Chou and Williams (ca. 1985)

Environment Maps Cylindrical Panoramas Cubical Environment Map 180 degree fisheye Photo by R. Packo

Reflectance Maps • Reflectance Maps (Index by N) • Horn, 1977 • Irradiance (N) and Phong (R) Reflection Maps • Hoffman and Miller, 1984 Mirror Sphere Chrome Sphere Matte Sphere

Irradiance Environment Maps R Incident Radiance (Illumination Environment Map) N Irradiance Environment Map

Assumptions • Diffuse surfaces • Distant illumination • No shadowing, interreflection Hence, Irradiance is a function of surface normal

Diffuse Reflection Reflectance (albedo/texture) Radiosity (image intensity) = Irradiance (incoming light) × quake light map

Analytic Irradiance Formula Lambertian surface acts like low-pass filter 0 0 1 2

9 Parameter Approximation Order 0 1 term Exact image RMS error = 25 % 0 1 2 -2 -1 0 1 2

9 Parameter Approximation Order 1 4 terms Exact image RMS Error = 8% 0 1 2 -2 -1 0 1 2

9 Parameter Approximation Order 2 9 terms Exact image RMS Error = 1% For any illumination, average error < 3% [Basri Jacobs 01] 0 1 2 -2 -1 0 1 2

Comparison Incident illumination 300 x 300 Irradiance map Texture: 256 x 256 Hemispherical Integration 2 Hrs Irradiance map Texture: 256 x 256 Spherical Harmonic Coefficients 1 sec

Rendering Irradiance approximated by quadratic polynomial 4 x 4 matrix (depends linearly on coefficients Llm) Surface Normal vector column 4 -vector

Hardware Implementation Simple procedural rendering method (no textures) • Requires only matrix-vector multiply and dot-product • In software or NVIDIA vertex programming hardware

Reflectance Space Shading Cabral, Olano, Nemec 1999

OLF Parameterization N V N L

OLF Parameterization N V N Reparameterize V by reflection vector R

OLF Parameterization N V N Reparameterize V by reflection vector • Captures structure of BRDF (and hence OLF) better • Reflective BRDFs become low-dimensional R

OLF Structure: Lafortune Viewy • Single 2 D reflection map no longer sufficient • But variation with viewing direction is slow Viewx

Viewy Reflectiony OLF Structure: Lafortune Viewx Reflectionx View maps vary slowly

Viewy Reflectiony A Simple Factorization Viewx Reflectionx *

Precomputed Radiance Transfer for Real-Time Rendering in Dynamic, Low-Frequency Lighting Environments Peter-Pike Sloan, Microsoft Research Jan Kautz, MPI Informatik John Snyder, Microsoft Research

Previous Work – Where We Fit Lighting full env. map [Blinn 76] ? ? our technique [Moeller 02] single area light [Ashikhmin 02] Non-Interactive [Crow 77] point lights [Matusik 02] [Heidrich 00] simple shadows interreflections Transport Complexity

Motivation • Better light integration and transport • dynamic, area lights • self-shadowing • interreflections • For diffuse and glossy surfaces point light area light • At real-time rates area lighting, no shadows area lighting, shadows

Diffuse Self-Transfer 2 D example, piecewise constant basis, shadows only Preprocess Project Light Rendering light • • = = =

Precomputation. . . Basis 16 Basis 17 Basis 18. . . illuminate result

Diffuse Transfer Results No Shadows/Inter Shadows+Inter

Arbitrary BRDF Results Anisotropic BRDFs Other BRDFs Spatially Varying

Shadow Mapping with Today’s Open. GL Hardware CEDEC 2001 Tokyo, Japan Mark J. Kilgard Graphics Software Engineer NVIDIA Corporation

Motivation • Shadows increase scene realism • Real world has shadows • More control of the game’s feel • dramatic effects • spooky effects • Other art forms recognize the value of shadows • But yet most games lack realistic shadows

Common Real-time Shadow Techniques Projected planar shadows Shadow volumes Hybrid approaches Light maps

Problems • Mostly tricks with lots of limitations • Projected planar shadows • well works only on flat surfaces • Stenciled shadow volumes • determining the shadow volume is hard work • Light maps • totally unsuited for dynamic shadows • In general, hard to get everything shadowing everything

Shadow Mapping • Image-space shadow determination • Lance Williams published the basic idea in 1978 • By coincidence, same year Jim Blinn invented bump mapping (a great vintage year for graphics) • Completely image-space algorithm • means no knowledge of scene’s geometry is required • must deal with aliasing artifacts • Well known software rendering technique • Pixar’s Render. Man uses the algorithm • Basic shadowing technique for Toy Story, etc.

References • Important SIGGRAPH papers • Lance Williams, “Casting Curved Shadows on Curved Surfaces, ” SIGGRAPH 78 • William Reeves, David Salesin, and Robert Cook (Pixar), “Rendering antialiased shadows with depth maps, ” SIGGRAPH 87 • Mark Segal, et. al. (SGI), “Fast Shadows and Lighting Effects Using Texture Mapping, ” SIGGRAPH 92

Shadow Mapping Concept (1) • Depth testing from the light’s point-of-view • Two pass algorithm • First, render depth buffer from the light’s point-of-view • the result is a “depth map” or “shadow map” • essentially a 2 D function indicating the depth of the closest pixels to the light • This depth map is used in the second pass

Shadow Mapping Concept (2) • Shadow determination with the depth map • Second, render scene from the eye’s point-of-view • For each rasterized fragment • determine fragment’s XYZ position relative to the light • this light position should be setup to match the frustum used to create the depth map • compare the depth value at light position XY in the depth map to fragment’s light position Z

Shadow Mapping Concept (3) • The Shadow Map Comparison • Two values • A = Z value from depth map at fragment’s light XY position • B = Z value of fragment’s XYZ light position • If B is greater than A, then there must be something closer to the light than the fragment • then the fragment is shadowed • If A and B are approximately equal, the fragment is lit

Visualizing Shadow Mapping • A fairly complex scene with shadows the point light source

Visualizing Shadow Mapping • Compare with and without shadows

Visualizing Shadow Mapping • The scene from the light’s point-of-view FYI: from the eye’s point-of-view again

Visualizing Shadow Mapping • The depth buffer from the light’s point-of-view FYI: from the light’s point-of-view again

Visualizing Shadow Mapping • Projecting the depth map onto the eye’s view FYI: depth map for light’s point-of-view again

Visualizing Shadow Mapping • Comparing light distance to light depth map Green is where the light planar distance and the light depth map are approximatel y equal Non-green is where shadows should be

Visualizing Shadow Mapping • Scene with shadows Notice how specular highlights never appear in shadows Notice how curved surfaces cast shadows on each other

Hardware Shadow Map Filtering • “Percentage Closer” filtering • Normal texture filtering just averages color components • Averaging depth values does NOT work • Solution [Reeves, SIGGARPH 87] • Hardware performs comparison for each sample • Then, averages results of comparisons • Provides anti-aliasing at shadow map edges • Not soft shadows in the umbra/penumbra sense

Hardware Shadow Map Filtering GL_NEAREST: blocky GL_LINEAR: antialiased edges Low shadow map resolution used to heightens filtering artifacts

Soft Shadows?

Conclusion • Variety of techniques • Shadows • Complex illumination • Complex materials • Limited manipulability in most cases • Cannot change lighting, geometry, viewpoint etc. • Is it possible to have everything simultaneously?