Efficient Irradiance Normal Mapping Ralf Habel Michael Wimmer

Efficient Irradiance Normal Mapping Ralf Habel, Michael Wimmer Institute of Computer Graphics and Algorithms Vienna University of Technology

Motivation Combining Light Mapping and Normal Mapping Also know as: Radiosity Normal Mapping Directional Light Mapping Spherical Harmonics Light Mapping Popular in Games Half-Life 2, Halo 3 … Cheap and good looking: Normal maps can be reused Per vertex/per texel light map pipeline Fast and trivial evaluation Ralf Habel 1

Motivation Light mapped Ralf Habel 2

Motivation Irradiance normal mapped Ralf Habel 3

Motivation Irradiance normal mapped no albedo Ralf Habel 4

Introduction Goal: Represent irradiance on all surfaces for all possible directions (S x Ω) Allows illumination to be stored sparsely similar to light mapping Local variation is transported by normal maps Representation: Environment maps (piecewise linear) Basis function sets (Spherical Harmonics) Evaluation: Look up/calculate irradiance value in normal direction Ralf Habel 5

Irradiance Environment Maps Ramamoorthi et al. 2001: Spherical Harmonics up to the quadratic band (RGB: 27 coefficients) is enough for an accurate representation (avg. error < 3%). 9 RGB textures containing SH coefficients Ø Irradiance over all directions is a low frequency signal Can we do better? Only hemispherical signal (Ω+) needed on opaque surfaces Other basis functions than Spherical Harmonics? Ralf Habel 6

Hemispherical bases Set of functions defined over the hemisphere (Ω+) Desired attributes for irradiance: No discontinuities for smooth interpolation Orthonormality: simplifies projections and other calculations (just like in Euclidian space) Band structure for LOD/increasing accuracy (like Spherical Harmonics) Not important: Locality High-frequency behavior Ralf Habel 7

Half-Life 2 Basis Consists of 3 orthonormal cosine lobes (linear SRBFs) Orthonormal over Ω+ Equivalent to Directional occlusion (one general cosine lobe) Linear Spherical Harmonics band normed on Ω+ All require 3 coefficients and are linear No quadratic terms Ralf Habel 8

Hemispherical bases General orthonormal hemispherical bases: Hemispherical Harmonics [Gautron et al. 04] Makhotkin Basis [Makhotkin 96] All basis functions are 0 or constant on border of Ω+ due to generation through shifting Ø Non-polynomial Ø Zernike Basis [Koenderink 96] Different band structure: 1, 2, 3. . instead of 1, 3, 5. . Ø Non-polynomial Ø Ralf Habel 9

Creating Directional Irradiance We need irradiance on all surface points in all Ω+ directions: Convolution with diffuse kernel far too expensive in Cartesian coordinates Tens of millions of convolutions Instead: Spherical Harmonics as an intermediate basis [Ramamoorthi 01, Basri and Jacobs 00] Ralf Habel 10

Creating Directional Irradiance Create radiance estimate in precomputation From photon mapping, path tracing, shadow mapping… In tangent space (for tangent space normal maps) Expand radiance into Spherical Harmonics by integrating against SH basis functions: Ralf Habel 11

Creating Directional Irradiance Perform diffuse convolution directly in SH to get Using Funk-Hecke Theorem, diffuse convolution is carried out by scaling SH coefficients in each band l with al : a 0 = 1, a 1 = 2/3, a 2 = ¼, a 3 = 0, a 4 = -1/24 There is never a cubic contribution in an SH irradiance signal All l >=4 are very small Ø This is why SH up to the quadratic band is so efficient for irradiance! Ralf Habel 12

H-Basis We would like something similar to SH on Ω+ Polynomial As fast as SH to evaluate Same interpolation behavior Orthonormal on Ω+ Targeted for irradiance representation Take a close look at SH functions and polynomial Hilbert space to derive basis functions Ralf Habel 13

H-Basis SH functions that are symmetric to the z-axis are orthogonal on the hemisphere as well: Y 00, Y 1 -1, Y 11, Y 2 -2, Y 22 Renormed to Ω+ Ralf Habel 14

H-Basis SH functions that are symmetric to the z-axis are orthogonal on the hemisphere as well: Y 00, Y 1 -1, Y 11, Y 2 -2, Y 22 Renormed to Ω+ Ralf Habel 15

H-Basis Apply shifting to Y 10 (cos θ = 2 cos θ -1) Similar to Hemispherical Harmonics/ Makhotkin basis Ralf Habel 16

H-Basis Ø Ø Results in Ω+ orthonormal polynomial basis with 1 constant, 3 linear and 2 quadratic basis functions There is a mathematical rigorous derivation! Ralf Habel 17

H-Basis Band structure allows to use only the constant+linear functions (H 4) or all six (H 6) similar to SH Ralf Habel 18

SH to H-Basis Directional irradiance signals are calculated in SH Project SH coefficient vector into H-Basis with matrix multiplication: Sparse due to closeness to SH Both bases are polynomial No loss due to change in used function space Ralf Habel 19

Bases Comparison Visual/perceptual comparison of all bases Replace H-Basis with any other In “very bad case” lighting situation Ø All basis functions are contributing SH comparison is least-square hemispherically projected [Sloan 03] Makes optimal use of SH on Ω+ Shown with increasing number of coefficients Only few are shown See paper for all of them Ralf Habel 20

Bases Comparison: 3 Coefficients Ground truth Half-Life 2 Zernike 2 bands (not how the game evaluates) Ralf Habel 21

Bases Comparison: 4 Coefficients Ground truth H 4 SH 2 bands (Ω+ projected) Ralf Habel Makhotkin 2 bands (Artefacts at border) 22

Bases Comparison: 6/9 Coefficients Ground truth H 6 (6 coefficients) Ralf Habel SH 3 bands (9 coefficients) 23

Bases Comparison Integrated Mean Square Error averaged over 10 000 random irradiance signals 6 coefficients is enough for a numerically accurate representation What about difference between H 4 and H 6? Ralf Habel 24

H-Basis Comparison H 4 - 4 coefficients Ralf Habel 25

H-Basis Comparison H 6 - 6 coefficients Ralf Habel 26

Conclusion H-Basis is very efficient and very simple solution for hemispherical irradiance signals 4 coeffs. for perceptually accurate representation Probably sufficient for almost all practical cases 6 coeffs. for numerically accurate representation Some lighting situations may benefit from 6 coeffs. Orthonormality : Shader LOD (functions are delocalized) Easy expansion of other low frequency signals Ralf Habel 27

Future Work There is a general mathematical description and derivation similar to Spherical Harmonics H-Basis is a special case Efficient generating procedures Clarify correlations to SH Other hemispherical signals Visibility? BRDFs? Ralf Habel 28

Thanks for your attention Ralf Habel 29