Deferred Lighting and Shading 2004 Blue Shift Inc
Deferred Lighting and Shading © 2004 Blue Shift, Inc. With Implementation Details and Demos for the PC (DX 9), Xbox, and PS 2 Platforms
Presenters • Rich Geldreich, Ensemble Studios, rgeldreich@ensemblestudios. com • Matt Pritchard, Ensemble Studios, mpritchard@ensemblestudios. com • John Brooks, Blue Shift, Inc. , jbrooks@blueshiftinc. com
Session Overview • High-level description • PS 2, Xbox demos • Xbox demo will be shown early • • Pros and cons Theory and concepts Platform specific details Advanced stuff, future research
What is it? • • • Deferred lighting and shading is the decoupling of geometry rasterization from lighting and (maybe) shading. – Scene passes only write attributes, not lit pixels • Light independent attributes are dumped to buffers in one or more deferment passes – Some common attributes for deferred lighting: » normal, position/depth – Common attributes for deferred shading: » albedo, specular color/power • Basically, raw data needed for lighting (and maybe shading) is saved and reused in later passes Lights become 2 D image space ops From now on, when we say “Deferred Shading” we usually mean “Deferred Lighting/Shading”.
Deferred Shading Demos – Xbox: Gladiator 2 K 4 • 2 K 3 version was shown at last year’s GDC to a limited audience. This year’s version is a bit more optimized, packs albedo colors differently (YCb. Cr instead of s. RGB), and implements a “metalness” hack for better specular highlights. Created by Blue Shift, Inc. : – PS 2: Bunny Demo • Object space normal maps, gloss maps • Directional lights with specular highlights • Two hemisphere area lighting • HDR accumulation, tone mapping
Deferred Lighting, Deferred Shading, or Both? • Lighting – Light accumulation – Process can take into account a pixel’s location, normal (or full tangent frame for anisotropic shading), and some aspect(s) of the illumination model’s BRDF function (i. e. specular power for Phong) • Basically, “lighting” = Irradiance computation for direct light sources, diffuse and specular contributions separately accumulated • Shading – The process of computing an output pixel’s final color (or exit radiance for HDR) using all of the pixel’s attributes (such as albedo, specular color, etc. ), accumulated light, and a shading function. – Shading can be done at many places: • During lighting passes (only final color is accumulated) • All at once as a final image space post process • During a final scene render pass
Why Bother with Deferred? • • • Some shadow techniques (stencil shadow volumes, “forward” shadow buffering) require a Z pre-pass anyway. – Why not write some per-pixel attributes too, instead of just depth? Predictability: The smaller or further away the light, the less processor time it takes to compute its influence, independent of the quantity of objects the light illuminates. Consistency: Entire scene can be lit using the same illumination model, shadowing technique, etc. Quality: Most of the lighting equation recomputed per-pixel – Good example is light attenuation (falloff): No per-vertex approximations that break down on large triangles Simplicity: Scene rasterization passes require attribute dumping only – No need to handle a zillion combinations of dir/omni/spot/etc. lights in your vertex/pixel programs
Conceptual Operation 1) 2) I. II. 3) 4) • • • Render scene to attribute buffer(s) Example attributes: Normal, albedo, specular color, depth For each light: • • Shadow pass (stencil ops, shadow buffering, etc. ) Shadow buffering can benefit from having the scene’s depth buffer available as a texture Render light’s bounding volume, accumulate diffuse/specular light contributions, or lit/shaded pixels May use a screenspace quad (directional light/full screen light), CPU projected/clipped screenspace n-gon, a 3 D object with front face culling and projective texturing, tilization, etc. (Optional) Shade scene Render a full-screen quad, apply same shader to all pixels Or… Render scene again to support arbitrary material shaders Perform image space post processing effects Particles, blooming, streaks, fog, tone mapping, etc.
Other Advantages • Eliminates hard limits on the number of lights influencing individual objects – Number of active lights per object in forward shading engines is usually constrained in some way by the max. size of a single vertex/pixel program • Potentially reduced overdraw cost – Depends on the lighting complexity – Not as important on GPU’s with fast Z occlusion hardware • Lighting passes can take advantage of the fast Z occlusion hardware on modern GPU’s – Occluded lights, or lights totally in front of stuff can be almost free • On some platforms (PS 2) it may be the only way of achieving bump mapped, per-pixel lighting with many lights – PS 2 is quite capable of rasterizing per-pixel attributes to buffers – Lighting and shading performed at full floating point precision!
Cons of Deferred Shading 1/2 • Can be bandwidth/memory intensive – On some platforms, attributes must be written once, and read back many times. – Multiple attribute buffers at high framebuffer resolutions gobble memory • Very high hardware requirements – Especially at typical PC game resolutions, or with FSAA
Cons of Deferred Shading 2/2 • Per-material shaders or multiple BRDF’s are impractical on some platforms – On some platforms/shader models it’s infeasible to use multiple pixel shader programs to shade the scene – Fully texture driven shaders take on more importance • Alpha blending – “Real” alpha blending (over operator [6]) is difficult • To do it “correctly”, multiple pixels must be independently lit and shaded before applying the over operator • Excuse: Alpha blending is a total hack anyway • We’ll show some hacks that can be decent alternatives on some platforms
What Attributes to Render? • Minimal attributes: – Albedo (typically s. RGB or YCb. Cr) – Gloss (specular intensity, scalar) – Normal (typically unit vector) • Can also write Yaw/Pitch, (ZSign, Y, X), etc. – Depth (scaler, float, int, or spread over several components) • Could also write full 3 D position, but this is unnecessary as the full position can easily be “recovered” given the pixel’s screenspace location and Z depth. • Additional attributes: – – Specular color, power, BRDF or NDF (Normal Distribution Function) index, etc. For anisotropic shading: Perturbed Tangent (XYZ) and/or Binormal (XYZ) Special effects: Object ID (int) Additional shader function parameters (example: flags)
Deferred on DX 9 Hardware • Attribute passes can use Multiple Render Targets (MRT) • Unfortunately, must write depth or position, so up to one whole RT is lost. Can’t alias Z-buffer to a texture like on Xbox. • Can write higher precision normals – 10 -10 -10 looks substantially better than 8 -8 -8
Deferred on DX 9 Hardware HDR Light Accumulation • Alpha blending to HDR surfaces isn’t supported by current DX 9 class hardware • Lighting passes are just 2 D screenspace ops, so it’s easy to predict which portions of the framebuffer are affected by each light • We’ve accumulated HDR light in-place (i. e. coherent read-modify-write to a single buffer set as a render target and texture source) on the Radeon 9800 series by rendering 32 x 32 tiles and switching to dummy render targets after each light – Render targets must be temporarily switched to unrelated surfaces to flush the R 3 xx’s backside caches (? ).
Deferred on Xbox Round One: Two Scene Passes • Two scene rendering passes: – Render Albedo and Gloss (“C Buffer”) – Render Normal and Object ID (“N Buffer”) – Alias Depth/Stencil Buffer to linear 32 -bit A 8 R 8 G 8 B 8 texture • Tricky part: Omni lights – Use texm 3 x 3 pad / texm 3 x 3 tex to “unproject” [9] and transform pixels to Normalized Light Space (NLS) [10] • • • Volume texture lookup fetches NLS light vector and attenuation Rotate light vector to view space, renormalize in the combiners Dump light vector and attenuation to temp. buffer Perform usual Phong lighting calcs. in another pass in the combiners Accumulate lit and shaded pixels
Deferred on Xbox Round Two: One Scene Pass • We don’t have Multiple Render Targets on Xbox, but we can fake it by tightly packing attributes: – High word = Packed albedo and gloss (4 -4 -6 -2) – Low word = Packed normal (1 -7 -8) • Other stuff doesn’t differ from the two pass technique in any major ways, except for a more refined and optimized implementation. • See Rich’s Gladiator presentation for more details. • Xbox could be pushed further – Should be possible to do a scene ~2 x more visually complex at 30 Hz.
Deferred Shading on PS 2 Benefits of Deferred on the PS 2 • • Allows per-pixel shading on PS 2! Extensive per-pixel programmability • • • Unlimited shader length Flexible shader memory use (shader program, constants, etc) Enables high-end effects like normal mapping & per-pixel lighting – – – – floating point (including divide, sqrt, random, etc) integer data swizzling lookup tables data load/store (pixel local or between pixels) looping branching
Deferred Shading on PS 2 Drawbacks to Deferred on the PS 2 • GPU fill-rate • CPU performance • VRAM memory • DRAM memory – DMA attribute buffers from VRAM – DMA shaded pixels to VRAM – Multiple render passes needed to store per-pixel attributes – Based on shader length & number of pixels shaded – Focus on efficient algorithms and tight asm code – Attribute buffers – Attribute textures – Per-pixel attributes – Shaded pixel data
Alpha Blending - It’s a Pain • • • Stippling/screen door transparency [4] [9] – An old school hack useful on console games. Console video encoders and most displays form a big low pass filter, so the pattern is invisible except near edges – Can alternate the stipple pattern every frame – Stippled surfaces interact with stencil shadows in a natural way • Can also use stippling while rendering shadow volumes, for less than fully dark shadows [11] – No explicit sorting required Depth peeling [5] – For surfaces that need blending: Peel back 1 -3 layers – Independently light each layer, composite everything together Hybrid techniques – Forward (immediate) shade surfaces that need blending – Alpha blend overtop of the deferred engine’s output
Deferred on DX 9 Hardware Other Neat Stuff We’ve Tried • • • 2 D Normal Distribution Functions (NDFs) [8] • Anisotropic shading, direction of anisotropy maps • Half-angle rotated to Perturbed Tangent Space, XY used as texcoords for specular & iridescence map lookup • Requires storage of per-pixel perturbed tangent vectors Arbitrary material shaders • Deferred lighting, but forward (immediate) shading • Rerender scene after lighting, use PS 3. 0 “vpos” register or texture projection to read accumulated light, shade fragment using any material pixel shader SH Encoded Irradiance Volumes [2] • Use renderer to create HDR “Radiance Probes” for any position in a scene – output is HDR environment map (cubemap) • Radiance probes can be quickly converted to irradiance environment maps [7] using Spherical Harmonics, completely on the GPU • SH coefficients can be packed into multiple volume textures and reused in later passes
References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. Nicolas Thibieroz, “Deferred Shading with Multiple Render Targets”. Shader. X 2 - Shader Tips & Tricks Peter-Pike Sloan, “Efficient Evaluation of Irradiance Environment Maps”. Shader. X 2 - Shader Tips & Tricks Dean Calver, “Photo-realistic Deferred Lighting”. http: //www. beyond 3 d. com/articles/deflight Tom Mc. Reynolds, “Advanced Graphics Programming Techniques Using Open. GL”. http: //www. sgi. com/software/opengl/advanced 98/notes/node 145. html Cass Everrit, NVIDIA Order Independent Transparency demo Jim Blinn, “Jim Blinn's Corner - Dirty Pixels” Ravi Ramamoorthi, Pat Hanrahan, “An Efficient Representation for Irradiance Environment Maps” Jan Kautz, “Rendering with Hand Crafted Shading Models”. Game Programming Gems 3 Atman Binstock, private conversation, 2001 Alex Vlachos, John Isidoro, Chris Oat, “Textures as Lookup Tables for Per-Pixel Lighting”. Game Programming Gems 3 Atman Binstock, private conversation, 2003
- Slides: 21