Shader Components Modular and High Performance Shader Development

Shader Components: Modular and High Performance Shader Development Yong He (Carnegie Mellon University) Tim Foley (NVIDIA) Teguh Hofstee (Carnegie Mellon University) Haomin Long (Tsinghua University) Kayvon Fatahalian (Carnegie Mellon University)

Atmosphere Scattering Geometry / Shader LOD Sub-surface Scattering Double-sided lighting Pre-baked Lighting Complex Wear Pattern Layered Terrain Texturing Vertex-animated Vegetation Skeletal Animated Character Dynamic Soft Shadow Epic Games, Inc.

Geometry / Animation Static. Mesh Vertex. Anim Skeletal. Anim Plastic Glass Point. Light Skylight Material Metal Light Spot. Light

Geometry / Animation Static. Mesh Vertex. Anim Skeletal. Anim Plastic Glass Point. Light Skylight Material Metal Light Spot. Light

On CPU, performing these two tasks is easy Object-oriented programming is a good model for representing this mental model IMaterial Metal Plastic ILighting Glass Spotlight Skylight

High performance GPU accelerated shading GPU CPU Specialized Shader Code Efficient CPU-GPU communication GPU shader code statically specialized to effects in use Avoid redundant shader parameter update Minimize CPU overhead (API calls) No dynamic control flow

Goals: modularity and performance Modularity High Performance 1. Enable large shader code bases to be authored as composable effects 2. Specialized GPU shader code for the effects in use 3. Update shader parameters efficiently (only as necessary)

Updating Shader Parameters Efficiently

Skylight. Probe strength 2. 0 shadow. Map Displacement displacement. Map normal. Map Metal Material roughness tint [0. 4]

Skylight. Probe strength 2. 0 shadow. Map Displacement displacement. Map normal. Map Metal Material roughness tint [0. 5]

Skylight. Probe strength 2. 0 shadow. Map Metal Material roughness tint [0. 2]

Skylight. Probe strength 2. 0 shadow. Map Brick Material diffuse tiling uv. Offset [0. 4] [0. 0, 0. 0]

Direct. X 12

Block 0 Efficient parameter communication via parameter blocks in D 3 D 12 / Vulkan light. Probe strength 2. 0 shadow. Map Block 1 Bind. Parameter. Block(0, block 0); Bind. Parameter. Block(1, block 1); Bind. Parameter. Block(2, block 2); Draw(); Bind. Parameter. Block(2, block 3); Draw(); displacement. Map normal. Map Block 3 Block 2 roughness tint [0. 4 0. 5 0. 8] tint [0. 3]

Problem: shading language lacks features to match API Skylight Params Shader Code for Skylight sampler 2 D float Shader Code for Metal 2. 0 sampler 2 DShadow Displacement Params sampler 2 D Metal Params sampler 2 D vec 3 Shading Code for Displacement [0. 3] ? sampler. Cube light. Probe; float strength; sampler 2 DShadow shadow. Map; sampler 2 D displacement. Map; sampler 2 D normal. Map; sampler 2 D roughness. Map; vec 3 tint; . . . void main() {. . . }

Workaround 1: single parameter block (low performance) All Params Shader Code for Skylight sampler 2 D float Shader Code for Metal 2. 0 sampler 2 DShadow Displacement Params sampler 2 D Metal Params sampler 2 D vec 3 Shading Code for Displacement [0. 3] sampler. Cube light. Probe; float strength; sampler 2 DShadow shadow. Map; sampler 2 D displacement. Map; sampler 2 D normal. Map; sampler 2 D roughness. Map; vec 3 tint; . . . void main() {. . . }

Workaround 2: explicit annotations (breaks modularity) Skylight Params Shader Code for Skylight sampler 2 D float Shader Code for Metal 2. 0 sampler 2 DShadow Displacement Params sampler 2 D Metal Params sampler 2 D vec 3 Shading Code for Displacement [0. 3] layout(set=0, binding=0) layout(set=0, binding=1) layout(set=1, binding=2) layout(set=1, binding=0) layout(set=1, binding=1) layout(set=2, binding=0) layout(set=2, binding=1) sampler. Cube light. Probe; float strength; sampler 2 DShadow shadow. Map; sampler 2 D displacement. Map; sampler 2 D normal. Map; sampler 2 D roughness. Map; vec 3 tint; . . . void main() {. . . }

Our Contribution: Shader Components A single abstraction that is used to • Define a module of shader code • Group parameters into blocks • Compose effects into the final specialized shaders

Shader Components

A shader component defines both parameters and code component Metal { param sampler 2 D roughness. Map; param vec 3 tint; Engine-side Parameters . . . vec 3 eval. Reflectance() { return GGX(roughness. Map. Sample(uv)) * tint; } } Code

A shader library consists of many components Camera / View Transform component Camera param vec 3 pos; param vec 3 dir; param mat 4 view. Transform; Static Mesh / Skeletal Animation component Static. Mesh : IObject. Transform component Skeletal. Anim : IObject. Transform param mat 4 bone. Transforms[]; @Mesh. Vertex vec 3 vert. Pos; @Mesh. Vertex uvec 4 bone. Ids, bone. Weights; . . . vec 3 transformed. Pos {. . . } Material Patterns component Wood. Material : IMaterial. Pattern component Metal. Material : IMaterial. Pattern param sampler 2 D roughness. Map; param vec 4 tint; public vec 3 color = ggx(. . . ) //. . . Lighting component Dynamic. Lighting : ILighting param vec 3 light. Pos; component Static. Lighting : ILighting param vec 3 light. Intensity; param texture 2 D lightmap; public vec 3 light. Result =. . . ; public vec 3 light. Result = lightmap. Sample(. . . ); Other: tessellation, geometry displacement etc.

Parameters for a component are placed in the same parameter block component Metal { param sampler 2 D roughness. Map; param vec 3 tint; . . . vec 3 eval. Reflectance() { return GGX(roughness. Map. Sample(uv)) * tint; } } 0 1 roughness. Map tint Parameter Block Layout

Type: Metal roughness tint [0. 3] tint [0. 4 0. 5 0. 8] Type: Metal roughness tint [0. 2] tint [1. 2 0. 8 0. 2] Type: Metal roughness tint [0. 4] tint Metal Component Instances [0. 5 0. 4]

Type: Brick diffuse. Map tiling uv. Offset [0. 4] [0. 0, 0. 0] tiling uv. Offset [0. 3] [0. 1, -0. 1] tiling uv. Offset Brick Component Instances [0. 5] [0. 3, 0. 3]

Component Instances (Parameter Blocks in GPU memory) Bind. Component. Inst(0, inst 0); inst 0: Skylight inst 1: Displac… inst 2: Metal inst 3: Metal Bind. Component. Inst(1, inst 1); Bind. Component. Inst(2, inst 2); Draw(); Bind. Component. Inst(2, inst 3); Draw(); Components in use GPU Pipeline State Parameter Block Binding Skylight Displacement Metal Shader Binding. hlsl <Shader 0>

What we did • Added shader component construct to shading language • A shader component instance corresponds to a parameter block • Observation: the frequency of parameter update aligns with modular constructs

Related work • Modular shading constructs only organize code not parameters • • CG Interfaces [Mark 2003] HLSL classes / interfaces [Microsoft 2011] Spark [Foley 2011] Sh [Mc. Cool 2004]

Related work • Modular shading constructs only organize code not parameters • • CG Interfaces [Mark 2003] HLSL classes / interfaces [Microsoft 2011] Spark [Foley 2011] Sh [Mc. Cool 2004] • Bungie TFX [Natalya 2017]

Evaluation

We implemented an engine with a large, modular shader library • Shader library contains 40 components, 2500 lines of code • • • Skeletal Animation Material-defined vertex animation PN-triangle tessellation [Vlachos et al. 2001] Parallax occlusion mapping [Tatarchuk 2006] Transparency and alpha masking Per-material pattern generation (ported 20 unique patterns from UE 4) Double-sided lighting Cascaded shadow maps [Engel 2006] Directional lighting Physically based environment lighting [Karis 2013] Atmosphere scattering [Bruneton and Neyret 2008]

Three Renderer Implementations for Performance Evaluation • COMPONENTS: On Vulkan (using descriptor set as parameter block) • BASELINE_VK: On Vulkan with D 3 D 11 style binding • Uses only one parameter block for all parameters, which means the engine dynamically re-allocates parameter block storage every frame • This is what UE 4/Source 2 are doing, and we’d like to compare performance against. • BASELINE_GL: On Open. GL (without bindless texture)

Test scenes BOXES Draw Calls: 17, 431 Variants: 2 Materials: 1 Shader Changes: 5 BOXES 1 K Draw Calls: 17, 431 Variants: 200 Materials: 1, 000 Shader Changes: 3, 635 FACTORY 1 Draw Calls: 8, 755 Variants: 26 Materials: 84 Shader Changes: 130 FACTORY 2 Draw Calls: 10, 988 Variants: 26 Materials: 84 Shader Changes: 220 ROME Draw Calls: 26, 834 Variants: 24 Materials: 67 Shader Changes: 129

The COMPONENTS renderer uses 2 x less CPU time than BASELINE_VK CPU Performance Comparison (single core) COMPONENT CPU Time BASELINE_VK CPU Time

Speedup of CPU work translates to reduced whole frame time Overall Frame Time COMPONENT CPU Time BASELINE_VK CPU Time COMPONENT whole frame time BASELINE_VK whole frame time

Both of the Vulkan renderers have lower CPU overhead than Open. GL renderer CPU Time

Integration into NVIDIA Falcor rendering framework • Over 30% faster CPU time per frame • Up to 2 x faster shading performance from specialized shader kernels • Cleaner engine code, no ad-hoc code preprocessor

Summary • Identified why performance and modularity are at odds in D 3 D 12/Vulkan • Introduced shader component abstraction, and identified necessary compiler guarantees to ensure efficient parameter binding • A simple extension of modern shading systems

Thank you We had valuable conversations with: Support: Nir Bentey (NVIDIA) NVIDIA Research Natasha Tartachuck (Unity) National Science Foundation Wade Brainard (Activison) Activision Michael Vance (Activision) Peter-Pike Sloan (Activision) Hao Chen (Amazon)