Slang Language Mechanisms for Building Extensible RealTime Shading

Slang: Language Mechanisms for Building Extensible Real-Time Shading Systems github. com/shader-slang/slang Yong He (Carnegie Mellon University) Kayvon Fatahalian (Stanford University) Tim Foley (NVIDIA)

Images Rendered in Unreal Engine 4 2

Geometry / Animation Features Static. Mesh Displacement Skeletal. Anim Material Features Metal Cloth Glass Point. Light Skylight Light Features Spot. Light 3

Geometry / Animation Features Static. Mesh Displacement <geom>. compute. Geometry(); float 4 color = <material> Skeletal. Anim . compute. Material(); <lighting>. compute. Lighting(color); Material Features Metal void my. Shader(. . . ) { } Cloth Glass Point. Light Skylight Light Features Spot. Light 4

What features should be in a modern shading language? 5

Shading System Design Goals • Modular: Implement a library of shading features as modules • Composable: Assemble a full shader from shader “templates” • Extensible: Add new features quickly • Explicit: Define interfaces clearly and explicitly • Performant: high performance on both GPU and CPU 6

GPU Performance: generate highly specialized shader code Geometry / Animation Features void my. Shader(. . . ) { <geom>. compute. Geometry(); float 4 color = Static. Mesh Displacement Skeletal. Anim Material Features Metal <material> . compute. Material(); <lighting>. compute. Lighting(color); } Cloth Glass Point. Light Skylight Light Features Spot. Light 7

Existing Approaches 8

Reflecting the graphics mental model in system decompositions Material Clay Plastic Matte Skin Light Point. Light Spot. Light Area. Light 9

Custom shader code generation tools Bungie TFX [Tatarchuk 2017] Unreal Engine Material Editor 10

Preprocessor Techniques for Shader Code Generation • Static “uber shaders” • #include HLSL code snippets 11

Static uber shader struct V 2 F { #if defined(NORMAL_MAPPING) float 3 tangent, bitangent; #endif. . . }; #if defined(NORMAL_MAPPING) Texture 2 D normal. Map; #endif V 2 F my. Vertex. Shader(. . . ) { #if defined(NORMAL_MAPPING) v 2 f. tangent =. . . ; v 2 f. bitangent =. . . ; #endif. . . } float 3 my. Fragment. Shader() { #if defined(NORMAL_MAPPING) float 3 normal = apply. Normal. Map(normal. Map, v 2 f. tangent, v 2 f. bitangent, v 2 f. uv); #endif. . . } 12

Achieving modularity: implement shading features in separate files Materials Lights Metal. Material. hlsl Point. Light. hlsl Skin. Material. hlsl Sh. Probe. Light. hlsl Cloth. Material. hlsl Area. Light. hlsl Hair. Material. hlsl 13

Shading System Implementation Strategies • C++ style object hierarchies • Dynamic dispatch, low performance • Custom Engine Specific Shader Compilation Tools • Engine specific, ad-hoc replication of standard compiler functionality • Preprocessor Techniques in plain HLSL • Difficult to maintain and extend • Error prone: lack of clearly defined interfaces 14

Contribution: Slang Shading Language • Modern shading system goals can be achieved by extending HLSL with a small set of general programming language constructs • • • Modular Composable Extensible Explicit Performant • Demonstration of Slang’s performance and extensibility benefits 15

Slang: extension of HLSL with modern language mechanisms • New language features include: • • • Interface constrained generics Associated types Parameter. Block<T> Retroactive extensions Global generic parameters Module import Git. Hub Page: https: //github. com/shader-slang/slang 16

Slang supports first class interface ILight { float 3 compute. Lighting(); } ILight Spotlight Skylight struct Spotlight : ILight { float 3 compute. Lighting() {. . . } } struct Skylight : ILight {. . . } 17

Support specialization via generics Shader entry-points are generic functions void forward. Pass. Shader<TGeometry, TMaterial, TLight> ( TGeometry geom, TMaterial material, TLight light) {. . . } Static. Mesh Displacement Skeletal Metal Cloth Glass forward. Pass. Shader<Displacement, Metal, Skylight> Spotlight Skylight 18

Type parameters are constrained by interfaces interface IGeometry {. . . } interface IMaterial {. . . } interface ILight { ILight float 3 compute. Lighting(); } void forward. Pass. Shader<TGeometry : IGeometry, TMaterial : IMaterial, TLighting : ILight > (TGeometry geom, TMaterial mat, TLighting lighting) {. . . lighting. compute. Lighting(); } Similar to Haskell’s type classes, Rust’s type traits and C# interfaces. Spotlight Skylight 19

Material shading returns a BRDF closure BRDF 1. Material Shading f = eval. Material(P) 2. Lighting Integration Lo = integrate(Li, f, Wi, Wo); 20

Material shading returns a BRDF closure Interface IBRDF{. . . } interface IMaterial{. . . } float 3 forward. Shader<M: IMaterial> (M material) { IBRDF f = material. eval. Material(p); float 3 l = 0; for (int i = 0; i<N; i++) l += f. eval(lights[i], p); return l; IMaterial Metal } 21

Material shading returns a BRDF closure Interface IBRDF{. . . } interface IMaterial{. . . } float 3 forward. Shader<M: IMaterial> (M material) { IBRDF f = material. eval. Material(p); float 3 l = 0; for (int i = 0; i<N; i++) l += f. eval(lights[i], p); return l; IMaterial Metal } 22

Material shading returns a BRDF closure Interface IBRDF{. . . } interface IMaterial{. . . } float 3 forward. Shader<M: IMaterial> (M material) { IBRDF f = material. eval. Material(p); float 3 l = 0; for (int i = 0; i<N; i++) l += f. eval(lights[i], p); return l; IMaterial Metal } 23

Associated types as interface requirement interface IMaterial { associatedtype B : IBRDF; B eval. Material(Position p); } struct Metal. Material : IMaterial { typedef Disney. BRDF B; B eval. Material(Position p) {. . . } } As in Haskell (as type families), Rust, Swift struct Skin. Material : IMaterial { typedef Skin. BRDF B; B eval. Material(Position p) {. . . } } 24

Summary Achieved both modularity and static specialization through two extensions to HLSL: • Interface Constrained Generics • Associated Types • Parameter. Block<T> type 25

Rearchitecting a shading system using Slang 26

27

Falcor’s shading system • Contains over 5000 lines of shader code, supporting: • • • A flexible, layered material system Point, spot, directional, and ambient light type Glossy reflection using an environment map Cascaded, exponential and variance shadow map algorithms Post processing effects, such as screen space ambient occlusion and tone mapping 28

Benefits of adopting Slang • Cleaner Code Structure • Easier to extend with new features • Achieves higher CPU and GPU performance 29

Cleaner Code Structure 30

Falcor implements a layered material library #if USE_DIFFUSE_LAYER compute. Diffuse. Layer(mat, . . . ); #endif #if USE_SPECULAR_LAYER compute. Specular. Layer(mat, . . . ); #endif #if USE_EMISSIVE_LAYER compute. Emissive. Layer(mat, . . . ); #endif Diffuse Specular Emissive 31

Make entry point specializable on materials 1. Define an IMaterial interface IMaterial {. . . } 2. Change shader entry point to use a generic material parameter void main<TMaterial : IMaterial>(Parameter. Block<TMaterial> material) {. . . } 32

Generics also useful for layered material specialization #if USE_DIFFUSE_LAYER compute. Diffuse. Layer(mat, . . . ); #endif #if USE_SPECULAR_LAYER compute. Specular. Layer(mat, . . . ); #endif #if USE_EMISSIVE_LAYER compute. Emissive. Layer(mat, . . . ); #endif struct Layered. Material < let has. Diffuse : bool, let has. Specular : bool, let has. Emissive: bool > : IMaterial { BRDF eval. Material(…) { if (has. Diffuse) compute. Diffuse. Layer(mat, . . . ); if (has. Specular) compute. Specular. Layer(mat, . . . ); if (has. Emissive) compute. Emissive. Layer(mat, . . . ); } } 33

Refactoring lighting to enable specialization interface ILight { float 3 compute. Lighting(TSurface surf); } struct Point. Light : ILight {. . . } struct Directional. Light : ILight {. . . } 34

Easier to Extend 35

New light type: polygonal area lights [Heitz et al. 2016] 36

Fewer changes to code in Slang code base Branch Original Refactored Sites Changed Files Changed Lines of Code 7 4 246 1 1 249 Changes are localized in one place in refactored branch, but distributed at 7 different places in original branch Localization of a feature’s implementation is a sign of good extensibility 37

Fewer changes to code in Slang code base Branch Original Refactored Sites Changed Files Changed Lines of Code 7 4 246 1 1 249 Lines of code is similar in both branches, Refactored architecture is not introducing significant boiler-plate 38

Higher Rendering Performance 39

Performance Evaluation: Environment Setup Goal: measure the CPU and GPU time required to draw each frame using both original and refactored Falcor branch Temple Bistro Interior Images rendered at 1920 * 1080 resolution, on Intel i 7 -5820 CPU and NVIDIA Titan V GPU Bistro Exterior 40

Higher GPU performance due to light specialization Original Branch Refactored Branch 41

Refactored branch achieved 30% speed-up in CPU time Original Branch Refactored Branch 42

Falcor main branch is using Slang exclusively • Over 17, 000 lines of shader code currently compiled by Slang • 20+ Researchers and developers from various groups within NVIDIA using Falcor to prototype rendering techniques are interacting with Slang code 43

Next Steps in Shading Languages 44

C++ for shaders ? ? • Should HLSL adopt C++ features or consider other modern language constructs? 45

Next Steps in Real-Time Shading Languages • Shading language support for real-time ray tracing 46

Next Steps in Shading Languages • Shading language support for ray tracing • More support for dynamic dispatch behavior • Unified CPU-GPU programming • Slang is currently a language for GPU code only, and interacts with CPU code via compiler runtime API • Next step: Unified programing language for CPU-GPU graphics programming 47

Thank you! github. com/shader-slang/slang Thank you to Nir Benty for valuable conversations. Support: NVIDIA Research National Science Foundation 48

Adding light specialization to original branch brings GPU performance on par with refactored branch Original Branch (+Light Specialization) Refactored Branch 49

Refactored branch achieved 30% speed-up in CPU time Original Branch (+Light Specialization) Refactored Branch 50

Adding light specialization makes original branch more difficult to extend Branch Original+LS Refactored Sites Changed 7 8 1 Files Changed 4 5 1 Lines of Code 246 253 249 51