Slang Language Mechanisms for Building Extensible RealTime Shading
Slang: Language Mechanisms for Building Extensible Real-Time Shading Systems github. com/shader-slang/slang Yong He (Carnegie Mellon University) Kayvon Fatahalian (Stanford University) Tim Foley (NVIDIA)
Images Rendered in Unreal Engine 4 2
Geometry / Animation Features Static. Mesh Displacement Skeletal. Anim Material Features Metal Cloth Glass Point. Light Skylight Light Features Spot. Light 3
Geometry / Animation Features Static. Mesh Displacement <geom>. compute. Geometry(); float 4 color = <material> Skeletal. Anim . compute. Material(); <lighting>. compute. Lighting(color); Material Features Metal void my. Shader(. . . ) { } Cloth Glass Point. Light Skylight Light Features Spot. Light 4
What features should be in a modern shading language? 5
Shading System Design Goals • Modular: Implement a library of shading features as modules • Composable: Assemble a full shader from shader “templates” • Extensible: Add new features quickly • Explicit: Define interfaces clearly and explicitly • Performant: high performance on both GPU and CPU 6
GPU Performance: generate highly specialized shader code Geometry / Animation Features void my. Shader(. . . ) { <geom>. compute. Geometry(); float 4 color = Static. Mesh Displacement Skeletal. Anim Material Features Metal <material> . compute. Material(); <lighting>. compute. Lighting(color); } Cloth Glass Point. Light Skylight Light Features Spot. Light 7
Existing Approaches 8
Reflecting the graphics mental model in system decompositions Material Clay Plastic Matte Skin Light Point. Light Spot. Light Area. Light 9
Custom shader code generation tools Bungie TFX [Tatarchuk 2017] Unreal Engine Material Editor 10
Preprocessor Techniques for Shader Code Generation • Static “uber shaders” • #include HLSL code snippets 11
Static uber shader struct V 2 F { #if defined(NORMAL_MAPPING) float 3 tangent, bitangent; #endif. . . }; #if defined(NORMAL_MAPPING) Texture 2 D normal. Map; #endif V 2 F my. Vertex. Shader(. . . ) { #if defined(NORMAL_MAPPING) v 2 f. tangent =. . . ; v 2 f. bitangent =. . . ; #endif. . . } float 3 my. Fragment. Shader() { #if defined(NORMAL_MAPPING) float 3 normal = apply. Normal. Map(normal. Map, v 2 f. tangent, v 2 f. bitangent, v 2 f. uv); #endif. . . } 12
Achieving modularity: implement shading features in separate files Materials Lights Metal. Material. hlsl Point. Light. hlsl Skin. Material. hlsl Sh. Probe. Light. hlsl Cloth. Material. hlsl Area. Light. hlsl Hair. Material. hlsl 13
Shading System Implementation Strategies • C++ style object hierarchies • Dynamic dispatch, low performance • Custom Engine Specific Shader Compilation Tools • Engine specific, ad-hoc replication of standard compiler functionality • Preprocessor Techniques in plain HLSL • Difficult to maintain and extend • Error prone: lack of clearly defined interfaces 14
Contribution: Slang Shading Language • Modern shading system goals can be achieved by extending HLSL with a small set of general programming language constructs • • • Modular Composable Extensible Explicit Performant • Demonstration of Slang’s performance and extensibility benefits 15
Slang: extension of HLSL with modern language mechanisms • New language features include: • • • Interface constrained generics Associated types Parameter. Block<T> Retroactive extensions Global generic parameters Module import Git. Hub Page: https: //github. com/shader-slang/slang 16
Slang supports first class interface ILight { float 3 compute. Lighting(); } ILight Spotlight Skylight struct Spotlight : ILight { float 3 compute. Lighting() {. . . } } struct Skylight : ILight {. . . } 17
Support specialization via generics Shader entry-points are generic functions void forward. Pass. Shader<TGeometry, TMaterial, TLight> ( TGeometry geom, TMaterial material, TLight light) {. . . } Static. Mesh Displacement Skeletal Metal Cloth Glass forward. Pass. Shader<Displacement, Metal, Skylight> Spotlight Skylight 18
Type parameters are constrained by interfaces interface IGeometry {. . . } interface IMaterial {. . . } interface ILight { ILight float 3 compute. Lighting(); } void forward. Pass. Shader<TGeometry : IGeometry, TMaterial : IMaterial, TLighting : ILight > (TGeometry geom, TMaterial mat, TLighting lighting) {. . . lighting. compute. Lighting(); } Similar to Haskell’s type classes, Rust’s type traits and C# interfaces. Spotlight Skylight 19
Material shading returns a BRDF closure BRDF 1. Material Shading f = eval. Material(P) 2. Lighting Integration Lo = integrate(Li, f, Wi, Wo); 20
Material shading returns a BRDF closure Interface IBRDF{. . . } interface IMaterial{. . . } float 3 forward. Shader<M: IMaterial> (M material) { IBRDF f = material. eval. Material(p); float 3 l = 0; for (int i = 0; i<N; i++) l += f. eval(lights[i], p); return l; IMaterial Metal } 21
Material shading returns a BRDF closure Interface IBRDF{. . . } interface IMaterial{. . . } float 3 forward. Shader<M: IMaterial> (M material) { IBRDF f = material. eval. Material(p); float 3 l = 0; for (int i = 0; i<N; i++) l += f. eval(lights[i], p); return l; IMaterial Metal } 22
Material shading returns a BRDF closure Interface IBRDF{. . . } interface IMaterial{. . . } float 3 forward. Shader<M: IMaterial> (M material) { IBRDF f = material. eval. Material(p); float 3 l = 0; for (int i = 0; i<N; i++) l += f. eval(lights[i], p); return l; IMaterial Metal } 23
Associated types as interface requirement interface IMaterial { associatedtype B : IBRDF; B eval. Material(Position p); } struct Metal. Material : IMaterial { typedef Disney. BRDF B; B eval. Material(Position p) {. . . } } As in Haskell (as type families), Rust, Swift struct Skin. Material : IMaterial { typedef Skin. BRDF B; B eval. Material(Position p) {. . . } } 24
Summary Achieved both modularity and static specialization through two extensions to HLSL: • Interface Constrained Generics • Associated Types • Parameter. Block<T> type 25
Rearchitecting a shading system using Slang 26
27
Falcor’s shading system • Contains over 5000 lines of shader code, supporting: • • • A flexible, layered material system Point, spot, directional, and ambient light type Glossy reflection using an environment map Cascaded, exponential and variance shadow map algorithms Post processing effects, such as screen space ambient occlusion and tone mapping 28
Benefits of adopting Slang • Cleaner Code Structure • Easier to extend with new features • Achieves higher CPU and GPU performance 29
Cleaner Code Structure 30
Falcor implements a layered material library #if USE_DIFFUSE_LAYER compute. Diffuse. Layer(mat, . . . ); #endif #if USE_SPECULAR_LAYER compute. Specular. Layer(mat, . . . ); #endif #if USE_EMISSIVE_LAYER compute. Emissive. Layer(mat, . . . ); #endif Diffuse Specular Emissive 31
Make entry point specializable on materials 1. Define an IMaterial interface IMaterial {. . . } 2. Change shader entry point to use a generic material parameter void main<TMaterial : IMaterial>(Parameter. Block<TMaterial> material) {. . . } 32
Generics also useful for layered material specialization #if USE_DIFFUSE_LAYER compute. Diffuse. Layer(mat, . . . ); #endif #if USE_SPECULAR_LAYER compute. Specular. Layer(mat, . . . ); #endif #if USE_EMISSIVE_LAYER compute. Emissive. Layer(mat, . . . ); #endif struct Layered. Material < let has. Diffuse : bool, let has. Specular : bool, let has. Emissive: bool > : IMaterial { BRDF eval. Material(…) { if (has. Diffuse) compute. Diffuse. Layer(mat, . . . ); if (has. Specular) compute. Specular. Layer(mat, . . . ); if (has. Emissive) compute. Emissive. Layer(mat, . . . ); } } 33
Refactoring lighting to enable specialization interface ILight { float 3 compute. Lighting(TSurface surf); } struct Point. Light : ILight {. . . } struct Directional. Light : ILight {. . . } 34
Easier to Extend 35
New light type: polygonal area lights [Heitz et al. 2016] 36
Fewer changes to code in Slang code base Branch Original Refactored Sites Changed Files Changed Lines of Code 7 4 246 1 1 249 Changes are localized in one place in refactored branch, but distributed at 7 different places in original branch Localization of a feature’s implementation is a sign of good extensibility 37
Fewer changes to code in Slang code base Branch Original Refactored Sites Changed Files Changed Lines of Code 7 4 246 1 1 249 Lines of code is similar in both branches, Refactored architecture is not introducing significant boiler-plate 38
Higher Rendering Performance 39
Performance Evaluation: Environment Setup Goal: measure the CPU and GPU time required to draw each frame using both original and refactored Falcor branch Temple Bistro Interior Images rendered at 1920 * 1080 resolution, on Intel i 7 -5820 CPU and NVIDIA Titan V GPU Bistro Exterior 40
Higher GPU performance due to light specialization Original Branch Refactored Branch 41
Refactored branch achieved 30% speed-up in CPU time Original Branch Refactored Branch 42
Falcor main branch is using Slang exclusively • Over 17, 000 lines of shader code currently compiled by Slang • 20+ Researchers and developers from various groups within NVIDIA using Falcor to prototype rendering techniques are interacting with Slang code 43
Next Steps in Shading Languages 44
C++ for shaders ? ? • Should HLSL adopt C++ features or consider other modern language constructs? 45
Next Steps in Real-Time Shading Languages • Shading language support for real-time ray tracing 46
Next Steps in Shading Languages • Shading language support for ray tracing • More support for dynamic dispatch behavior • Unified CPU-GPU programming • Slang is currently a language for GPU code only, and interacts with CPU code via compiler runtime API • Next step: Unified programing language for CPU-GPU graphics programming 47
Thank you! github. com/shader-slang/slang Thank you to Nir Benty for valuable conversations. Support: NVIDIA Research National Science Foundation 48
Adding light specialization to original branch brings GPU performance on par with refactored branch Original Branch (+Light Specialization) Refactored Branch 49
Refactored branch achieved 30% speed-up in CPU time Original Branch (+Light Specialization) Refactored Branch 50
Adding light specialization makes original branch more difficult to extend Branch Original+LS Refactored Sites Changed 7 8 1 Files Changed 4 5 1 Lines of Code 246 253 249 51
- Slides: 51