RealTime Shading Using Programmable Graphics Hardware Introduction Setup

Real-Time Shading Using Programmable Graphics Hardware Introduction, Setup and Examples Wan-Chun Ma National Taiwan University

Course Infomation n Instructor Wan-Chun Ma, Alex ¨ Dept. of Computer Science and Information Engineering, National Taiwan University ¨ http: //graphics. csie. ntu. edu. tw/~firebird ¨ n Course 4/28, 5/5, 5/12, 5/14 ¨ Suggested readings ¨ n n ¨ R. Fernando and M. J. Kilgard. The Cg Tutorial: The Definitive Guide to Programmable Real-Time Graphics, Addison-Wesley, 2003 (beginners only!) R. J. Rost. Open. GL Shading Language, Addison-Wesley, 2004 http: //graphics. csie. ntu. edu. tw/~firebird/dokuwiki/doku. php? id=tech: courses: dci_rts: home

The Student. . . n The student should be familiar with ¨ C/C++ programming ¨ Graphics basics Transformations in 3 D (translation, rotation, modelview, projection) n Rasterization n Texturing n ¨ Open. GL GLUT, GLUI n Use texturing in Open. GL n

Today’s Schedule Introduction n Setup of Programming Environment n Real-Time Shading Examples n

Introduction

Evolution of GPUs Virtual Fighter SEGA Dead or Alive 3 Temco Dawn Demo NVIDIA NV 1 Xbox (NV 2 A) Ge. Force FX (NV 30) 50 K triangles/sec 1 M pixel ops/sec 1 M transistors 100 M triangles/sec 1 G pixel ops/sec 20 M transistors 200 M triangles/sec 2 G pixel ops/sec 120 M transistors 1995 2001 2003

The 5 Generations of GPU n 1 st generation (up to 1998) NVIDIA TNT 2, ATI Rage, 3 dfx Voodoo 3 ¨ Lack of transform vertices of 3 D objects. Vertex transformation are done by CPU ¨ Limited math operations for combining textures to compute the color of pixels ¨ n 2 nd generation (1999 -2000) NVIDIA Ge. Force 256, Ge. Force 2, ATI Radeon 7500 ¨ GPU has the ability to do transformation and lighting. Both Open. GL and Direct. X 7 support vertex transformation by hardware ¨ Configurable (in driver level) but not programmable ¨

The 5 Generations of GPU n 3 rd generation (2001) NVIDIA Ge. Force 3, Ge. Force 4 Ti, Xbox, ATI Radeon 8500 Vertex programmability: Direct. X 8 vertex shader and Open. GL ARB vertex program ¨ Pixel-level configurable ¨ ¨ n 4 th generation (2002) ¨ ¨ ¨ n NVIDIA Ge. Force FX, ATI Radeon 9700 Vertex and pixel programmability High-level shading language (NVIDIA Cg, Microsoft HLSL, Open. GL GLSL) 5 th generation (2004) ¨ ¨ ¨ NVIDIA Ge. Force 6, ATI Radeon X Infinite length shader program Dynamic flow control

GPU Model (Old) n Fixed function pipeline

GPU Model (Current) n Programmability!

GPU Process Vertex Processing Fragment Processing

Programming GPU n However, programming in assembly is painful DP 3 R 0, c[11]. xyzx; RSQ R 0, R 0. x; MUL R 0, R 0. x, c[11]. xyzx; MOV R 1, c[3]; MUL R 1, R 1. x, c[0]. xyzx; DP 3 R 2, R 1. xyzx; RSQ R 2, R 2. x; MUL R 1, R 2. x, R 1. xyzx; ADD R 2, R 0. xyzx, R 1. xyzx; DP 3 R 3, R 2. xyzx; RSQ R 3, R 3. x; MUL R 2, R 3. x, R 2. xyzx; DP 3 R 2, R 1. xyzx, R 2. xyzx; MAX R 2, c[3]. z, R 2. x; MOV R 2. z, c[3]. y; MOV R 2. w, c[3]. y; LIT R 2, R 2; DX 8 shader instructions n Basic: mov, add, mul, mad, rsq… n Vector operation: dp 3, dp 4… n Miscellaneous: lit, exp, log, min, max, tex… Oh my god!

Programming GPU n The need of high level shading language Compile DP 3 R 0, c[11]. xyzx; RSQ R 0, R 0. x; MUL R 0, R 0. x, c[11]. xyzx; MOV R 1, c[3]; MUL R 1, R 1. x, c[0]. xyzx; DP 3 R 2, R 1. xyzx; RSQ R 2, R 2. x; MUL R 1, R 2. x, R 1. xyzx; ADD R 2, R 0. xyzx, R 1. xyzx; DP 3 R 3, R 2. xyzx; RSQ R 3, R 3. x; MUL R 2, R 3. x, R 2. xyzx; DP 3 R 2, R 1. xyzx, R 2. xyzx; MAX R 2, c[3]. z, R 2. x; MOV R 2. z, c[3]. y; MOV R 2. w, c[3]. y; LIT R 2, R 2; // A Phong model shader COLOR c = k_a + k_d * dot(N, L) + k_s * pow(max(0, dot(N, H)), k_exp); High level shading language n Easier to read and modify n Cross-platform n Code reuse

Cg: A Shading Language n n n Cg is a high level language from NVIDIA for programming GPUs, developed in close collaboration with Microsoft Cg stands for “C for Graphics” Cg enables a dramatic productivity increase for graphics development developers of: ¨ Games ¨ CAD tools ¨ Scientific visualizations

Cg: A C-like Language n n n Syntax, operators, functions from C Conditionals and flow control (for, if) Particularly suitable for GPUs: Express data flow of pipeline/stream architecture of GPUs (e. g. vertex-to-pixel) ¨ Vector and matrix operations ¨ Support hardware data types for maximum performance ¨ Exposes GPU functions for convenience and speed: ¨ n n n Intrinsic: (mul, dot, sqrt, exp, pow) Built-in: extremely useful and GPU optimized math, utility and geometric functions (noise, ddx, ddy, reflect) Compiler uses hardware profiles to subset Cg as required for particular hardware feature sets

Cg Workflow n Architecture

Cg Workflow Shader Development // Diffuse lighting float d = dot(normalize(N), normalize(L)); if (d < 0) d = 0; c = d*tex 2 D(texture, T)*diffuse; Cg program source code Cg Compiler Shader program assembly code DP 3 r 0. x, f[TEX 0]; RSQ r 0. x, r 0. x; MUL r 0, r 0. x, f[TEX 0]; DP 3 r 1. x, f[TEX 1]; RSQ r 1. x, r 1. x; MUL r 1, r 1. x, f[TEX 1]; DP 3 r 0, r 1; MAX r 0. x, 1. 0; MUL r 0, r 0. x, DIFFUSE; TEX r 1, f[TEX 1], 0, 2 D; MUL r 0, r 1; Shader Compiler Shader binary 0000 h: 0010 h: 0020 h: 0030 h: 0040 h: 54 65 32 20 61 68 69 73 20 65 2 D 54 65 58 2 C 20 56 72 73 69 6 F 6 E 20 33 2 E 31 34 31 35 39 32 2 D 2 E 31 20 28 4 D 69 4 B 54 65 58 20 32 2 E 34 29 28 70 72 65 6 C 6 F 61 64 65 64 20 66 6 F 72 6 D 74 3 D 6 C 61 74 65 78 20 32 30 30 34 2 E 36 2 E 1. 2. 3. 4. Application Load/bind program Specify program parameter Specify vertex inputs Render

What Cg can do? n Real-time visual effects

What Cg can do? n Lots of effects…

Coffee Break n Next section: Setup of Programming Environment

Setup of Programming Environment

Requirement n Hardware ¨ The computer should be equipped with programmable graphics hardware ¨ NVIDIA FX, NVIDIA 6, ATI 9 x 00, ATI X series n Software ¨ Microsoft Visual Studio. NET 2003 ¨ GLUT, GLUI. . .

Installation n Cg Toolkit 1. 3 (10 MB) ¨ http: //developer. nvidia. com/object/cg_toolkit. ht ml ¨ Check the “Cg Installer for Windows” n NVIDIA SDK 9. 0 (340 MB, not required) ¨ http: //developer. nvidia. com/object/sdk_home. h tml n FX Composer 1. 6 (60 MB, not required) ¨ http: //developer. nvidia. com/object/fx_compose r_home. html ¨ Check the “FX Composer 1. 6 Installer”

Installation n n If default installation locations are used, all the packages are installed in the folder of C: Program FilesNVIDIA Corporation ¨ Cg ¨ NVIDIA FX Composer ¨ SDK 9. 0

My Stuff n Several useful codes I collect ¨ http: //graphics. csie. ntu. edu. tw/~firebird /download/dci_rts/class. zip n Download it and unpack it into a folder, say ¨ D: My ProjectsClass ¨ Any folder is ok, but remember where you put it

VC++ Directories n Execute visual studio ¨ Tools, Options, Projects, VC++ Directories ¨ Show the directories for: ¨ Include files D: My ProjectClass (remember My Stuff? ) n C: Program FilesNVIDIA CorporationCginclude n ¨ Library n files C: Program FilesNVIDIA CorporationCglib

Ready to Go n A small engine ¨ http: //graphics. csie. ntu. edu. tw/~firebird /download/dci_rts/env. zip ¨ I will use this engine for shader development during these courses n The first example ¨ http: //graphics. csie. ntu. edu. tw/~firebird/ download/dci_rts/ex 01. zip

Compilation n cgc –profiles filename ¨ profiles: graphics hardware profiles Vertex: arbvp 1, vp 20, vp 30, vp 40. . . n Fragment: arbfp 1, fp 20, fp 30, fp 40. . . n ¨ filename: n Examples ¨ cgc filename of the shader –profile vp 30 test_vtx. cxx ¨ cgc –profile fp 30 test_frg. cxx

Debugging is very hard (it is GPU, not CPU) n However, you may still use intermediate visualization to debug your program n ¨ Output intermediate data (e. g. position, normal, textures…) as color

Coffee Break n Next section: Real-time Shading Examples

Real-Time Shading Examples

Progression n Games push hardware, hardware advances games

Effects in Games Reflection Level of detail Smoke Shading Shadows

Effects in Games Per-pixel lighting Multi-texturing Light mapping Bump mapping

Multi-pass Rendering n The rendering pass is not fixed anymore. A single rendering pass may consists of many functional programs

Multi-pass Rendering n Each different program (effect) is handled individually, and finally summed up to become rendering result

Cg Samples n Check out the effect samples in NVIDIA SDK Browser

The First Cg Example n n A Phong model shader with color texture Shaders ¨ Vertex: ex 1_vtx. cxx ¨ Fragment: ex 1_frg. cxx n Texture ¨ Diffuse: wood. bmp

Vertex Shader n Vertex-to-fragment data structure struct v 2 f { float 4 P 2 D float 4 C float 4 T float 3 P 3 D float 3 N float 3 G float 3 B }; : : : : POSITION; // projected 2 D position COLOR 0; // color TEXCOORD 0; // texture coord TEXCOORD 1; // vertex 3 D position TEXCOORD 2; // normal TEXCOORD 3; // tangent TEXCOORD 4; // binormal

Vertex Shader Main (application-to-vertex) arguments v 2 f main( n float 4 C : COLOR, float 4 P : POSITION, float 4 N : NORMAL, float 4 T : TEXCOORD 0, uniform float 4 x 4 Model. View. Proj, uniform float 4 x 4 Model. View. IT)

Vertex Shader n Main body { v 2 f OUT; OUT. P 2 D = OUT. P 3 D = OUT. T = OUT. N = OUT. G = OUT. B = return OUT; } mul(Model. View. Proj, P); P. xyz; T; normalize(N. xyz); // normalize(2. 0*C. xyz - 1. 0); // tangent normalize(cross(OUT. G, OUT. N));

Fragment Shader n Fragment-to-screen data structure struct f 2 s { float 4 C }; : COLOR 0;

Fragment Shader n Main (vertex-to-fragment) arguments f 2 s main( v 2 f IN, uniform sampler 2 D uniform float 3 tex 01, // texture 01 L, V)

Fragment Shader n Main body { f 2 s OUT; OUT. rgb = 0; L = normalize(L); V = normalize(V); float 3 H = normalize(L+V); float diff = dot(normalize(IN. N), L); if(diff > 0) { float spec = 2*pow(dot(IN. N, H), 128); OUT. C. rgb = diff*tex 2 D(tex 01, IN. T. xy) + spec; } return OUT; }

Result

Try This. . . n Output red color for all fragments { f 2 s OUT; OUT. rgb = 0; // L = normalize(L); V = normalize(V); // float 3 H = normalize(L+V); // float diff = dot(normalize(IN. N), L); // if(diff > 0) // { // float spec = 2*pow(dot(IN. N, H), 128); OUT. C. rgb = float 3(1. 0, 0. 0); // } return OUT; }

Try This. . . n Visualize normal vectors { f 2 s OUT; OUT. rgb = 0; // L = normalize(L); V = normalize(V); // float 3 H = normalize(L+V); // float diff = dot(normalize(IN. N), L); // if(diff > 0) // { // float spec = 2*pow(dot(IN. N, H), 128); OUT. C. rgb = (IN. N+1)/2; // } return OUT; }

End