Programmable Pipelines Computer Graphics Introduction Recent major advance

Programmable Pipelines Computer Graphics

Introduction Recent major advance in real time graphics is programmable pipeline First introduced by NVIDIA Ge. Force 3 (2001) Supported by high-end commodity cards ▪ NVIDIA, ATI AMD, 3 D Labs Software Support ▪ Direct X 8 , 9, 10 ▪ Open. GL Extensions ▪ Open. GL Shading Language (GLSL) ▪ Nvidia Cg 3 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Background Two components Vertex programs (shaders) Fragment programs (shaders) Requires detailed understanding of two seemingly contradictory apporachs Open. GL pipeline ▪ Real time Render. Man ideas ▪ offline 4 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Black Box View vertices CPU 5 fragments vertices Geometry Processor Rasterizer Hw-swgl 1 Hw-swgl 2 fragments Fragment Processor Hw-swgl 3 Frame Buffer

Geometric Calculations Geometric data: set of vertices + type Can come from program, evaluator, display list type: point, line, polygon Vertex data can be ▪ (x, y, z, w) coordinates of a vertex (gl. Vertex) ▪ Normal vector ▪ Texture Coordinates ▪ RGBA color ▪ Other data: color indices, edge flags ▪ Additional user-defined data in GLSL 8 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Per-Vertex Operations Vertex locations are transformed by the model-view matrix into eye coordinates Normals must be transformed with the inverse transpose of the model-view matrix so that v·n=v’ ·n’ in both spaces Assumes there is no scaling May have to use autonormalization Textures coordinates are generated if autotexture enabled and the texture matrix is applied 9

Lighting Calculations Consider a per-vertex basis Phong model I = kd Id l · n + ks Is (v · r )a + ka Ia diffuse specular ambient Phong model requires computation of r and v at every vertex 10

Primitive Assembly Vertices are next assembled into objects Polygons Line Segements Points Transformation by projection matrix Clipping Against user defined planes View volume, x=±w, y=±w, z=±w Polygon clipping can create new vertices Perspective Division Viewport mapping 11

Rasterization Geometric entities are rasterized into fragments Each fragment corresponds to a point on an integer grid: a displayed pixel Hence each fragment is a potential pixel Each fragment has A color Possibly a depth value Texture coordinates 12

Fragment Operations Texture generation Fog Antialiasing Scissoring Alpha test Blending Dithering Logical Operation Masking 13

Vertex Processor Takes in vertices Position attribute Possibly color Open. GL state Produces Position in clip coordinates Vertex color 14

Fragment Processor Takes in output of rasterizer (fragments) Vertex values have been interpolated over primitive by rasterizer Outputs a fragment Color Texture Fragments still go through fragment tests Hidden-surface removal alpha 15 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Programmable Shaders Replace fixed function vertex and fragment processing by programmable processors called shaders Can replace either or both If we use a programmable shader we must do all required functions of the fixed function processor 16

GLSL I Open. GL 2. X

Objectives Shader applications Vertex shaders Fragment shaders Programming shaders GLSL 21

Vertex Shader Applications General Vertex Lighting More realistic models Cartoon shaders Off loading CPU Dynamic Meshes Morphing 22

Fragment Shader Applications Per fragment lighting calculations per vertex lighting 23 per fragment lighting

Fragment Shader Applications Texture mapping smooth shading 24 environment mapping Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009 bump mapping

Writing Shaders First programmable shaders were programmed in an assembly-like manner Open. GL extensions added for vertex and fragment shaders Cg (C for graphics) C-like language for programming shaders Works with both Open. GL and Direct. X Interface to Open. GL complex Open. GL Shading Language (GLSL) 25 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

GLSL Open. GL Shading Language Part of Open. GL 2. 0 High level C-like language New data types Matrices Vectors Samplers Open. GL state available through built-in variables 26

Simple Vertex Shader const vec 4 red = vec 4(1. 0, 0. 0, 1. 0); void main(void) { gl_Position = gl_Projection. Matrix *gl_Model. View. Matrix*gl_Vertex; } gl_Front. Color = red; 27

Execution Model 28 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Simple Fragment Program void main(void) { } gl_Frag. Color = gl_Color; 29

Execution Model 30 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Data Types C types: int, float, bool Vectors: float vec 2, vec 3, vec 4 Also int (ivec) and boolean (bvec) Matrices: mat 2, mat 3, mat 4 Stored by columns Standard referencing m[row][column] C++ style constructors vec 3 a =vec 3(1. 0, 2. 0, 3. 0) vec 2 b = vec 2(a) 31

Pointers There are no pointers in GLSL We can use C structs which can be copied back from functions Because matrices and vectors are basic types they can be passed into and output from GLSL functions, e. g. mat 3 func(mat 3 a) 32 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Qualifiers GLSL has many of the same qualifiers such as const as C/C++ Need others due to the nature of the execution model Variables can change Once per primitive (uniform) Once per vertex (attribute) Once per fragment At any time in the application Vertex attributes are interpolated by the rasterizer into fragment attributes (varying) 33 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Attribute Qualifier Attribute-qualified variables can change at most once per vertex Cannot be used in fragment shaders Built in (Open. GL state variables) gl_Color gl_Model. View. Matrix User defined (in application program) attribute float temperature attribute vec 3 velocity 34 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Uniform Qualified Variables that are constant for an entire primitive Can be changed in application outside scope of gl. Begin and gl. End Cannot be changed in shader Used to pass information to shader such as the bounding box of a primitive 35 gl. Uniform( , ) gl. Begin(); gl. Vertex() gl. End(); Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Varying Qualified Variables that are passed from vertex shader to fragment shader Automatically interpolated by the rasterizer Built in Vertex colors Texture coordinates User defined Requires a user defined fragment shader 36 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Example: Vertex Shader const vec 4 red = vec 4(1. 0, 0. 0, 1. 0); varying vec 3 color_out; void main(void) { gl_Position = gl_Model. View. Projection. Matrix*gl_Vertex; } color_out = red; 37 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Required Fragment Shader varying vec 3 color_out; void main(void) { gl_Frag. Color = color_out; } Vertex Shader Fragment Shader 38

Passing values call by value-return Variables are copied in Returned values are copied back Three possibilities in out inout 39 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Operators and Functions Standard C functions Trigonometric Arithmetic Normalize, reflect, length Overloading of vector and matrix types mat 4 a; vec 4 b, c, d; c = b*a; // a column vector stored as a 1 d array d = a*b; // a row vector stored as a 1 d array 40 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Swizzling and Selection Can refer to array elements by element using [] or selection (. ) operator with x, y, z, w r, g, b, a s, t, p, q a[2], a. b, a. z, a. p are the same Swizzling operator lets us manipulate components vec 4 a; a. yz = vec 2(1. 0, 2. 0); 41 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Shading language changes after Open. GL 3. X

Qualifiers in, out Copy vertex attributes and other variable to/ from shaders in vec 2 tex_coord; out vec 4 color; uniform: variable from application uniform float time; uniform vec 4 rotation;

Built-in Variables gl_Position: output position from vertex shader gl_Frag. Color: output color from fragment shader Only for ES, Web. GL and older versions of GLSL Present version use an out variable

Simple Vertex Shader for Cube Example #version 330 core in vec 4 v. Position; in vec 4 v. Color; out vec 4 color; void main() { color=v. Color; gl_Position=v. Position; } // in CPU side GLuint v. Position = gl. Get. Attrib. Location( program, "v. Position" ); gl. Enable. Vertex. Attrib. Array( v. Position ); gl. Vertex. Attrib. Pointer( v. Position, 4, GL_FLOAT, GL_FALSE, 0, BUFFER_OFFSET(0) ); GLuint v. Color = gl. Get. Attrib. Location( program, "v. Color" ); gl. Enable. Vertex. Attrib. Array(

The Simplest Fragment Shader in vec 4 color; out vec 4 Frag. Color; void main() { Frag. Color = color; }

GLSL II

Objectives Coupling GLSL to Applications Example applications 48 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Linking with Application In Open. GL application, we must: Read shaders Compile shaders Define variables Link everything together

Reading a Shader are added to the program object and compiled Usual method of passing a shader is as a nullterminated string using the function gl. Shader. Source If the shader is in a file, we can write a reader to convert the file to a string 50 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Shader Reader #include <stdio. h> char* read. Shader. Source(const char* shader. File) { FILE* fp = fopen(shader. File, "r"); char* buf; long size; if(fp == NULL) return(NULL); fseek(fp, 0 L, SEEK_END); /* end of file */ size = ftell(fp); fseek(fp, )L, SEEK_SET); /* start of file */ 51 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Shader Reader (cont) buf = (char*) malloc(stat. Buf. st_size + 1 * sizeof(char)); fread(buf, 1, size, fp); buf[size] = ''; /null termination */ fclose(fp); return buf; } 52 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Adding a Vertex Shader GLint v. Shader; GLunit my. Vertex. Obj; GLchar v. Shaderfile[] = “my_vertex_shader”; GLchar* v. Source = read. Shader. Source(v. Shader. File); my. Vertex. Obj = gl. Create. Shader(GL_VERTEX_SHADER); gl. Shader. Source(my. Vertex. Obj, 1, & v. Source, NULL); gl. Compile. Shader(my. Vertex. Obj); 53

Program Object Container for shaders Can contain multiple shaders Other GLSL functions GLuint my. Prog. Obj; my. Prog. Obj = gl. Create. Program(); /* define shader objects here */ gl. Use. Program(my. Prog. Obj); gl. Attach. Shader(my. Prog. Obj, vertex. Shader ); gl. Attach. Shader(my. Prog. Obj, fragment. Shader ); gl. Link. Program(my. Prog. Obj); 54 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Vertex Attributes Vertex attributes are named in the shaders Linker forms a table Application can get index from table and tie it to an application variable Similar process for uniform variables 55 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Vertex Attribute Example GLint color. Attr; color. Attr = gl. Get. Attrib. Location(my. Prog. Obj, "my. Color"); /* my. Color is name in shader */ ---------------------GLfloat color[4]; gl. Vertex. Attrib 4 fv(color. Attrib, color); /* color is variable in application */ 56 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Uniform Variable Example GLint angle. Param; angle. Param = gl. Get. Uniform. Location(my. Prog. Obj, "angle"); /* angle defined in shader */ /* my_angle set in application */ GLfloat my_angle; my_angle = 5. 0 /* or some other value */ gl. Uniform 1 f(angle. Param, my_angle); 57 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Glsl. h #include "glsl. h“ cwc: : gl. Shader. Manager SM; cwc: : gl. Shader *shader; shader = SM. loadfrom. File("brick. vert", "brick. frag"); shader->begin(); shader->set. Uniform 3 f(“ucolor", 1, 0, 0); Draw. Something(); shader->end();

Vertex Shader with uniform vec 4 ucolor; void main(void) { gl_Position = gl_Projection. Matrix *gl_Model. View. Matrix*gl_Vertex; } gl_Front. Color = ucolor; 59

Vertex Shader Applications Moving vertices Morphing Wave motion Fractals Lighting More realistic models Cartoon shaders 60 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Wave Motion Vertex Shader uniform float time; uniform float xs, zs, // frequencies uniform float h; // height scale void main() { vec 4 t =gl_Vertex; t. y = gl_Vertex. y + h*sin(time + xs*gl_Vertex. x) + h*sin(time + zs*gl_Vertex. z); gl_Position = gl_Model. View. Projection. Matrix*t; } 61 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Particle System uniform vec 3 init_vel; uniform float g, m, t; void main() { vec 3 object_pos; object_pos. x = gl_Vertex. x + vel. x*t; object_pos. y = gl_Vertex. y + vel. y*t + g/(2. 0*m)*t*t; object_pos. z = gl_Vertex. z + vel. z*t; gl_Position = gl_Model. View. Projection. Matrix * vec 4(object_pos, 1); } 62 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Modified Phong Vertex Shader I void main(void) /* modified Phong vertex shader (without distance term) */ { gl_Position = gl_Model. View. Projection. Matrix * gl_Vertex; vec 4 ambient, diffuse, specular; vec 4 eye. Position = gl_Model. View. Matrix * gl_Vertex; vec 4 eye. Light. Pos = gl_Light. Source[0]. position; vec 3 N = normalize(gl_Normal. Matrix * gl_Normal); vec 3 L = normalize(eye. Light. Pos. xyz - eye. Position. xyz); vec 3 E = -normalize(eye. Position. xyz); vec 3 H = normalize(L + E); 63 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Modified Phong Vertex Shader II /* compute diffuse, ambient, and specular contributions */ float Kd = max(dot(L, N), 0. 0); float Ks = pow(max(dot(N, H), 0. 0), gl_Front. Material. shininess); float Ka = 0. 0; ambient = Ka*gl_Front. Light. Product[0]. ambient; diffuse = Kd*gl_Front. Light. Product[0]. diffuse; specular = Ks*gl_Front. Light. Product[0]. specular; gl_Front. Color = ambient+diffuse+specular; } 64 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Pass Through Fragment Shader /* pass-through fragment shader */ void main(void) { gl_Frag. Color = gl_Color; } 65 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Vertex Shader for per Fragment Lighting varying vec 3 N, L, E, H; void main() { gl_Position = gl_Model. View. Projection. Matrix * gl_Vertex; vec 4 eye. Position = gl_Model. View. Matrix * gl_Vertex; vec 4 eye. Light. Pos = gl_Light. Source[0]. position; N = normalize(gl_Normal. Matrix * gl_Normal); L = normalize(eye. Light. Pos. xyz - eye. Position. xyz); E = -normalize(eye. Position. xyz); H = normalize(L + E); } 66 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Fragment Shader for Modified Phong Lighting I varying vec 3 N; varying vec 3 L; varying vec 3 E; varying vec 3 H; void main() { vec 3 Normal = normalize(N); vec 3 Light = normalize(L); vec 3 Eye = normalize(E); vec 3 Half = normalize(H); 67 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Fragment Shader for Modified Phong Lighting II float Kd = max(dot(Normal, Light), 0. 0); float Ks = pow(max(dot(Half, Normal), 0. 0), gl_Front. Material. shininess); float Ka = 0. 0; vec 4 diffuse = Kd * gl_Front. Light. Product[0]. diffuse; vec 4 specular = Ks * gl_Front. Light. Product[0]. specular; vec 4 ambient = Ka * gl_Front. Light. Product[0]. ambient; gl_Frag. Color = ambient + diffuse + specular; } 68 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Vertex vs Fragment Lighting per vertex lighting 69 per fragment lighting Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Samplers Provides access to a texture object Defined for 1, 2, and 3 dimensional textures and for cube maps In shader: uniform sampler 2 D my. Texture; Vec 2 texcoord; Vec 4 texcolor = texture 2 D(mytexture, texcoord); In application: tex. Map. Location = gl. Get. Uniform. Location(my. Prog, “my. Texture”); gl. Uniform 1 i(tex. Map. Location, 0); /* assigns to texture unit 0 */ 70 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Fragment Shader Applications Texture mapping smooth shading environment mapping Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009 bump mapping 71

Cube Maps We can form a cube map texture by defining six 2 D texture maps that correspond to the sides of a box Supported by Open. GL Also supported in GLSL through cubemap sampler vec 4 tex. Color = texture. Cube(mycube, texcoord); Texture coordinates must be 3 D 72 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Environment Map Use reflection vector to locate texture in cube map 73 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Environment Maps with Shaders Environment map usually computed in world coordinates which can differ from object coordinates because of modeling matrix May have to keep track of modeling matrix and pass it shader as a uniform variable Can also use reflection map or refraction map (for example to simulate water) 74 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Reflection Map Vertex Shader varying vec 3 R; void main(void) { gl_Position = gl_Model. View. Projection. Matrix*gl_Vertex; vec 3 N = normalize(gl_Normal. Matrix*gl_Normal); vec 4 eye. Pos = gl_Model. View. Matrix*gl_Vertex; R = reflect(-eye. Pos. xyz, N); } 75 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Refelction Map Fragment Shader varying vec 3 R; uniform sampler. Cube tex. Map; void main(void) { gl_Frag. Color = texture. Cube(tex. Map, R); } 76 Angel: Interactive Computer Graphics 5 E © Addison-Wesley 2009

Bump Mapping Perturb normal for each fragment Store perturbation as textures 77