Introduction to Web GL Web GL n n

Introduction to Web. GL

Web. GL n n n <canvas> has 3 D option—Web. GL—for lowlevel 3 D graphics Web. GL ≈ Open. GL ES 2. 0 (embedded systems) Supported by all major browsers except IE Working group: Apple, Google, Mozilla, Opera (not MS) Low-level API, not for faint of heart (Most users will use higher-level libraries) Good book: Web. GL: Up and Running

Three. js n n Web. GL is low-level; 3 D is hard work Need libraries for higher-level capabilities n n n Object models Scene graphs Display lists Physics We’ll start with raw Web. GL examples, then move to Three. js

Web. GL overview n Steps to 3 D graphics: n n n n Create a canvas element Obtain drawing context Initialize the viewport Create buffers of data (vertices) to be rendered Create model and view matrices Create shaders Draw

Graphics Pipeline

How would you do this?

Web. GL Concepts n n n n n Buffers Render. Buffer Frame. Buffer Textures Blending Depth buffer Stencil buffer Uniform variables Attribute variables

Shaders n n n GLSL: GL Shader Language C-like syntax Vertex shaders: per-vertex computation Fragment shaders: per-pixel computation SIMD-like architecture

Vertex Shaders n n Little program to process a vertex Inputs: n n n Outputs: “varying variables” Tasks n n Per-vertex inputs supplied as vertex arrays (locations, normals, colors, texture coords, etc. ) Uniforms (non-varying variables) Samplers (textures, displacement maps, etc. ) Shader program Transformations Per-vertex lighting Generating or transforming texture coordinates Simplest one?

Example Vertex Shader uniform mat 4 u. MVMatrix; uniform mat 4 u. PMatrix; // modelview matrix // perspective matrix attribute vec 4 a. Vertex. Position; attribute vec 4 a. Vertex. Color; // position of vertex // color of vertex // varying variables: input to fragment shader varying vec 4 v. Color; // output vertex color void main() { gl_Position = u. PMatrix * u. MVMatrix * a. Vertex. Position; v. Color = a. Vertex. Color; }

Primitive Assembly n n Individual vertices are assembled into primitives (triangles, lines, or point-sprites) Trivial accept-reject culling (is the primitive entirely outside the view frustum? ) Backface culling Clipping (cut away parts of primitive outside view frustum)

Rasterization n n Convert primitives into 2 D “fragments” (representing pixels on the screen) Different algorithms for triangles, lines, and point-sprites

Fragment Shaders n n Little program to process a fragment (pixel) Inputs: n n n Output n n gl_Frag. Color Tasks n n Varying variables (outputs of vertex shader, interpolated) Uniforms Samplers Shader program Per-vertex operations such as Phong shading Simplest one?

Example Fragment Shader precision highp float; varying vec 4 v. Color; void main(void) { gl_Frag. Color = v. Color; } // numeric precision // (lowp, mediump, highp) // input vertex color

Per-Fragment Operations n Operations on fragment data: n n n Pixel ownership test Scissor test Stencil test Depth test Blending Dithering

Graphics Pipeline in Detail Application n Scene/Geometry database traversal n Movement of objects, camera n Animated movement of models n Visibility check, occlusion culling n Select level of detail Geometry n Transform from model frame to world frame n Transform from world frame to view frame (modelview matrix) n Project (projection matrix) n Trivial accept/reject culling n n n Backface culling Lighting Perspective division Clipping Transform to screen space Rasterization n Scanline conversion n Shading n Texturing n Fog n Alpha tests n Depth buffering n Antialiasing n Display

Distributed Computing n n n Some work is done on the CPU, some on processors on the graphics card E. g. read an object file on the CPU. Set it up on the various processors on the graphics card for rendering How to get the data to the graphics card?

Vertex Buffer Objects n n Vertex data must be sent to the graphics card for display Web. GL uses Vertex Buffer Objects n n n Create an array (chunk of memory) for vertex data (position, color, etc) and vertex indices Put it in a Vertex Buffer Object Send it to the graphics card, where it is stored

Hello Web. GL n n n Lots of machinery to draw a triangle But once the framework is in place, the rest is “easy”… Steps: n n n Compile the shaders Attach to program object Link Connect vertex outputs to fragment inputs Connect other variables and uniforms

The Shaders var frag. Shader = " precision highp float; varying vec 4 v. Color; void main(void) { gl_Frag. Color = v. Color; } "; var vert. Shader = " attribute vec 3 a. Vertex. Position; attribute vec 4 a. Vertex. Color; uniform mat 4 u. MVMatrix; uniform mat 4 u. PMatrix; varying vec 4 v. Color; void main(void) { gl_Position = u. PMatrix * u. MVMatrix * vec 4(a. Vertex. Position, 1. 0); v. Color = a. Vertex. Color; }";

Compiling the Shaders (glx. js) glx. load. Shader = function(type, shader. Src) { var shader, compile. Status; shader = gl. create. Shader(type); if (shader == 0) return 0; gl. shader. Source(shader, shader. Src); gl. compile. Shader(shader); compile. Status = gl. get. Shader. Parameter(shader, gl. COMPILE_STATUS); if (!compile. Status) { alert(gl. get. Shader. Info. Log(shader)); gl. delete. Shader(shader); return 0; } return shader; }

Linking the Shaders (glx. js) glx. load. Programs = function(vert. Shader. Src, frag. Shader. Src) { var vert. Shader, frag. Shader, program. Object, link. Status; vert. Shader = glx. load. Shader(gl. VERTEX_SHADER, vert. Shader. Src); frag. Shader = glx. load. Shader(gl. FRAGMENT_SHADER, frag. Shader. Src); program. Object = gl. create. Program(); gl. attach. Shader(program. Object, vert. Shader); gl. attach. Shader(program. Object, frag. Shader); gl. link. Program(program. Object); // link programs link. Status = gl. get. Program. Parameter(program. Object, gl. LINK_STATUS); if (!link. Status) { alert(gl. get. Program. Info. Log(program. Object)); gl. delete. Program(program. Object); return 0; } return program. Object; }

Connecting Arguments var shader. Program; function init. Shaders() { shader. Program = glx. load. Programs(vert. Shader, frag. Shader); gl. use. Program(shader. Program); shader. Program. vertex. Position. Attribute = gl. get. Attrib. Location(shader. Program, "a. Vertex. Position"); gl. enable. Vertex. Attrib. Array(shader. Program. vertex. Position. Attribute); shader. Program. vertex. Color. Attribute = gl. get. Attrib. Location(shader. Program, "a. Vertex. Color"); gl. enable. Vertex. Attrib. Array(shader. Program. vertex. Color. Attribute); shader. Program. p. Matrix. Uniform = gl. get. Uniform. Location(shader. Program, "u. PMatrix"); shader. Program. mv. Matrix. Uniform = gl. get. Uniform. Location(shader. Program, "u. MVMatrix"); }

Setting Up the View function setup. View() { gl. viewport(0, 0, gl. viewport. Width, gl. viewport. Height); p. Matrix = mat 4. perspective(30, gl. viewport. Width / gl. viewport. Height, 0. 1, 100. 0); mat 4. identity(mv. Matrix); mat 4. translate(mv. Matrix, [0. 0, -6. 0]); //mat 4. look. At(0, 0, -6, 0, 0, 1, 0, mv. Matrix); gl. uniform. Matrix 4 fv(shader. Program. p. Matrix. Uniform, false, p. Matrix); gl. uniform. Matrix 4 fv(shader. Program. mv. Matrix. Uniform, false, mv. Matrix); }

Vertex Buffers n Array of vertex data to be sent to graphics card Each vertex may have 4 coords, 2 texture coords, 4 color values, 3 normal coords… 80 bytes or more n Setup: n n n gl. create. Buffer() gl. bind. Buffer() gl. buffer. Data() make a new buffer make it our “current buffer” put data in the buffer Draw: n n gl. vertex. Attrib. Pointer() use buffer for vertex attribute gl. draw. Arrays() draw using specified buffer

Draw Scene function draw. Scene() { setup. View(); gl. clear(gl. COLOR_BUFFER_BIT | gl. DEPTH_BUFFER_BIT); gl. bind. Buffer(gl. ARRAY_BUFFER, triangle. Vertex. Position. Buffer); gl. vertex. Attrib. Pointer(shader. Program. vertex. Position. Attribute, triangle. Vertex. Position. Buffer. item. Size, gl. FLOAT, false, 0, 0); gl. bind. Buffer(gl. ARRAY_BUFFER, triangle. Vertex. Color. Buffer); gl. vertex. Attrib. Pointer(shader. Program. vertex. Color. Attribute, triangle. Vertex. Color. Buffer. item. Size, gl. FLOAT, false, 0, 0); gl. draw. Arrays(gl. TRIANGLES, 0, triangle. Vertex. Position. Buffer. num. Items); }

Initialize function init. GL(canvas) { gl = canvas. get. Context("experimental-webgl"); gl. viewport. Width = canvas. width; gl. viewport. Height = canvas. height; gl. clear. Color(0. 0, 1. 0); gl. clear. Depth(1. 0); gl. enable(gl. DEPTH_TEST); gl. depth. Func(gl. LEQUAL); } function web. GLStart() { var canvas = document. get. Element. By. Id("canvas 1"); init. GL(canvas); init. Shaders(); init. Buffers(); set. Interval(draw. Scene, 20); }

Using Matrices (gl. Matrix. js) n learningwebgl. com uses gl. Matrix. js (glmatrix. net): n n Types: vec 3, mat 4, quat 4 Functions: n n n create, set, identity add, subtract, negate, multiply, scale, normalize dot, cross, transpose, determinant, inverse lerp translate, scale, rotate frustum, perspective, ortho, look. At

Animation example

Web. GL Primitives n draw. Arrays modes: n n n n POINTS LINE_LOOP LINE_STRIP TRIANGLES TRIANGLE_STRIP TRIANGLE_FAN Other shapes?

Polygons n n In Open. GL, to ensure correct display, polygons must be simple, convex, and flat Web. GL can only do triangles What about complex shapes? Non-flat shapes?

Polygon Triangulation n The Van Gogh algorithm n n O(n 2) time Better algorithms can achieve O(n log n) time (plane sweep) n Or O(n log n) time n Or O(n log* n) time n Or ? ?

Other primitives n Text n n use HTML, CSS Curved objects (Bezier curves, NURBS surfaces, etc)? n n Make triangles in JS Or use Open. GL

Hidden surface removal n n n How can we prevent hidden surfaces from being displayed? Painter's algorithm: paint from back to front. How can we do this by computer, when polygons come in arbitrary order?

HSR Example n Which polygon should be drawn first?

Depth buffer (z-buffer) alg n Hidden surface removal is accomplished on a per-pixel basis in hardware with a depth buffer (also called z-buffer): n n n When computing screen coordinates for each pixel, also compute distance Z from viewer When drawing each pixel, draw R, G, B, A in the color buffer and Z in the depth buffer Only draw the pixel if it's closer than what was there before.

Depth-buffer images Color buffer Depth buffer

Depth Buffer in Web. GL n Enable depth buffering gl. enable(gl. DEPTH_TEST); gl. depth. Func(gl. LEQUAL); n When you clear a buffer, also clear the depth buffer gl. clear(gl. COLOR_BUFFER_BIT | gl. DEPTH_BUFFER_BIT);

Depth Buffer Analysis n n n Every pixel of every polygon is drawn, even if most don't appear in final image – theoretically slow in some cases Supported in all modern 3 D graphics hardware Pixel-sized depth values results in aliasing

Open. GL buffers n n n Color Depth Stencil n n n Accumulation n Restrict drawing to certain portions of the screen E. g. cardboard cutout Can "add together" different versions of an image Anti-aliasing, motion blur, soft shadows, compositing E. g. how to do fog?

Phew. n n Lots of work to write a Web. GL program, set up buffers and shaders, etc. Can we do cool stuff with much less code?

Three. js Features

Three. js n n n Written by Mr. doob aka Cabello Miguel of Spain Perceived leader of Web. GL frameworks Documentation is thin, but 200+ examples

First Three. js Program n A document to draw on:

Three. js basics n To display something with Three. js we need: n n n A scene A camera A renderer

Adding geometry n Now we need to add an object to the scene:

Render the scene

Three. JS overview n n Documentation thin, incomplete. [More examples] Types of objects: n n n Cameras (orthographic, perspective) Controllers (firstperson, fly, path, roll, trackball) Scenes Renderers (Web. GL, Canvas, SVG) Objects (mesh, line, particle, bone, sprite, etc) Geometries (cube, cylinder, sphere, lathe, text, etc) Lights, Materials Loaders Animation (animation. Handler, morph. Target) Collision detection

Project: animated flower n n Make a 3 D flower Simple version: n n n Doesn’t have to be realistic Use a function for petals, etc. Make it rotate or move Trackball controller Fancier version: n n More realistic Animated, e. g. bends in the wind, slider to open/close flower, etc.

Geometry n How would you create geometry?

Creating Geometry n n Use an object like Cube. Geometry, Cylinder. Geometry, Polyhedron. Geometry, etc to create an object Add it to your scene Documentation: Check out example (or look at source code)

Creating Geometry

Virtual Trackball? n How would you figure out how to set up a virtual trackball?

Trackball controller n n n Use the Trackball. Controls camera controller Example Check out example (or look at source code)

Trackball controller

Lighting? n n n Lights: Ambient. Light, Directional. Light, Point. Light, Spot. Light Documentation: Check out an example anyway

Lighting in Three. js

Shading and material types n Material types: n n Basic Material Lambert Material Phong Material Parameters/properties: n Color, wireframe, shading, vertex. Colors, fog, light. Map, specular. Map, env. Map, skinning, morph. Targets

Shading and material types

Gradients n Use vertex colors

Moving your objects around n n object. positon. set(x, y, z) object. rotation. x = 90 * Math. PI / 180 n n Rotations occur in the order x, y, z With respect to object’s internal coord system If there is an x-rotation, y and z rotations may not be lined up with world axes Object properties (parent-relative): n n n Position Rotation Scale

Object Hierarchy n n What if you want to create an object with parts? Object transform hierarchy n n n Scene: top-level object in hierarchy Can add objects to other objects Move or rotate one part: its children move as well

Chapter 3 - 64 Interactive Computer Graphics

Chapter 3 - 65

How might you do this?

Morphing n n n Image/video morphing: smoothly shifting from one image to another First popularized in a Michael Jackson video Method for video: a combination of n n n Identifying corresponding points in images over time Warping both images, gradually moving control points from location in first image to location in the second Cross-fading from first image sequence to second

3 D Morphing n n Define 3 D before and after shapes Linear interpolation of point locations from first setting to second

Morphing in Three. js n n Create geometry Move vertices to create “morph targets” n n Set influence n n n geometry. morph. Targets. push( { name: “target” + i, vertices: vertices } ); mesh. morph. Target. Influences[0]=0. 3; mesh. morph. Target. Influences[1]=0. 7; Can also set up animations that can be played (people walking, etc)

Morphing in Three. js n n Morph. Anim. Mesh documentation: “todo” See morph target example [2] [3] [4]

Summary n n n Web. GL is Open. GL ES in the browser Distributed and SIMD-like programming Vertex and fragment shaders Web. GL graphics pipeline Depth buffer algorithm for hidden surface removal Three. js is nice!