Texture Mapping Kurt Akeley CS 248 Lecture 10

Texture Mapping Kurt Akeley CS 248 Lecture 10 25 October 2007 http: //graphics. stanford. edu/courses/cs 248 -07/

Texture mapping demo CS 248 Lecture 10 Kurt Akeley, Fall 2007

Texture mapping Paints images onto triangles Paints images onto points, lines, and other images CS 248 Lecture 10 Kurt Akeley, Fall 2007

Complete Open. GL pipeline Pixel pipeline Application Vertex pipeline Vertex assembly Pixel assembly (unpack) Texture mapping ties the vertex and pixel pipelines together Vertex operations Primitive assembly Pixel operations Primitive operations Pixel pack Rasterization Texture memory Fragment operations Application Framebuffer CS 248 Lecture 10 Kurt. Display Akeley, Fall 2007

Texture mapping Paints images onto triangles Paints images onto points, lines, and other images Ties the vertex and pixel pipelines together n Rendered images can be used as textures n To modify the rendering of new images – That can be used as textures … Implements general functions of one, two, or three parameters n Specified as 1 -D, 2 -D, or 3 -D tables (aka texture images) n With interpolated (aka filtered) lookup Drives the hardware architecture of GPUs n Multi-thread latency hiding n “shader” programmability Adds many capabilities to Open. GL n Volume rendering n Alternate color spaces n Shadows … CS 248 Lecture 10 Kurt Akeley, Fall 2007

Complete Open. GL pipeline Pixel pipeline Application Vertex pipeline Vertex assembly Pixel assembly (unpack) Vertex operations Primitive assembly Pixel operations Primitive operations Pixel pack Rasterization Texture memory Fragment operations Application Framebuffer CS 248 Lecture 10 Kurt. Display Akeley, Fall 2007

Fundamentals of texture mapping Texture mapping requires specification of n The texture image n The mapping from object to texture coordinates n The sampling mechanism n The application of the resulting value(s) CS 248 Lecture 10 Kurt Akeley, Fall 2007

The Open. GL Pipeline (texture mapping example) CS 248 Lecture 10 Kurt Akeley, Fall 2007

Open. GL textured-quad code Load. Texture(“Textures\rooster. tga”, 1); // lots hidden here gl. Tex. Envi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE); gl. Enable(GL_TEXTURE_2 D); gl. Bind. Texture(GL_TEXTURE_2 D, 1); gl. Clear. Color(1, 1, 1, 1); // white gl. Clear(GL_COLOR_BUFFER_BIT); gl. Load. Identity(); gl. Ortho(0, 100, -1, 1); gl. Color 3 f(1, 1, 1); // white gl. Begin(GL_TRIANGLE_STRIP); gl. Tex. Coord 2 f(0, 0); gl. Vertex 2 i(11, 31); gl. Tex. Coord 2 f(0, 1); gl. Vertex 2 i(37, 71); gl. Tex. Coord 2 f(1, 0); gl. Vertex 2 i(91, 38); gl. Tex. Coord 2 f(1, 1); gl. Vertex 2 i(65, 71); gl. End(); gl. Flush(); CS 248 Lecture 10 Texture mapping is not corrected for perspective! Kurt Akeley, Fall 2007

Texture image specification Unpack from memory format Pixel pipeline into canonical format (pixel structures) Vertex pipeline Application Vertex assembly Pixel assembly (unpack) Pixel operations Pixel pack struct { float r, g, b, a; } pixel; Vertex operations Scale and offset Primitive assembly Color table lookup Optional: Primitive operations Convolution Histogram Min/max computation Rasterization Texture memory Fragment operations Application Framebuffer CS 248 Lecture 10 Kurt. Display Akeley, Fall 2007

Vertex assembly gl. Color 3 f(1, 1, 1); gl. Begin(GL_TRIANGLE_STRIP); gl. Tex. Coord 2 f(0, 0); gl. Vertex 2 i(11, 31); Pixel pipeline Pixel assembly (unpack) Pixel operations Pixel pack Application Vertex pipeline struct { float xo, yo, zo, wo; float nxo, nyo, nzo; float r, g, b, a; float so, to, ro, qo; } vertex; Vertex assembly Vertex operations Primitive assembly struct { Primitive operations float 11, 31, 0, 1; float 0, 0, 1; float 1, 1, 1, 1; Rasterization float 0, 0, 0, 1; } vertex; Texture memory Fragment operations Application Framebuffer CS 248 Lecture 10 Kurt. Display Akeley, Fall 2007

Vertex operations Pixel pipeline Application Vertex pipeline Vertex assembly Pixel assembly (unpack) Pixel operations struct { float xc, yc, zc, wc; float r, g, b, a; float sc, tc, rc, qc; } clipvertex; Pixel pack Vertex operations Primitive assembly Primitive operations Rasterization Texture memory Fragment operations Application Framebuffer CS 248 Lecture 10 Kurt. Display Akeley, Fall 2007

Primitive assembly Pixel pipeline Pixel assembly (unpack) Pixel operations Application struct { float xc, yc, zc, wc; float r, g, b, a; float sc, tc, rc, qc; } clipvertex; struct { clipvertex v 0, v 1, v 2; } triangle; Pixel pack Vertex pipeline Vertex assembly Vertex operations Primitive assembly Primitive operations Rasterization Texture memory Fragment operations Application Framebuffer CS 248 Lecture 10 Kurt. Display Akeley, Fall 2007

Primitive operations Pixel pipeline Pixel assembly (unpack) Pixel operations Application struct { float xw, yw, zw, wc; float r, g, b, a; float sc, tc, rc, qc; } winvertex; struct { winvertex v 0, v 1, v 2; } triangle; Pixel pack Vertex pipeline Vertex assembly Vertex operations Primitive assembly Primitive operations Rasterization Texture memory Fragment operations Application Framebuffer CS 248 Lecture 10 Kurt. Display Akeley, Fall 2007

Rasterization Pixel pipeline Pixel assembly (unpack) Pixel operations Application struct { float xw, yw, zw, wc; float r, g, b, a; float sc, tc, rc, qc; } winvertex; struct { winvertex v 0, v 1, v 2; } triangle; Pixel pack Vertex pipeline Vertex assembly Vertex operations Primitive assembly Primitive operations Rasterization Texture memory Fragment operations Application Framebuffer CS 248 Lecture 10 Kurt. Display Akeley, Fall 2007

Perspective-correct attribute evaluation (x 1, y 1, w 1, f 1) (x, y, f ) All w’s are wc’s (x 0, y 0, w 0, f 0) a 2 a 0 a 1 (x 2, y 2, w 2, f 2) CS 248 Lecture 10 Kurt Akeley, Fall 2007

Perspective-correct tex coord evaluation wc’s, sc’s, qc’s xw’s, yw’s (x 0, y 0, w 0, s 0) a 2 (x 1, ycoordinates Texture 1, w 1, s 1) homogenized here Supports image (x, remapping y, s ) (from q-specified warp to render-specified warp) a 0 a 1 (x 2, y 2, w 2, s 2) CS 248 Lecture 10 Kurt Akeley, Fall 2007

Rasterization Pixel pipeline Application Vertex pipeline Vertex assembly Pixel assembly (unpack) Vertex operations Pixel operations struct { float xw, yw, zw; float r, g, b, a; float sw, tw, rw; } fragment; Pixel pack Primitive assembly Primitive operations Rasterization Texture memory Fragment operations Application Framebuffer CS 248 Lecture 10 Kurt. Display Akeley, Fall 2007

Fragment operations Pixel pipeline Application Vertex pipeline Vertex assembly Pixel assembly (unpack) Vertex operations Pixel operations struct { float xw, yw, zw; float r, g, b, a; float sw, tw, rw; } fragment; Pixel pack Primitive assembly Primitive operations Rasterization Texture memory Fragment operations Application Framebuffer CS 248 Lecture 10 Kurt. Display Akeley, Fall 2007

Texture lookup 1 struct { float xw, yw, zw; float r, g, b, a; float sw, tw, rw; } fragment; tw 0 0 sw 1 struct { float rt, gt, bt, at; } color; If sw or tw is outside the range [0, 1]: - Clamp to edge color - Clamp to border color - Wrap repeatedly - Wrap with mirror reflections CS 248 Lecture 10 Kurt Akeley, Fall 2007

Texture application 1 struct { float xw, yw, zw; float r, g, b, a; float sw, tw, rw; } fragment; tw 0 0 Alternatives include: - GL_REPLACE - GL_BLEND - GL_ADD 1 sw GL_MODULATE: CS 248 Lecture 10 struct { float rt, gt, bt, at; } color; struct { float xw, yw, zw; float r’, g’, b’, a’; } fragment; Kurt Akeley, Fall 2007

Fragment / framebuffer operations Pixel pipeline Application Vertex pipeline Vertex assembly Pixel assembly (unpack) Vertex operations Pixel operations struct { float xw, yw, zw; float r’, g’, b’, a’; } fragment; Pixel pack Primitive assembly Primitive operations Rasterization Texture memory Fragment operations Application Framebuffer CS 248 Lecture 10 Kurt. Display Akeley, Fall 2007

Open. GL textured-quad code Load. Texture(“Textures\rooster. tga”, 1); // lots hidden here gl. Tex. Envi(GL_TEXTURE_ENV, GL_TEXTURE_ENV_MODE, GL_MODULATE); gl. Enable(GL_TEXTURE_2 D); gl. Bind. Texture(GL_TEXTURE_2 D, 1); gl. Clear. Color(1, 1, 1, 1); // white gl. Clear(GL_COLOR_BUFFER_BIT); gl. Load. Identity(); gl. Ortho(0, 100, -1, 1); gl. Color 3 f(1, 1, 1); // white gl. Begin(GL_TRIANGLE_STRIP); gl. Tex. Coord 2 f(0, 0); gl. Vertex 2 i(11, 31); gl. Tex. Coord 2 f(0, 1); gl. Vertex 2 i(37, 71); gl. Tex. Coord 2 f(1, 0); gl. Vertex 2 i(91, 38); gl. Tex. Coord 2 f(1, 1); gl. Vertex 2 i(65, 71); gl. End(); gl. Flush(); run CS 248 Lecture 10 Kurt Akeley, Fall 2007

Texture Sampling CS 248 Lecture 10 Kurt Akeley, Fall 2007

Texel coordinates 2 m 1 v 0 256 tw 256 0 0 0 sw u 1 struct { float xw, yw, zw; float r, g, b, a; float sw, tw, rw; } fragment; struct { float rt, gt, bt, at; } color; 2 n For this image n = m = 256 CS 248 Lecture 10 Kurt Akeley, Fall 2007

Image resampling We are resampling the texture image at the pixel rate We have seen resampling before … CS 248 Lecture 10 Kurt Akeley, Fall 2007

Supersample antialiasing Supersampling algorithm: 1. Over-sample, e. g. , at 4 x the pixel rate 2. Reconstruct at the over-sampled rate 3. Band-limit to match the pixel rate 4. Resample at the pixel rate to yield pixel values 5. Reconstruct to display CS 248 Lecture 10 Kurt Akeley, Fall 2007

Texture resampling (to avoid aliasing) Texture resampling algorithm: 1. Over-sample, e. g. , at 4 x the pixel rate 2. Reconstruct at the texture-sample rate 3. Band-limit to match the pixel rate 4. Resample at the pixel rate to yield pixel values 5. Reconstruct to display CS 248 Lecture 10 Kurt Akeley, Fall 2007

Supersampling optimizations The over-sample reconstruction convolution and the band-limiting convolution steps can be combined: n Convolution is associative n n (f * g) * h = f * (g * h) And g and h are constant n f * (g * h) = f * filter The filter convolution can be reduced to a simple weighted sum of sample values: n The result is sampled at pixel rate n So only values at pixel centers are needed n These are weighted sums of the 4 x samples CS 248 Lecture 10 Kurt Akeley, Fall 2007

Texture resampling optimizations The reconstruction convolution and the band-limiting convolution steps can be combined: n Convolution is associative n n (f * g) * h = f * (g * h) And g and h are constant n f * (g * h) = f * filter The filter convolution can be reduced to a simple weighted sum of sample values: n The result is sampled at pixel rate n So only values at pixel centers are needed How texels to be summed to the get atexels good answer? n many These areneed weighted sums of CS 248 Lecture 10 Kurt Akeley, Fall 2007

Size of the filter n typically differs for every fragment! CS 248 Lecture 10 Kurt Akeley, Fall 2007

Magnification One pixel corresponds to less than one texel Resulting image is “larger” (more pixels than corresponding texels) Efficient to implement: Rule 1 CS 248 Lecture 10 Kurt Akeley, Fall 2007

Minification One pixel corresponds to many texels Resulting image is “smaller” (fewer pixels than corresponding texels) Difficult to implement efficiently Could require visiting every texel in the entire texture image! CS 248 Lecture 10 Kurt Akeley, Fall 2007

Two approaches to antialiasing Supersampling Pre-filtering CS 248 Lecture 10 Kurt Akeley, Fall 2007

MIPmap pre-filtering Prefilter repeatedly to ½ resolution Reduce resolution equally in all dimensions Stop at a single texel CS 248 Lecture 10 Kurt Akeley, Fall 2007

MIPmap filtering Choose the pre-filtered images that most closely match the pixel sample rate Sample each of these images Combine the resulting values with a MIPmap filter whose weights are determined by the fractional value of p Notes: n Any filter can be chosen for the in-image sampling n Usually a bilinear filter is chosen for the MIPmap filter n Hence two pre-filtered images are chosen and sampled Choose the single nearest MIPmap level improves preformance Kurt Akeley, Fall 2007 CS 248 Lecture 10 n

Tri-linear filtering A horrible term! Usually means sampling a 2 -D MIPmapped texture image using bilinear interpolation for both the in-image filtering and the MIPmap filtering Open. GL allows separate specification of in-image and MIPmap filters CS 248 Lecture 10 Kurt Akeley, Fall 2007

2 -D and 3 -D textures Spatial rates typically differ in different dimensions: This conservative approach will result in image blurring Can we do better? CS 248 Lecture 10 Kurt Akeley, Fall 2007

RIPmap pre-filtering CS 248 Lecture 10 Kurt Akeley, Fall 2007

Anisotropic texture filtering Combine pre-filtering and supersampling Typical algorithm: n Pre-filter to MIPmap images n Supersample the pixel area n For each sample perform a full MIPmap sample operation n Combine the MIPmap samples using pixel-customized filter weights CS 248 Lecture 10 Kurt Akeley, Fall 2007

Summary Texture mapping is a powerful, general-purpose mechanism n It’s not just painting pictures onto triangles! Texture mapping requires specification of n The texture image n The mapping from object to texture coordinates n The sampling mechanism n The application of the resulting value(s) Of these, sampling is the greatest challenge n Use pre-filtering and supersampling approaches to n CS 248 Lecture 10 n Avoid aliasing Avoid blurring Kurt Akeley, Fall 2007

Assignments Tuesday 30 October n Kurt will be out of town n n Saturday morning until Tuesday late Lecture with be given by Andrew Adams Reading assignment for Tuesday’s class n Work through the tutorial at: n http: //www. lighthouse 3 d. com/opengl/glsl/ Project 3: n Write a computer-graphics game n Will be assigned this coming Tuesday Office hours today ? CS 248 Lecture 10 Kurt Akeley, Fall 2007

End CS 248 Lecture 10 Kurt Akeley, Fall 2007