Chapters VIII Image Texturing Introduction to Computer Graphics

Chapters VIII Image Texturing Introduction to Computer Graphics with Open. GL ES (J. Han)

Where are We? § While the vertex shader outputs the clip-space vertices, the rasterizer outputs a set of fragments at the screen space. § The per-fragment attributes may include a normal vector, a set of texture coordinates, etc. § Using these data, the fragment shader determines the final color of each fragment. § Two most important things to do in the fragment shader § Lighting § Texturing § Before moving on to the fragment shader, let’s see the basics of texturing. Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -2 2

Texture Coordinates § The simplest among the various texturing methods is image texturing. § A texture is typically represented as a 2 D array of texels (texture elements). A texel’s location can be described by the coordinates of its center. § Image texturing is often described as pasting an image on an object's surface. Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -3 3

Texture Coordinates (cont’d) § For texturing, we assign the texture coordinates, (s, t), to each vertex of the polygon mesh “at the modeling step, ” as will be presented soon. § The rasterizer interpolates them for the fragments. § The normalized texture coordinates (s, t) are projected into the texture space. Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -4 4

Texture Coordinates (cont’d) § It is customary to normalize the texture coordinates such that s and t range from 0 to 1. The normalized texture coordinates do not depend on a specific texture resolution and can be freely plugged into various textures. Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -5 5

Texture Coordinates (cont’d) § Multiple images of different resolutions can be glued to a surface without changing the texture coordinates. Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -6 6

Surface Parameterization § The texture coordinates are assigned to the vertices of the polygon mesh. This process is called surface parameterization or simply parameterization. In general, parameterization requires unfolding a 3 D surface onto a 2 D planar domain. Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -7 7

Chart and Atlas § The surface to be textured is often subdivided into a (hopefully small) number of patches. § Each patch is unfolded and parameterized. Then, the artist draws an image for each patch. An image for a patch is often called a chart. § Multiple charts are usually packed and arranged in a texture, which is often called an atlas. Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -8 8

Texturing in GL § Assume that an image texture file is loaded into the GL program to fill tex. Data. § Observe that the above is similar to the process of creating buffer objects for the vertex array. Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -9 9

Texturing in GL (cont’d) Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -10 10

Texture Wrapping Mode § The texture coordinates (s, t) are not necessarily in the range of [0, 1]. The texture wrapping mode handles (s, t) that are outside of the range. § § § Clamp-to-Edge: The out-of-range coordinates are rendered with the edge colors. Repeat: The texture is tiled at every integer junction. Mirrored-Repaet: The texture is mirrored or reflected at every integer junction. Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -11 11

Texture Filtering – Magnification & Minification § Consider a quad. For each fragment located at (x, y) in the screen, its texture coordinates (s, t) are projected onto (s', t') in the texture space. § Note that (s', t') are floating-point values in almost all cases. § Consequently, texels around (s', t') are sampled. This sampling process is called texture filtering. § The screen-space quad may appear larger than the image texture, and therefore the texture is magnified so as to fit to the quad. There are more pixels than texels. Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -12 12

Texture Filtering – Magnification & Minification (cont’d) § In contrast, the screen-space quad may appear smaller than the image texture, and the texture is minified. Sparsely projected onto the texture space, the pixels take large jumps in the space. § Summary § Magnification: more pixels than texels § Minification: less pixels than texels Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -13 13

Filtering for Magnification § Option 1: Nearest point sampling § § A block of pixels can be mapped to a single texel. Consequently, adjacent pixel blocks can change abruptly from one texel to the next texel, and a blocky image is often produced. Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -14 14

Filtering for Magnification (cont’d) § Option 2: Bilinear interpolation § It is preferred to nearest point sampling not only because the final result suffers much less from the blocky image problem but also because the graphics hardware is usually optimized for bilinear interpolation. Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -15 15

Filtering for Minification § Consider a checker-board image texture. § If all pixels are surrounded by dark-gray texels, the textured quad appears dark gray. If all pixels are surrounded by light-gray texels, the textured quad appears light gray. Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -16 16

Mipmap § The aliasing problem observed in minification is caused by the fact that we have more texels than pixels. The pixels take large jumps in the texture space, leaving many texels not involved in texture ltering. § Then, a simple solution is to reduce the number of texels such that the texel count becomes as close as possible to the pixel count. § For decreasing the texture size, down-sampling is adopted. § Given an original texture of 2 lx 2 l resolution, a pyramid of (l + 1) levels is constructed, where the original texture is located at level 0. The pyramid is called a mipmap. The level to filter is named level of detail, and is denoted by λ. We will look for the level whose texel count is close to the pixel count. Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -17 17

Mipmapping (cont’d) § A pixel covers an area on the screen. For simplicity, take the area as square. Then, a pixel’s projection onto the texture space is not a point but an area centered at at (s', t'). The projected area is called the footprint of the pixel. § In this contrived example, the quad and level-1 texture have the same size. A pixel’s footprint covers 2 x 2 texels in level 0, but covers a single texel in level 1. § When a pixel footprint covers mxm texels of level-0 texture, λ is set to log 2 m. Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -18 18

Mipmapping (cont’d) § In the example, the screen-space quad and level-2 texture have the same size. A pixel's footprint covers exactly a single texel in level-2 texture, which is then filtered. § In this and previous examples, observe that all texels are involved in filtering. Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -19 19

Mipmapping (cont’d) § In the following example, λ = log 23 = 1. 585. . , and so we see levels 1 and 2, more formally λ and λ . Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -20 20

Mipmapping (cont’d) § Between λ and λ , which one to choose? § We can take the nearest level, λ+0. 5 § At the level, we can take the nearest point. § In contrast, we can adopt bilinear interpolation. § We can take both of them and linearly interpolate the filtering results. § At each level, we can take the nearest point. § In contrast, if bilinear interpolation is performed at each level, this is called trilinear interpolation. Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -21 21

Texture Filtering in GL § GL provides void gl. Generate. Mipmap(GLenum target) where target can be GL_TEXTURE_2 D. § Given a screen-space pixel, GL itself computes its footprint to determine whether the texture is magnified or minified. In either case, the filtering method has yet to be specified by invoking gl. Tex. Parameteri. § gl. Tex. Parameteri(GLenum target, GLenum pname, GLint param) § target is either GL_TEXTURE_2 D or GL_TEXTURE_CUBE_MAP § pname can be the following: § GL_TEXTURE_MAG_FILTER § GL_TEXTURE_MIN_FILTER § GL_TEXTURE_WRAP_S § GL_TEXTURE_WRAP_T § param § If pname is GL_TEXTURE_MAG_FILTER, param is either GL_NEAREST or GL_LINEAR (for bilinear interpolation) Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -22 22

Texture Filtering in GL (cont’d) § gl. Tex. Parameteri(GLenum target, GLenum pname, GLint param) § param § If pname is GL_TEXTURE_MIN_FILTER, param can be: § GL_NEAREST no mipmapping which level(s) to choose? § GL_LINEAR § GL_NEAREST_MIPMAP_NEAREST § GL_NEAREST_MIPMAP_LINEAR how to filter § GL_LINEAR_MIPMAP_NEAREST each level? § GL_LINEAR_MIPMAP_LINEAR § When we do not use the mipmap, the level-0 texture is filtered either by nearest point sampling or by bilinear interpolation. § GL_LINEAR_MIPMAP_LINEAR corresponds to the trilinear interpolation. Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -23 23

Vertex Shader and Rasterizer (revisited) § Our first vertex shader § The per-vertex texture coordinates are interpolated for each fragment and then passed to the fragment shader. Introduction to Computer Graphics with Open. GL ES (J. Han) 6 -24 24

Vertex and Fragment Shaders § The inputs to the fragment shader § Inputs: The per-vertex output variables produced by the vertex shader are interpolated to determine the per-fragment ones. § Uniforms to be used by the fragment shader. § Samplers = Textures. § The output: one or more fragment colors that are passed to the output merger. Introduction to Computer Graphics with Open. GL ES (J. Han) 9 -25 25

Fragment Shader § The fragment shader runs once for each fragment and each fragment is processed independently of the others. § The built-in function, texture, filters the sampler with the filtering method set up by gl. Tex. Parameteri. Introduction to Computer Graphics with Open. GL ES (J. Han) 8 -26 26

Texturing Examples texturing only Introduction to Computer Graphics with Open. GL ES (J. Han) texturing + lighting (CH 9) 8 -27 27