Chapter X Advanced Texturing 3 D Graphics for

Chapter X Advanced Texturing 3 D Graphics for Game Programming (J. Han)

Environment Mapping § Environment mapping simulates a shiny object reflecting its surrounding environment. Remember the morphing cyborg in T 2!! 3 D Graphics for Game Programming (J. Han) 10 -2 2

Environment Mapping (cont’d) § The first task for environment mapping is to capture the environment images in a texture called an environment map. § The most popular implementation is using a cube map. It’s cube mapping. 3 D Graphics for Game Programming (J. Han) 10 -3 3

Cube Mapping § To determine the reflected color at p, a ray is fired from the viewpoint toward p. The ray denoted by I is reflected with respect to the surface normal n at p. § HLSL library function tex. CUBE() takes a cube map and R as input, and returns an RGB color. 3 D Graphics for Game Programming (J. Han) 10 -4 4

Cube Mapping (cont’d) § The cube map faces are named {+x, -x, +y, -y, +z, -z}. 3 D Graphics for Game Programming (J. Han) 10 -5 5

Cube Mapping (cont’d) § The cube map face intersected by R is identified using R's coordinate that has the largest absolute value. The remaining coordinates are divided by the largest absolute value, and then range-converted from [-1, 1] to [0, 1]. 3 D Graphics for Game Programming (J. Han) 10 -6 6

Cube Mapping (cont’d) § We have to use (u, v), but R returns (u', v'). § Note that (u, v) and (u', v') will produce the same filtering result only when the environment is infinitely far away from p. § Fortunately, people are fairly forgiving about physical incorrectness that results when the environment is not far away. 3 D Graphics for Game Programming (J. Han) 10 -7 7

Dynamic Cube Mapping § A cube map is created at run time, and then immediately used for environment mapping. § Two-pass algorithm § § 1 st pass: The Geometry Shader replicates the incoming primitive into six separate primitives, and each primitive is rasterized on its own render target which refers to a general off-screen buffer. Six images are generated. 2 nd pass: The scene is rendered using the cube map. § For the first pass, multiple render targets (MRT) are used for enabling the rendering pipeline to produce images to multiple render target textures at once. 3 D Graphics for Game Programming (J. Han) 10 -8 8

Light Mapping § Suppose a character navigating a static environment illuminated by static light sources. § Even though the viewpoint is moving, the diffuse reflection remains constant over the environment surfaces. § Then, we can pre-compute part of the diffuse reflection ahead of time, store the result in a light map, and look up the light map at run time. 3 D Graphics for Game Programming (J. Han) 10 -9 9

Light Mapping (cont’d) § When creating a light map, there is no real-time constraint. Therefore, the light map is computed using a global illumination algorithm, especially the radiosity algorithm. § The light map is combined with the image texture at run time, i. e. , the diffuse reflectance is read from the image texture, incoming light is read from the light map, and they are multiplied. 3 D Graphics for Game Programming (J. Han) 10 -10 10

Light Mapping (cont’d) § Quake II released in 1997 was the first commercial game that used the light map. 3 D Graphics for Game Programming (J. Han) 10 -11 11

Radiosity Normal Mapping § Radiosity computation adopts the concept of hemisphere. Its orientation is defined by the surface normal np. See below. § Consider combining light mapping and normal mapping. The normal n(u, v) fetched from the normal map is the perturbed instance of np. If lighting were computed on the fly, n(u, v) would be used. However, the light map that is going to partly replace lighting computation was created using the unperturbed normal np. § A simple solution to this discrepancy would be to use the perturbed normals stored in the normal map when the light map is created. Then, the light map would have the same resolution as the normal map. It is unnecessarily large. 3 D Graphics for Game Programming (J. Han) 10 -12 12

Radiosity Normal Mapping (cont’d) § In the solution of Half-Life 2, three vectors, v 0, v 1, and v 2, which form an orthonormal basis, are pre-defined in the tangent space. § Each vi is transformed into the world space, a hemisphere is placed, and the radiosity preprocessor is run. Then, incoming light is computed per vi, and three different colors are computed for p. If we repeat this for all sampled points of the object surface, we obtain three light maps for the object. § Each light map is often called a directional light map. 3 D Graphics for Game Programming (J. Han) 10 -13 13

Radiosity Normal Mapping (cont’d) § At run time, the directional light maps are blended. § The normal fetched from the normal map, (nx, ny, nz), is a tangent-space normal and can be redefined in terms of the basis {v 0, v 1, v 2}. § Let (n'x, n'y, n'z) denote the transformed normal. They represent the coordinates with respect to the basis {v 0, v 1, v 2}, and are used for computing the weights needed for blending the three light maps. 3 D Graphics for Game Programming (J. Han) 10 -14 14

Radiosity Normal Mapping (cont’d) 3 D Graphics for Game Programming (J. Han) 10 -15 15

Radiosity Normal Mapping (cont’d) § The resolutions of the directional light maps and the normal map. § § § In (a), the macrostructure is densely sampled for creating the normal map. In (b), the macrostructure is sparsely sampled for creating the directional light maps. When two fragment, f 1 and f 2, are processed at run time, the directional light maps would provide appropriate colors for both f 1 and f 2. In (c), the traditional light map with the same resolution as the directional light maps would provide an incorrect light color f 2. 3 D Graphics for Game Programming (J. Han) 10 -16 16

Why Shadows? § Shadows help us understand the spatial relationships among objects in a scene, especially between the occluders and receivers. (Occluders cast shadows onto receivers. ) 3 D Graphics for Game Programming (J. Han) 10 -17 17

Why Shadows? (cont’d) § Shadows increase the realism. 3 D Graphics for Game Programming (J. Han) 10 -18 18

Terminologies in Shadows § A point light source generates only fully shadowed regions, called hard shadows. § An area light source generates soft shadows, which consist of § § Umbra: fully shadowed regions Penumbra: partially shadowed regions § This chapter will focus on hard shadows even though a lot of real-time algorithms for soft shadow generation are available. 3 D Graphics for Game Programming (J. Han) 10 -19 19

Shadow Mapping § Two-pass algorithm § Pass 1 § § Render the scene from the position of the light source. Store the depths into the shadow map, which is a depth map with respect to the light source. 3 D Graphics for Game Programming (J. Han) 10 -20 20

Shadow Mapping (cont’d) § Pass 2 § § Render the scene from the camera position. It’s real rendering. For each pixel, compare its distance d to the light source with the depth value z in the shadow map. § If d > z, the pixel is in shadows. § Otherwise, the pixel is lit. 3 D Graphics for Game Programming (J. Han) 10 -21 21

Shadow Mapping Artifact – Surface Acne § The algorithm works, but its brute-force implementation suffers from two major problems. § § Surface acne - a mixture of shadowed and lit areas Shadow-map resolution sensitivity 3 D Graphics for Game Programming (J. Han) 10 -22 22

Shadow Mapping Artifact – Surface Acne (cont’d) § What’s the problem? The shadowed and lit pixels coexist on a surface area that should be entirely lit. § Note that the scene points sampled at the second pass are usually different from the scene points sampled at the first pass. § Suppose the nearest point sampling. § In the example, p 1 is to be lit, but judged to be in shadows because d 1 > z 1. § For p 1, it should be d 1 < z 1. 3 D Graphics for Game Programming (J. Han) 10 -23 23

Shadow Mapping Artifact – Surface Acne (cont’d) § At the 2 nd pass, add a small bias value to z 1 such that d 1 < z 1´. 3 D Graphics for Game Programming (J. Han) 10 -24 24

Shadow Mapping Artifact – Surface Acne (cont’d) § The bias value is usually fixed after a few trials. § Anyway, the surface acne problem has been largely resolved. 3 D Graphics for Game Programming (J. Han) 10 -25 25

Shadow Mapping Artifact – Resolution Sensitivity § If the resolution of a shadow map is not high enough, multiple pixels may be mapped to a single texel of the shadow map. § This is a magnification example, but bilinear interpolation wouldn’t help as you will see very soon. 3 D Graphics for Game Programming (J. Han) 10 -26 26

Shadow Mapping Artifact – Resolution Sensitivity (cont’d) § A simple but expensive solution is to increase the shadow map resolution. § Many other solutions have been proposed. 3 D Graphics for Game Programming (J. Han) 10 -27 27

Shadow Mapping – Shader § After every vertex is transformed into the world space, the view parameters are specified with respect to the light source. Consequently, the viewtransformed vertices are defined in the so-called light space. The view frustum is then specified in the light space. 3 D Graphics for Game Programming (J. Han) 10 -28 28

Shadow Mapping – Shader (cont’d) 3 D Graphics for Game Programming (J. Han) 10 -29 29

Shadow Mapping – Shader (cont’d) 3 D Graphics for Game Programming (J. Han) 10 -30 30

Shadow Mapping – Shader (cont’d) 3 D Graphics for Game Programming (J. Han) 10 -31 31

Shadow Mapping – Filtering § Recall that, for filtering the shadow map, we assumed nearest point sampling. An alternative would be bilinear interpolation. § The problems of bilinear interpolation § § It is not good to have completely different results depending on the filtering options. The shadow quality is not improved at all and the shadow still reveals the jagged edges. 3 D Graphics for Game Programming (J. Han) 10 -32 32

Shadow Mapping – Filtering (cont’d) § A solution to this problem is to first determine the visibilities of a pixel with respect to the four texels, and then interpolate the visibilities. This value is taken as the “degree of being lit. ” § In general, the technique of taking multiple texels from the shadow map and blending the pixel's visibilities against the texels is named percentage closer filtering (PCF). 3 D Graphics for Game Programming (J. Han) 10 -33 33

Shadow Mapping – Filtering (cont’d) § Direct 3 D 10 supports PCF through a new function Sample. Cmp. Level. Zero(). § In the example, the visibility is computed for each of nine texels, and the visibilities are averaged. 3 D Graphics for Game Programming (J. Han) 10 -34 34

Ambient Occlusion § We have assumed that the ambient light arrives at a surface point from all directions. In reality, however, part of the directions may be occluded by the external environment. Ambient occlusion algorithm computes how much of the ambient light is occluded, which we call the occlusion degree. § Imagine casting rays along the opposite directions of the incident light. Some rays intersect the scene geometry. § The occlusion degree is defined as the ratio between the intersected and all cast rays. However, it’s expensive. 3 D Graphics for Game Programming (J. Han) 10 -35 35

Screen Space Ambient Occlusion § The occlusion degree can be approximated as the ratio between the occupied space and the entire hemisphere’s space. § Instead of computing the space volumes, let’s take a set of samples, and then determine if each sample is inside or outside. 3 D Graphics for Game Programming (J. Han) 10 -36 36

Screen Space Ambient Occlusion (cont’d) § Take the z-buffer of the scene (seen from the camera) as the discrete approximation of the scene geometry. It’s depth map that can be created using the algorithm presented in the first pass of shadow mapping. § At the second pass, a quad covering the entire screen (more precisely, the viewport) is rendered such that each fragment of the quad corresponds to a pixel of the z-buffer. § Let’s do comparison that is similar to the one performed for shadow mapping. In the example, s 1 is outside as d 1 < z 1. However, s 2 is inside as d 2 > z 2. The occlusion degree is 3/8. § Each of the occlusion degrees can be used to modulate sa in the ambient reflection term. 3 D Graphics for Game Programming (J. Han) 10 -37 37

Screen Space Ambient Occlusion (cont’d) § The result is great!! § However, we have errors!! 3 D Graphics for Game Programming (J. Han) 10 -38 38

Screen Space Ambient Occlusion (cont’d) § Gears of War example § Lighting is one of the elements influenced the most by GPU evolution. § The state of the art in real-time lighting has been gradually moving away from the classic implementation of the Phong model. 3 D Graphics for Game Programming (J. Han) 10 -39 39

Deferred Shading § A most notable technique that extensively utilizes the MRT is deferred shading. This technique does not shade a fragment that will fail the depth test, i. e. , shading is deferred until all “visible surfaces” of the scene are determined. § It is split into geometry pass and lighting passes. § At the geometry pass, various per-pixel attributes are computed and stored into MRT by a fragment shader. The per-pixel attributes may include texture colors, depths, and normals “of the visible surfaces. ” 3 D Graphics for Game Programming (J. Han) 10 -40 40

Deferred Shading (cont’d) § The MRT filled at the geometry pass is often called the G-buffer which stands for geometric buffer. Screen-space image texture Depth map Screen-space normal map Shading using the G-buffer. 3 D Graphics for Game Programming (J. Han) 10 -41 41

Deferred Shading (cont’d) § At the lighting passes, a large number of light sources can be applied to the G-buffer. § All light sources are iterated. For a light source, a full-screen quad is rendered (as was done by the SSAO algorithm) such that each fragment of the quad corresponds to a pixel on the screen. The fragment shader takes the G-buffer as input, computes lighting, and determines the color of the current pixel. It is blended with the current content of the color buffer, which was computed using the previous light sources. 3 D Graphics for Game Programming (J. Han) 10 -42 42