CS 248 Graphics Performance Debugging and Optimisation Dave

CS 248: Graphics Performance, Debugging and Optimisation Dave Oldcorn November 13 th 2007 Graphics Performance and Optimisation

Your Guest Instructor • Back in the mists of time, I wrote games… • The last ten years have all been about 3 D hardware • Since 2001 at ATI, joining forces with AMD last year • Optimisation specialist: linking the software and the hardware 2 Tweaking games Understanding the hardware Driver performance Shader code optimisation – (I find assembly language fun) November 13 th 2007 Graphics Performance and Optimisation

Overview • Three basic sections GPU Architecture Efficient Open. GL Practical Optimisation And Debugging • There’s a lot in here Broad overview of all issues I’ve prioritised the biggest issues and the ones most likely to help with Project 3 More details with respect to GPU architecture included as appendix 3 November 13 th 2007 Graphics Performance and Optimisation

GPU Architecture 4 November 13 th 2007 Graphics Performance and Optimisation

Graphics hardware architecture • Parallel computation • All about pipelines • The Open. GL vertex pipeline shown right will be familiar… 5 November 13 th 2007 Graphics Performance and Optimisation

Graphics hardware architecture • Extend the top of the pipeline with some more implementation detail • Ideally, every stage is working simultaneously Application API Video Drivers CPU GPU • Could also decompose to smaller blocks • And eventually to individual hardware pipeline stages As shown last week, the hardware implementation may be considerably more complex than a linear pipeline 6 November 13 th 2007 Graphics Performance and Optimisation Command Buffers Parser Vertex Assembly Vertex Operations Primitive Assembly

Draw commands • Data enters the GPU pipeline via command buffers containing state and draw commands • The draw command is a packet of primitives • Occurs in the context of the current state As set by gl. Enable, gl. Blend. Func, etc. The full set of state is often referred to as a state vector Driver translates API state into hardware state State changes may be pipelined; different parts of the GPU pipeline may be operating with different state vectors (even to the level of per-vertex data such as gl. Color) 7 November 13 th 2007 Graphics Performance and Optimisation

Pipeline performance • The performance of a pipelined system is measured by throughput and latency Can subdivide at any level from the full pipeline down to individual stages • Throughput: the rate at which items enter and exit • Latency: the time taken from entrance to exit Latency is not typically a major issue for API users It is a huge issue for GPU designers Even GPU-local memory reads may be hundreds of cycles Substantial percentage of both design effort and silicon is devoted to latency compensation The system will generally run at full throughput until the latency compensation is exceeded 8 November 13 th 2007 Graphics Performance and Optimisation

Pipeline throughput • Given a particular state vector, each part of the pipeline has its own throughput • The throughput of a system can be no higher than the slowest part: this is a bottleneck More generally, if input is ready but output is not, it is a bottleneck 9 November 13 th 2007 Graphics Performance and Optimisation

Pipeline bottlenecks • Consider system shown right Stage 1 can run at 1 per clock and is 100% utilised Stage 2 can only accept on each other clock; still 100% utilised Stage 3 is therefore starved on half of the cycles it could be working; 50% utilised Although stage 3 has the longest latency, it has no effect on the throughput of the system Items enter 1 per clock Stage 1 Throughput 1/clock Latency 5 cycles Items pass at 1 per clock Stage 2 Throughput 1 per 2 clocks Latency 10 cycles Half throughput, result only every alternate clock Stage 3 Throughput 1/clock Latency 15 cycles Still only alternate clock results, despite per-clock throughput 10 November 13 th 2007 Graphics Performance and Optimisation

Pipeline bottlenecks • A key subtlety; for this to work as shown, there must be load balancing between stages 1 and 2 (probably a FIFO) • Once the FIFO is full, the input buffer will exert backpressure on stage 1 Happens after equilibrium is reached • This pipeline therefore runs at the speed of the slowest part as soon as the FIFO fills Items enter 1 per clock Stage 1 Throughput 1/clock Latency 5 cycles Items pass at 1 per clock; eventually queue Input Buffer Stage 2 Throughput 1 per 2 clocks Latency 10 cycles Half throughput, result only every alternate clock Stage 3 Throughput 1/clock Latency 15 cycles Still only alternate clock results, despite per-clock throughput 11 November 13 th 2007 Graphics Performance and Optimisation

Variable throughput • In general, throughput is data dependent Example: clipping is a complex operation which often isn’t required Example: texture fetch depends on the filtering chosen, which is data dependent • Some pipeline stages require different rates at the input and the output Example: back-face culling; primitive in, no primitive out Example: rasterisation of primitives to fragments; few primitives in, many fragments out • Buffering between stages takes up the slack 12 November 13 th 2007 Graphics Performance and Optimisation

Pipeline bottlenecks • A particular state vector will tend to have a characteristic set of bottlenecks The input data does also have an effect • Small changes to the state vector can make substantial changes to the bottleneck • As a state change filters through the pipeline and for a short period afterwards, bottlenecks shift into the new equilibrium For usual loads, where the render time is much larger than the pipeline depth, this time can be ignored • Can be hard to determine bottlenecks if the states in the pipe are disparate Smearing effect 13 November 13 th 2007 Graphics Performance and Optimisation

Pipeline bottlenecks • There may be multiple bottlenecks if the throughput is not constant at all parts of the pipeline In general it is not constant • GPU buffering absorbs changes in load Measured in tens or hundreds of cycles at best Whole pipeline is thousands of cycles • The bottleneck could be outside the GPU Application, driver, memory management… • Bottleneck analysis is key to hardware performance Not easy: bottlenecks are always present separating expected and unexpected cases is the challenge 14 November 13 th 2007 Graphics Performance and Optimisation

Flushes and synchronisation • Some state cannot be pipelined; a flush occurs Various localities of flush For a whole-pipeline flush, the parser waits before allowing new data into the pipe CPU can carry on building and queuing command buffers Low cost ~ thousands of cycles (~5 us? ) Some operations can require the CPU to wait for the GPU Example: CPU wants to read memory the GPU is writing This is a serialising event Very expensive: wait for pipeline completion, flush all caches, and the restart time taken to build the next command buffer You can force this with gl. Finish: please don’t! 15 November 13 th 2007 Graphics Performance and Optimisation

Asynchronous system Input / Physics thread Runs continuously using input and time (fixed or delta) to update the game world – typically including the scene graph Typical runtime 10 -30 ms Render thread Runs continuously to convert the scene graph to rendering commands Generally cannot start until input/physics thread has processed whole frame The process of rendering a typical game image is massively asynchronous The boxes left show possible asynchronous actors The diagram below shows a possible timeline The shaded areas are the same frame GPU Renderer Runs on its command buffers DAC Loops over the display at 60 -100 Hz A command buffer operation changes the display at end of render; picked up at start of next frame (unless vsync is off) 16 November 13 th 2007 Input Render GPU DAC Graphics Performance and Optimisation

Synchronisation • GPU’s aim to run just under two frames ahead Block at Swap. Buffers if there is another Swap. Buffers in the pipe that is not yet reached • Reading any GPU memory on the CPU causes a sync gl. Read. Pixels is one method, for example; avoid • Writing to GPU memory generally does not The GPU, driver and memory manager work together to do uploads without serialisation No need to be unusually scared of gl. Tex. Image • If you have to lock GPU memory, look for discard or write-only flags that will allow asynchronous access 17 November 13 th 2007 Graphics Performance and Optimisation

Shaders • Texture lookup operations are relatively expensive Competition on GPU or system bus, cost of filtering, unpredictable Some of this is only a latency issue – but latency is not important… – … until the buffering is exceeded – Latency more than doubles for dependent texture operations Prefer ALU math to texture until the function is complex Might replace very small textures with shader constants • Shader – typically its texture operations – likely to be the limiting factor on performance 18 November 13 th 2007 Graphics Performance and Optimisation

Shaders • Each shader is run at a particular frequency Per-vertex, per-fragment now; per-primitive also exists; persample seems likely in the future Can view constants calculated on the CPU as another frequency (per-draw packet) Aim to do calculations at the lowest necessary frequency • Issues to be aware of: Data passed from vertex to fragment shader is interpolated linearly in the space of the primitive (i. e. with perspective correction) so can only use interpolators if this is appropriate (linear or nearly so); high tessellation can be a workaround Excessive use of interpolators can itself be a bottleneck; up to two interpolators per texture fetch, as a ballpark figure 19 November 13 th 2007 Graphics Performance and Optimisation

Shader constants • Shader constants are a large part of the state vector Updating hundreds on each draw call will not be free • Prefer inline constants (known at compile time) to state vector constants Gives the compiler and constant manager more information • For the same reason, avoid parameterising for its own sake • Don’t switch shader just to change a couple of constants 20 November 13 th 2007 Graphics Performance and Optimisation

Efficient Open. GL 21 November 13 th 2007 Graphics Performance and Optimisation

Efficient Open. GL • This is a data processing issue • What data does the GPU need to render a scene? State data, texture data, vertices / primitives • CPU-side performance can easily be dominated by inefficient management of this data • Of them all, vertex data is the most problematic Type of data State Vertex Texture Volume (per frame) Low (~k. B) Med-high (~MB) Very high (~GB) Rate of change Very high Low-med Very low 22 November 13 th 2007 Graphics Performance and Optimisation

Efficient vertex data • Application needs to feed mesh data in somehow • GL provides two basic methods gl. Begin/gl. End (known as ‘immediate mode’) Vertex arrays • Immediate mode is easy to use but has high overheads Many tiny, unaligned copies Non ‘v’ forms imply extra copies Command stream is unpredictable and irregular gl. Begin(GL_TRIANGLE_FAN); gl. Color 4 f(1, 1, 1, 1); gl. Vertex 3 f(0, 0, 0); // position gl. Vertex 3 f(0, 1, 0); // position gl. Color 4 f(1, 0, 0, 1); gl. Vertex 3 f(1, 1, 0); // position gl. Vertex 3 f(1, 0, 0); // position gl. End(); 23 November 13 th 2007 + colour only Graphics Performance and Optimisation

Vertex arrays • Vertex arrays are an alternative The application probably has its data in arrays somewhere, so let GL read them en masse gl. Vertex. Pointer, gl. Color. Pointer, etc. specify the array gl. Draw. Elements to issue a draw command; takes index list Primitives are drawn using the indices into the arrays as set up by the gl*Pointer commands gl. Vertex. Pointer(3, GL_FLOAT, 16, vertex_array); gl. Color. Pointer(4, GL_UNSIGNED_BYTE, 0, color_array); gl. Enable. Client. State(GL_VERTEX_ARRAY ); gl. Enable. Client. State(GL_COLOR_ARRAY ); gl. Draw. Elements(GL_TRIANGLES , 12, GL_UNSIGNED_INT, indices); • Easier for the driver and GPU to handle State vector is instantiated at the gl. Draw. Elements command The GPU can process all the primitives in a single draw packet 24 November 13 th 2007 Graphics Performance and Optimisation

Vertex arrays • Did you hear a but? • The vertex data still belongs to the application Until the gl. Draw. Elements call is entered, the GPU knows nothing of the data After the call completes the app can change the data Therefore, the driver must copy the data on every gl. Draw. Elements call Even if the data never changes – the GL can’t know • Wouldn’t it be great if we could avoid the copy? We don’t supply textures on every call, just upload them to the GPU and let the driver manage them… 25 November 13 th 2007 Graphics Performance and Optimisation

Buffer Objects • This facility is provided with the Vertex Buffer Objects (VBO) extension allows the creation of buffer objects in GPU memory with access mediated by the driver Data can be uploaded at any time with gl. Buffer. Data – As with gl. Tex. Image, done through command buffer to avoid serialisation Bind. Buffer <-> Bind. Texture, Buffer. Data <-> Tex. Image // During program initialisation gl. Bind. Buffer(GL_ARRAY_BUFFER, 0); gl. Buffer. Data(GL_ARRAY_BUFFER, 16*4*sizeof(GL_FLOAT), vertex_array, GL_STATIC_DRAW); . . . // In render loop gl. Bind. Buffer(GL_ARRAY_BUFFER, 0); gl. Vertex. Pointer(3, GL_FLOAT, 16, 0); gl. Enable. Client. State(GL_VERTEX_ARRAY); gl. Draw. Elements(GL_TRIANGLES, 12, GL_UNSIGNED_INT, indices); 26 November 13 th 2007 Graphics Performance and Optimisation

Index data • Draw. Elements only needs to send the indices • Actually we can optimise that away too; Element Arrays allow buffer objects to contain index data Index data is far smaller in volume, and tends to come in larger batches if state changes are minimised, so this can be overoptimisation • Keep batches as large as possible Keep state changes to a minimum Primarily use triangle lists Don’t mess with locality of reference Strips can be marginally more efficient 27 November 13 th 2007 Graphics Performance and Optimisation

Display Lists • These offer the driver opportunity for unlimited optimisation • It’s hard for the driver to do The list can contain literally any GL command • Not recommended for games or other consumer apps • Professional GL apps do make heavy use of display lists (and immediate mode) The effort required to efficiently optimise these is one reason professional GL cards are more expensive 28 November 13 th 2007 Graphics Performance and Optimisation

Visibility optimisations • It’s far more efficient not to render something at all • Try to avoid sending primitives that can’t be seen Not in the view frustum Obscured • Send it, but have it rejected at some early point in the pipeline Cull primitives before rasterisation Reject fragments before shading 29 November 13 th 2007 Graphics Performance and Optimisation

Bounds • Bounding boxes or spheres to reject objects wholly outside the view frustum • Optimal methods for using these were in lecture 11 30 November 13 th 2007 Graphics Performance and Optimisation

Occlusion culling • PVS (Potentially Visible Set) culling For each location in the set of locations, store which other locations might be visible Precalculate before render process starts • If you are standing anywhere in A, you absolutely cannot see C and vice versa View frustum checks cannot solve this part of the problem; consider the position of the observer shown A frustum test is still useful; if the observer was standing in B looking the same way, bounds could cull C • Very effective on room-based games; not so useful on outdoor games Fewer large-scale occluders 31 November 13 th 2007 Graphics Performance and Optimisation A B C

Other visibility methods • Portals – as discussed in lecture 11 • BSP – Binary Space Partition – trees Complex but efficient way to store large static worlds for fast frustum visibility calculations Combine with PVS and portals; all need precalculation phase • Abrash - Graphics Programming Black Book ch. 59 -64, 70 Detailed information on these and other research he and John Carmack did on visibility while developing Quake Still in use today on modern FPS games (with many enhancements!) 32 November 13 th 2007 Graphics Performance and Optimisation

Model LOD • Need to render something, render less of it • Demonstrated two weeks ago: A model close to the camera requires many triangles Carry reduced detail models and select on each render – Like mipmapping, memory cost not prohibitive. – Target sizes near GPU’s high efficiency 100 pixel region • Visualise with wireframe 33 November 13 th 2007 Graphics Performance and Optimisation

Model LOD • Non trivial implementation Popping a well known issue; morph or blend common solutions Must generate reduced detail models Can reuse vertex data, just change indices • Terrain offers particular challenges LOD systems essential for really large worlds Terrain tiles must match between different LODs • Can also solve sampling issues As with undersampling textures, semirandom triangles can be picked; occur if triangles are smaller than 1 pixel 34 November 13 th 2007 Graphics Performance and Optimisation

GPU primitive culling • Degenerate primitives (example: triangles with two indices the same) will be culled at index fetch • A primitive with all vertices outside the same clip plane will be culled • Back-face culling is a simple optimisation and should be used for all closed opaque models • Zero area triangles will be culled before rasterisation This is rarely usefully exploitable • Scissor rectangles cull large parts of primitives during rasterisation 35 November 13 th 2007 Graphics Performance and Optimisation

GPU Z rejection • The Z test can occur before shading Reduces colour read/write load as well • Some states inhibit early Z test Write Z in the shader, obviously Gate Z update in the shader (pixel kill / alpha test with Z write) – Alpha test sounds like an optimisation, but it only saves colour read/write; use it for visual effect not performance – Shader kill acts as a shader conditional • Z unit can reject at hundreds of pixels per clock Accept rate is lower (at the very least Z has to be written) but as fast or faster than any other post-rasteriser operation • Stencil usually rejects at Z rates Having a stencil op that does something implies a stencil write 36 November 13 th 2007 Graphics Performance and Optimisation

Early Z rejection • Draw opaque geometry in roughly front-to-back order Do not work too hard to make this perfect, that’s what the Z buffer was created for in the first place Do not draw the sky first. Please! This assumes you’re bottlenecked in the shader • Consider a Z pass If the fragment shaders are very expensive If at any point rendering the colour buffer you need some algorithm that requires the Z buffer Disable colour writes (gl. Color. Mask) or fill the colour buffer with something cheap but useful (example: ambient lighting) Invariance issues should be rare nowadays (but be aware) 37 November 13 th 2007 Graphics Performance and Optimisation

Shader conditionals • Can also reduce shader load • Treat with care… • Use mostly for high coherency data the conditional is unlikely to have per-pixel granularity An if-then-else clause can have to execute both branches • For low coherency data, prefer conditional move type operations • Typically the shader compiler and optimiser can’t know much about the likely coherency So it guesses 38 November 13 th 2007 Graphics Performance and Optimisation

Triangle sizes • Larger triangles are more efficient than small ones • Rules of thumb: Over 1000 pixels is large 100 pixel triangles are considered typical and the GPU should be into the ballpark of its peak performance Under 25 pixel triangles are small Tiny triangles likely to cause granularity losses in the GPU • Often the type of object and size of triangle are related Example: world triangles tend to be larger than entities 39 November 13 th 2007 Graphics Performance and Optimisation

Bump mapping • Can trade off geometric complexity for more expensive fragment shading Textures in general offer this capability – light maps are an earlier example • Having a normal map available in the fragment shader useful for other reasons too Per-pixel lighting is an obvious use • Doom 3 an early pioneer: polygon counts are low compared with other games of the time bump mapping makes it hard to see except on silhouette edges 40 November 13 th 2007 Graphics Performance and Optimisation

Practical Optimisation and Debugging 41 November 13 th 2007 Graphics Performance and Optimisation

Optimising Applications • Always profile; never assume • Target optimisations Better to get a small gain in something that takes half the time than a big gain in something that takes a couple of percent Better to do easy things than hard things “Low-hanging fruit” 42 November 13 th 2007 Graphics Performance and Optimisation

Instrumentation for debugging • Logging • Visualisation: make particular rendering (more) visible • Simple interfaces into the high-level parts of the program to make low-level testing easier ‘God mode’ Skip to level N or subpart of the level – Saved games may seem to be an answer here, but minor changes during development usually break them Metadata display • Multiple monitors and remote debugging Key for fullscreen applications Useful to have ‘stable’ dev machine and separate debug target 43 November 13 th 2007 Graphics Performance and Optimisation

Instrumentation for performance • Feedback on what the performance actually is A simple onscreen frames per second (FPS) and/or timeper-frame counter Special benchmarking modes • Modify the performance Skip particular rendering passes Add known extra load – Examples: new entities, particle system load, force postprocessing effects on 44 November 13 th 2007 Graphics Performance and Optimisation

Real-world example: Doom 3 engine Heavily instrumented with developer console accessed with ctrl-alt Most commands prefixed according to their functional unit – r_ commands are to the renderer, s_ the sound system, sv_ the server, g_ the client (game), etc. Record demos; playback with playdemo or timedemo Capture individual frames with demoshot for debugging or performance Can also send console commands from the command line – essential for external tools Many debugging commands – noclip to fly anywhere on a level – r_showshadows 1 displays the shadow volumes – g_show. PVS 1 to show the PVS regions at work 45 November 13 th 2007 Graphics Performance and Optimisation

More Doom 3 convenience features PAK files are just ZIP files You can look at the ARB_fragment_program shaders Doom 3 uses (glprogs/ directory in the first pakfile). You can also modify them: real files (e. g. under the base/glprogs directory) override the pakfiles • Human-readable configuration files • TAB completion on the console Long commands not a problem – plus you can find the command you want! • Key bindings 46 November 13 th 2007 Graphics Performance and Optimisation

Doom 3 render: multipass process 1. Z pass: set the Z buffer for the frame 2. Lighting passes: for each light in the scene 2 A. Shadow pass: render shadow volumes into the stencil buffer 2 B. Interaction pass: accumulate the contribution from this light to the framebuffer. - Cheap Phong algorithm (per-pixel lighting with interpolated E; Prey calculates E on a per-pixel basis for better specular) Vertex/fragment shader pair 3. Effects rendering; mostly blended geometry for explosions, smoke, decals, etc. 4. One or more postprocessing phases for refraction and other screen-space effects 47 November 13 th 2007 Graphics Performance and Optimisation

Doom 3 benchmarking tools • Each render pass can be disabled from the console r_skipinteractions, r_shadows, r_skippostprocess Benchmark each pass individually Worth considering render time rather than just FPS; linear quantity 48 Rendered FPS Frame time (ms) Everything 55. 8 17. 9 - postproc 58. 5 - interactions - shadows November 13 th 2007 Isolated pass Pass time (ms) Pass load 17. 1 Postproc 0. 8 4% 104. 7 9. 6 Interaction 7. 5 42% 174. 5 5. 7 Shadows 3. 9 22% The rest 5. 7 32% Graphics Performance and Optimisation

Case study: Doom 3 interaction shader • The shader has 7 texture lookups Texture limited on most GPUs One of them was a simple function texture – Probably originally a point of customisation but unused We tested gain by eliminating the lookup – replaced with a constant – note not 0 or 1, which might allow the optimiser to eliminate other code – Provided the expected ~15% gain for the pass Replaced with a couple of scalar ALU instructions – Gain was still the same, as the scalar ALU scheduled into gaps in the existing shader • Quake 4 and later games all picked up the change 49 November 13 th 2007 Graphics Performance and Optimisation

Instrumenting applications • Be wary of profiling API calls Asynchronous system; Swap. Buffers is probably the only point of synchronisation Can’t easily measure hardware performance at a finer granularity than a frame Don’t try to profile the cost of rendering a mesh by timing Draw. Elements; only measures the time taken to validate state and fill the command buffer Which isn’t to say that’s never useful information 50 November 13 th 2007 Graphics Performance and Optimisation

Instrumenting applications • Don’t overprofile Query. Performance. Counter has a cost Even RDTSC does • Try to look at the high level and in broad terms first 30% physics, 20% walking scene graph, 30% in the driver, 20% waiting for end of frame Rather than 15. 26% inside Draw. Elements • Aim to be GPU limited, then optimise GPU workload Don’t waste time optimising CPU code if it’s waiting for the GPU Iterate as the GPU workload becomes more optimal • Try to avoid compromising readability for performance Rarely necessary Download the Quake 3 source to see how clear really fast code can be The games industry is really, incredibly, bad at this. 51 November 13 th 2007 Graphics Performance and Optimisation

Benchmark modes • Timed runs on repeatable scenes Two options – Fix the number and exact content of frames and time the run (could be one frame repeated N times) – Fix the run time, render frames as fast as possible, count the frames Former is more repeatable; often essential if tools require multiple runs to accumulate data Latter more convenient for benchmarkers and more realistic to how games behave in the real world Cynical reason for benchmarks: applications get more attention from press (and hence driver developers) 52 November 13 th 2007 Graphics Performance and Optimisation

Code. Analyst • Code. Analyst is an AMD tool that allows non-intrusive profiling of the application’s CPU usage A profiling session spawns the application under test – Make sure to avoid profiling startup and shutdown time Can drill down to individual source lines in your code and show you the cost Many examples on AMD’s web site using this Useful for all CPU-limited applications 53 November 13 th 2007 Graphics Performance and Optimisation

Code. Analyst hints A spike inside driver components may not be driver overhead The driver is probably waiting on the GPU to meet the Swap. Buffers limit If there's not a large spike in the driver, it's probably the application that's the limit. This is complicated by the fact that we may choose to block if the GPU is not yet ready, so time may move from the driver to being reported as 'system idle process‘, PID 0, or similar. Vary the resolution and check the how the traces change If the relative time in the driver or system idle doesn't change, the application is not pixel limited. Multicore systems make interpreting the results harder. You might be best off switching a core off if you can 54 November 13 th 2007 Graphics Performance and Optimisation

GPUPerf. Studio • Lets you look inside the GPU • Hardware performance counters 3 D busy is the most obvious and often important Vertex / pixel load can also be seen • The bad news: GL support is not in the currently downloadable version 1. 1. Coming soon… 55 November 13 th 2007 Graphics Performance and Optimisation

Shader Development • AMD GPUShader. Analyzer Available to download from AMD web site Will handle all GL shader types (GLSL, ARB_fp, ARB_vp) Good development environment; no need to run your app to compile Will show output code and statistics including estimated cycle counts for all AMD GPUs 56 November 13 th 2007 Graphics Performance and Optimisation

Scalability • Look to create consistent performance Better to run at 30 fps continuously than oscillate wildly between 15 fps and 100 fps. Target worst-case scenes You will need headroom to guarantee 60 fps • Is a particular gain useful? A 4% speedup won’t help anyone play your game Five 4% speedups would, though Gains in a lesser component allow more use of that component 57 November 13 th 2007 Graphics Performance and Optimisation

Scalability • PC environment is a huge scalability challenge Matrix of CPUs, GPUs and render resolutions is huge Performance is in tension with image quality Adjust quality to scale for GPU power and set higher loads – when CPU limited, more pixels probably have no cost Adjust quality in profiling – Resolution (or clock) scaling to test if CPU or GPU limited • Consoles have it easier: more fixed in every way Still need headroom, just less of it Now have resolution scaling issues - five TV resolutions in NTSC 480 i, PAL 576 i, 720 p, 1080 i/p 60 Hz / 50 Hz is a headache here 58 November 13 th 2007 Graphics Performance and Optimisation

Caveats on optimisation • Windowed mode GPUs can behave differently in windowed mode to fullscreen mode Windowed should still be your primary development mode unless you have remote debugging • Front Buffer rendering May be useful for debugging, but could have similar performance implications • Avoid misusing benchmarks Repeat runs – make sure everything’s ‘warm’. 59 November 13 th 2007 Graphics Performance and Optimisation

Guidelines for Project 3 • Concentrate on the scene graph first, GPU second, CPU cycle picking last Look for algorithms that cull monsters, trees and rooms rather than triangles or pixels • Work on model or texture data in the GPU, not CPU Primarily, use the shader to do the work Anywhere index data, primitive count and connectivity don’t change is a candidate If you have to generate a texture consider using the GPU 60 November 13 th 2007 Graphics Performance and Optimisation

Guidelines for Project 3 • Short of time to write shaders Write a few shaders that you use a lot • Don’t try to do everything in this lecture Many techniques won’t apply to your specific case Even those that do often won’t matter Profile-Guided Optimisation! 61 November 13 th 2007 Graphics Performance and Optimisation

Headline performance items Scene graph optimisations: visibility culling, model LOD Don’t touch model data on the CPU unless the algorithm absolutely requires it Use vertex arrays for complex mesh data (> 10 primitives); store static data in VBOs. Use mipmaps for all static textures; avoid undersampling textures without mipmaps Render roughly front to back; don’t kill yourself trying but give it a go for the largest geometry; draw the sky last! Use compressed textures by default; only disable if artifacts appear Disable unnecessary alpha testing; don’t do kills in shaders unless you have to Move work from fragment to vertex shaders where possible Prefer moderate math to texture lookups particularly if they increase the dependent fetch level 62 November 13 th 2007 Graphics Performance and Optimisation

Further reading • Abrash, Mike: The Graphics Programming Black Book Even in 1997 the asm and register programming section was dated Much of the Quake documentation isn’t – Clear explanation of BSP, PVS and some on portals. Rest is still worth reading to show the mindset – Skip asm-specific bits, concentrate on thought process Chapter 1 and chapter 70 are required reading Stencil shadows; the Wikipedia page has many links 63 November 13 th 2007 Graphics Performance and Optimisation

Samples and Tools http: //ati. amd. com/developer/ GPUPerfstudio, GPUShader. Analyzer and the Compressonator Tootle is also interesting; optimise meshes both for vertex cache and ‘internal’ front-to-backness Many other samples, documents and tools http: //www. amd. com/codeanalyst 64 November 13 th 2007 Graphics Performance and Optimisation

Questions • If we have time… 65 November 13 th 2007 Graphics Performance and Optimisation

Appendix Background information on more aspects of the GPU a. k. a. “The slides I knew I didn’t have time to go through” 66 November 13 th 2007 Graphics Performance and Optimisation

Texture and rendertarget tiling • Memory interface efficiency mostly determined by burst sizes The more useful memory fetched in one go the better Avoid fetching anything that isn’t then used – This is why mipmapping is so important: minifying a texture implies fetching memory that isn’t then used • Rearranging memory into tiles increases locality of reference 64 bytes might contain 4 x 4 pixels instead of 16 x 1 pixel Format is transparent to application 67 November 13 th 2007 Graphics Performance and Optimisation

Texture Compression • GL_ARB_texture_compression • The S 3 TC / DXTC / BC algorithm is a high quality method for typical image textures Designed such that the artifacts introduced in lossy compression tend to be smoothed out by texture filtering Function textures and unusual use textures may not meet acceptable quality Rearranging components can help Use high-quality compressors - The Compressonator • Compression isn’t just about memory bandwidth Reduces effective latency (one fetch brings in more useful texels) Effectively increases texture cache size 68 November 13 th 2007 Graphics Performance and Optimisation

Texture Filtering • A bilinear-filtered sample is the common basic unit of work for a texture unit Unlikely that point sampling is any faster than bilinear; can make this work for you in image processing shaders (rather than point sampling and doing some constant weighted sum) Each additional bilinear sample for trilinear or anisotropic filtering is probably consuming additional time • Smart algorithms ensure that only needed samples are taken No need for trilinear if magnifying No need for anisotropy if square-on Example: walls tend to have less anisotropy than floors • Gradient calculations may be dynamic Necessary to handle dependent texture reads Be wary with dependency; gradient can be unpredictable 69 November 13 th 2007 Graphics Performance and Optimisation

Render to texture • Useful for generating extra views or postprocessing Example: mirror in driving game Example: postprocessing for refraction • gl. Copy. Tex. Image copies the framebuffer to a texture CPU-GPU serialisation is not implied; this can probably be queued into a command buffer • Other methods exist such as pbuffers and framebuffer extensions Can be slightly more efficient Can return to a rendertarget after rendering on another More complex; don’t use without good reason 70 November 13 th 2007 Graphics Performance and Optimisation

Multisample antialiasing • The key gain is to run the fragment shader at pixel frequency rather than sample frequency • Also saves memory bandwidth; can compress Z and colour The buffer may need to be resolved to an uncompressed buffer for display or if used as a texture Triangle size may be worth extra consideration with MSAA; frame buffer and Z compression rate is likely to be in roughly inverse proportion to the number of visible edges in the scene 71 November 13 th 2007 Graphics Performance and Optimisation

Caching • Many caches inside the GPU • Different to what you might be familiar with in a CPU More about memory bursts and latency compensation than reuse In general you do need to hit the memory – Example: texture mapping the whole framebuffer at 1: 1; every pixel and texel will be touched exactly once Therefore, be pessimistic: assume this Choose to compensate memory latency with large buffers – Rather than using the cache to dodge the accesses • In a few places short-term ‘reuse’ is critical Bilinear filtering the most obvious case 72 November 13 th 2007 Graphics Performance and Optimisation

Caching • Can still be advantages to avoiding cycling Used to be a big thing, particularly in the days of visible caching (software controlled rather than auto) Caused sorting policy of hard sort by material Nowadays far less important, hence rough sort by depth In some pathological circumstances sort by shader and depth (or a Z pass followed by sort by shader) might be more efficient 73 November 13 th 2007 Graphics Performance and Optimisation

Disclaimer & Attribution 74 • • DISCLAIMER The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions and typographical errors. • AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION. • AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE LIABLE TO ANY PERSON FOR ANY DIRECT, INDIRECT, SPECIAL OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES. • • ATTRIBUTION © 2007 Advanced Micro Devices, Inc. All rights reserved. AMD, the AMD Arrow logo and combinations thereof are trademarks of Advanced Micro Devices, Inc. Other names are for informational purposes only and may be trademarks of their respective owners. November 13 th 2007 Graphics Performance and Optimisation