Windows Graphics Overview David Blythe Architect Windows Graphics

Windows Graphics Overview David Blythe Architect Windows Graphics & Gaming Technologies dblythe @ microsoft. com Microsoft Corporation

Session Outline Windows Graphics Big Picture Windows codenamed "Longhorn" Display Driver Model Basic, Advanced driver models Graphics APIs Legacy, Direct 3 D 9, Windows Graphics Foundation, Open. GL Desktop Window Manager Content Protection & Protected Video Path Glitch-Free Media Support Windows Graphics Foundation 2. 0

Windows Graphics in Longhorn Avalon/DWM WGF 2. 0 Legacy D 3 D APIs DX VA D 3 D 9 Common pipeline WGF 1. 0 Microsoftwritten code IHV-written code Open. GL 32 OGL→D 3 D Open. GL ICD User-Mode Driver User Mode Kernel Mode DXG Kernel-Mode Driver

Longhorn Display Driver Model Graphics Just Works Fundamental principle New driver model delivers: Stability Security Availability (virtualization) Performance Staged introduction of driver model Basic model supports current hardware (planned for Longhorn) e. g. , pre-Longhorn hardware Advanced model utilizes new hardware features (post-Longhorn) More efficient virtualization Old XP driver model still supported for compatibility No new features

Stability & Security Application graphics state isolated from other applications Surface data (textures, render targets, etc) Virtual address spaces Current rendering state Shaders, etc Command stream is validated Prevents access to unauthorized resources e. g. , another application’s surfaces Prevents illegal command streams Avoid sending bad commands to hardware But, still have a watchdog timer Time out if not making forward progress Basic driver model does sw validation in kernel Advanced driver model does hw validation

Virtualization Allow multiple applications to “time slice” the graphics hardware Scheduler determines which command stream to run Basic driver model schedules batches Batch = sequence of commands submitted by application Batch can’t be interrupted once started Advanced driver model schedules contexts Batches accumulated in application context Can interrupt a context and run a different one More responsive to input changes (new work, priority changes) Better user experience

Virtualization Memory manager determines which memory resources need to be resident Unused resources can be paged out Application resources can be larger than video memory In Window Graphics Foundation APIs only No duplication of data Either in graphics memory, system memory, or page file Basic driver model All referenced surfaces must be present for application to run Can fail if too many surfaces (WGF 2. 0 can be demanding) Pages for inactive surfaces can be paged out Advanced driver model Only referenced pages need to be resident Can fault pages in on demand More efficient utilization of physical memory

Device Lost and Device Removed Virtualization eliminates “device lost” Eliminate notion of exclusive access to device However, device can be removed Low frequency, but possible Hot-plug “Device hang” followed by reset equivalent to: Removal + insertion Best to not have device hang at all (hint) Use to avoid reboot on driver install Remove device, add device with new driver Applications should be written to handle “device removed” events

Performance New model designed with performance in mind Hardware command streams built directly in user mode New DDIs better match for modern hardware Support for renaming and other pipelining operations Both Basic and Advanced driver models Advanced model enables maximum performance Eliminates CPU processing Validation Handle mapping, patching

Advanced Driver Model is the Future More responsive scheduling More efficient memory management All future content protection, glitch free, etc. plans build on the Advanced Driver Model

Legacy APIs Avalon/DWM WGF 2. 0 Legacy D 3 D APIs DX VA D 3 D 9 Common pipeline WGF 1. 0 Microsoftwritten code IHV-written code Open. GL 32 OGL→D 3 D Open. GL ICD User-Mode Driver User Mode Kernel Mode DXG Kernel-Mode Driver

Legacy APIs Older Direct 3 D immediate mode APIs supported D 3 DRM (retained mode) API retired Map older runtimes onto Direct 3 D 9 DDIs e. g. , runtimes ≤ Direct 3 D 8 Fixed-function onto shaders, etc Direct 3 D 9 interface matches Direct. X 9. 0 c Device removed maps to device lost Memory virtualization “partially hidden” from Direct 3 D 9 Application limited to size of video memory Avoids breaking existing applications Port to WGF 1. 0 for full virtualization

Windows Graphics Foundation 1. 0 Avalon/DWM WGF 2. 0 Legacy D 3 D APIs DX VA D 3 D 9 Common pipeline WGF 1. 0 Microsoftwritten code IHV-written code Open. GL 32 OGL→D 3 D Open. GL ICD User-Mode Driver User Mode Kernel Mode DXG Kernel-Mode Driver

Windows Graphics Foundation 1. 0 Enhanced version of Direct 3 D 9 Also known as “DX 9. L” Cross-process shared surfaces “Unlimited memory” All memory resources are “managed” Creation fails when run out of pagefile Resource management controls Prioritization of resources Antialiased text rendering support Monochrome texture filter with large kernel size More accurate gamma control Extended, higher resolution gamma tables Still support legacy 256 x 3 table for GDI No more “device lost”

Open. GL Avalon/DWM WGF 2. 0 Legacy D 3 D APIs DX VA D 3 D 9 Common pipeline WGF 1. 0 Microsoftwritten code IHV-written code Open. GL 32 OGL→D 3 D Open. GL ICD User-Mode Driver User Mode Kernel Mode DXG Kernel-Mode Driver

Open. GL Continue Open. GL support Important component of the Windows platform, particularly for high-end workstation applications In-box hardware accelerated version of Open. GL Uses WGF 1. 0 as common graphics pipeline Upgraded API support compared to XP Open. GL 1. 4 Continue to support Open. GL ICDs Can be shipped by OEMs New ICDs required for LDDM Old XP ICDs work with XP driver model in Longhorn

Desktop Window Manager New desktop presentation built using Avalon and Direct. X Uses new WGF 1. 0 interfaces Desktop Window Manager (DWM) Composited desktop Window movement without repainting underlying windows No more dragging garbage Client areas from applications GDI, Direct 3 D, Windows Graphics Foundation, Avalon Video Non-client areas from the Window Manager Allows different views of application windows

Desktop Window Manager Supports high-dpi displays Composition with magnification Supports video playback Takes advantage of Direct 3 D 9 shaders For non-client areas, transitions Animated transitions, etc. New WGF 1. 0 cross-process shared surfaces Allows DWM to access application back buffers Turns off when certain application types running Overlay planes, front buffer rendering, …

Desktop Window Manager Avalon Application WGF/DX Application GDI Application surface Desktop Window Manager

Full-Screen Exclusive, Multimon, and DWM So, what happens to full-screen exclusive? Replaced with exclusive output control Full-screen application owns the display output Video mode, gamma, etc. Desktop window manager disappears Other applications can continue to run Doesn’t trigger device lost, etc. May bump the priorities of the output-exclusive app Applications should recognize case where client area is occluded Background processing on the GPU Multiple monitors independent Composited desktop image per monitor Exclusive output independent for each monitor

Content Protection Avalon/DWM WGF 2. 0 Legacy D 3 D APIs DX VA D 3 D 9 Common pipeline WGF 1. 0 Microsoftwritten code IHV-written code Open. GL 32 OGL→D 3 D Open. GL ICD User-Mode Driver User Mode Kernel Mode DXG Kernel-Mode Driver

Content Protection Preparing the PC for HD-DVD Prevent “high-value” content from escaping Work with content providers on protection rules Two areas of protection While content being processed on CPU While content being transmitted to GPU and processed on GPU Supported by Protected Media Path (PMP) and Protected Video Path (PVP)

Protected Video Path Protection requirements Ensure that graphics device is trusted Driver authenticates graphics chip, OS authenticates driver Protect content transmitted to graphics card Special case for data crossing a “user-accessible bus” (UAB) Encrypt content crossing UABs with an approved cipher E. g. , AES 128 in counter mode Encryption requires initialization using Diffie-Hellman algorithm Protect content transmitted from graphics card to display device Use “standard” output protection mechanisms (HDCP, etc) Control using Output Protection Manager (OPM)

Protected Video Path Roadmap Non-UAB systems planned for Longhorn: Small set of driver DDIs Driver authentication OPM Updates to Direct. X Video Acceleration (DXVA) Authentication of cross-process shared surfaces Basic driver model UAB systems require Advanced driver model Support for paging with encrypted transfers New DDIs Initialization, encrypted transfers, key management, paging

Glitch-Free Media Support Avalon/DWM WGF 2. 0 Legacy D 3 D APIs DX VA D 3 D 9 Common pipeline WGF 1. 0 Microsoftwritten code IHV-written code Open. GL 32 OGL→D 3 D Open. GL ICD User-Mode Driver User Mode Kernel Mode DXG Kernel-Mode Driver

Glitch-Free Media Support Multi-faceted plans to address causes of glitches Several graphics-related initiatives Memory, scheduler controls Statistics & feedback Video sync & flip counts with timestamps Method to calibrate GPU timestamps against CPU timestamps Compute current frame being displayed Hardware “flip” buffering Hardware-supported “flip queue” Allow playback to get several frames ahead Provide elasticity to cover unexpected delays Hardware support eliminates all software delays Comes at “modest” increase in latency Comes at a cost in additional memory

Windows Graphics Foundation 2. 0 Avalon/DWM WGF 2. 0 Legacy D 3 D APIs DX VA D 3 D 9 Common pipeline WGF 1. 0 Microsoftwritten code IHV-written code Open. GL 32 OGL→D 3 D Open. GL ICD User-Mode Driver User Mode Kernel Mode DXG Kernel-Mode Driver

Windows Graphics Foundation 2. 0 Next Version of Direct 3 D Supported in Longhorn Proposed Gold Logo requirement Major improvements in Hardware consistency Clearest, most detailed specification Elimination of capability bits Programmer expressiveness Unified shader programming model More capable shader unit Flexible memory model New pipeline stages LDDM + WGF 2. 0 are fundamental improvement to the Windows Gaming Platform

Windows Graphics Foundation 2. 0 Performance improvements Applications limited by state change performance LDDM helps by eliminating extra processing WGF 2. 0 refactors state for efficient management Enable GPU processing without CPU intervention More flexible memory model Stream output from middle of pipeline Feedback to front of pipeline, with predication Efficiency improvements Multiple samples from textures, integer addressing Compact surface formats (scenario-specific) HDR, normal map compression, … Arrayed resources

Windows Graphics Foundation 2. 0 Geometry shader stage “Sees” entire primitive (3 vertices of triangle) Can have adjacent vertices too (6 vertices total) Limited amplification Extrude edges, expand points, generate shells, … Per-primitive processing Generate extra per-primitive constant data for pixel shaders Constant colors, normals, etc Compute plane equations, barycentric parameters, etc. Combine with stream output or arrayed resources Render to cube map Render multiple shadow maps

Windows Graphics Foundation 2. 0 fixed programmable memory Input Assembler Constant Vertex Shader Geometry Shader Sampler Vertex Buffer Index Buffer Texture Sampler Texture Memory Constant Setup Rasterizer Stream out Stream Buffer Pixel Shader Output Merger Sampler Texture Depth Render Stencil Target

Call To Action Longhorn Display Driver Basic model for Longhorn launch Future Advanced model Support Windows Graphics Foundation 2. 0 Comprehensive hardware spec for next generation content Support Protected Video Path features Output protection management, for Longhorn launch Protected transfers (discrete cards) for validation in 2006 Support Glitch Free features Counters, timestamps, flip queue ASAP Advanced driver model

Community Resources Windows Hardware & Driver Central (WHDC) www. microsoft. com/whdc/default. mspx Technical Communities www. microsoft. com/communities/products/default. mspx Non-Microsoft Community Sites www. microsoft. com/communities/related/default. mspx Microsoft Public Newsgroups www. microsoft. com/communities/newsgroups Technical Chats and Webcasts www. microsoft. com/communities/chats/default. mspx www. microsoft. com/webcasts Microsoft Blogs www. microsoft. com/communities/blogs

Additional Resources Email directx @ microsoft. com Related Sessions Avalon Technologies “Advances in Display and Composition Architecture for Windows” PMP and PVP “Protected Media Path & Driver Interoperability Requirements” “Longhorn Output Content Protection” WAVE “Windows Audio Video Excellence”