Framebuffer Compression Using Dynamic Color Palettes DCP Ayub

Framebuffer Compression Using Dynamic Color Palettes (DCP) Ayub A. Gubran Tor M. Aamodt

Motivation: framebuffer life cycle Application 1 Application 2 Application 3 OS Graphics Libraries / Graphic API calls Refresh rates vary by application and user activity Legends App Level OS Level HW Level Device Screen CPU (Software Rendering) / GPU (Hardware Rendering) @ screen refresh rate (i. e. , 60 FPS) Render Framebuffer 1 Recompose after every framebuffer update Render Framebuffer 2 Render Framebuffer 3 Display Controller System Compositor (e. g. , Android’s Surface. Flinger) GPU or CPU Composition Display framebuffer Framebuffer Compression Using Dynamic Color Palettes 2

Motivation: The cost of accessing the framebuffer • Every frame is consumed/processed multiple times in multiple stages. • Energy cost: off-chip memory. Framebuffer Compression Using Dynamic Color Palettes 3

Motivation: Mobile Applications Usage Time Spent on i. OS and Android Connected Devices 1 Gaming 32% UI Apps [PERCENTAGE] 1 - Flurry Analytics: It’s an App World. The Web Just Lives in It. Framebuffer Compression Using Dynamic Color Palettes 4

Framebuffer Compression for UI Apps • Most of the time is spent on UI apps with simple content (solid backgrounds, text, icons…). • Simple content can be compressed well using color palettes (dictionaries). • But how we can accurately predict the right color palette for each frame before we start rendering it. Framebuffer Compression Using Dynamic Color Palettes 5

Observation: Inter-frame Temporal Coherence Pixels color change (new content) Per pixel change (moving content) • Web browsing (Chrome) Scrolling New web search Loading BBC news % of screen size 100% Loading Amazon 80% 60% 40% 20% 1 101 201 301 401 501 601 701 801 901 1001 1101 1201 1301 1401 1501 1601 1701 1801 1901 2001 2101 2201 2301 0% Mostly the same content Time (frames) Framebuffer Compression Using Dynamic Color Palettes 6

Dynamic Color Palettes (DCP) Framebuffer Compression Using Dynamic Color Palettes 7

Compression Steps FVC: Frequent Values Collector Construct FVC from current frame’s pixel values CCD: Common Colors Dictionary Frame 1 Frame 0 FVC CCD FVC Frame 2 CCD FVC To memory Construct the CCD dictionary using the FVC values To memory Framebuffer access compressed using the CCD Time Framebuffer Compression Using Dynamic Color Palettes 8

Associative search. Evict & replace on miss. Size 16 -128 Compression Pipeline 2 a Send Tile FVC (constructs a CCD for the next frame) Pixel tiles CCD (Compress current 1 tile) 2 b Lookup table GPU Write a compressed tile Yes Can compress all pixels in the current tile? No Write a noncompressed tile 3 Write to memory 4 9 Buffer compressed & non-compressed tiles of block Framebuffer Compression Using Dynamic Color Palettes 9

DCP Variations • Improving compression rates: § DCP: A fixed size color palette. § Variable DCP: Varying the color palette size every frame using the feedback from the previous frame. § Adaptive DCP: Encoding tiles based on the frequencies of their colors. • Reducing the cost of DCP: • Lower sampling rates: Only sending every nth pixel to the FVC to construct the palette. • Using the same palette for multiple frames. Framebuffer Compression Using Dynamic Color Palettes 10

Results Compression Rate Cost: using 64 FVC and CCD FVC: 456 bytes CCD: 264 bytes ADCP 32 RED: Locally compressing similar colors in a tile 2 RAS: A more general prediction based compression 3 4. 13: 1 RAS 2. 5: 1 VDCP 3. 5: 1 RED 2. 62: 1 DCP 2. 86: 1 16 RAS DCP ADCP RED VDCP 8 4 RAS 2 1 AB PH CH FB HO KD MS ST TW UI HM UI and 2 D applications Framebuffer Compression Using Dynamic Color Palettes TR GS NS FN 3 D games 3 D HM 11

Limitations and Potential • 3 D games: Complex color patterns. Color palettes perform poorly. • Multiple render targets: Need to construct a palette for every target but this is uncommon in UI apps. • Compressing non-color render targets in 3 D games: We see potential in compressing some non-color targets. Framebuffer Compression Using Dynamic Color Palettes 12

Thank you • Questions? Framebuffer Compression Using Dynamic Color Palettes 13

References 1 - Flurry Analytics: It’s an App World. The Web Just Lives in It. 2 - NVIDIA, 2015. NVIDIA Tegra X 1 Whitepaper. URL: international. download. nvidia. com/pdf/tegra/Tegra-X 1 -whitepaperv 1. 0. pdf 3 - RASMUSSON, J. , HASSELGREN, J. , AND AKENINE-MOLLER, T. 2007. Exact and error-bounded approximate color buffer compression and decompression. In SIGGRAPH/EUROGRAPHICS Conference On Graphics Hardware: Proceedings of the 22 nd ACM SIGGRAPH/EUROGRAPHICS symposium on Graphics hardware, vol. 4, 41– 48. Framebuffer Compression Using Dynamic Color Palettes 14

Backup slides Framebuffer Compression Using Dynamic Color Palettes 15

Workloads • • • • AB: Angry Birds. PH: Android Phone. CH: Chrome. FB: Facebook. HO: Android Home Screen. KD: Amazon Kindle. MS: Android Messaging. ST: Android Settings. TW: Twitter. TR: Temple Run 2. GS: Gun Ship 2. NS: Need for Speed. FN: Fruit Ninja. Framebuffer Compression Using Dynamic Color Palettes 16

Lower pixel sampling at the FVC Compression rate relative to 1: 1 sampling 1. 2 1. 1 1: 4096 1: 32 1: 8192 1: 256 1: 16384 1: 1024 1 0. 9 0. 8 0. 7 0. 6 AB PH CH FB HO KD MS ST TW HM

Using the same palette for multiple frames Compression rate relative to sampling every frame 1 0. 9 0. 8 0. 7 0. 6 0. 5 AB 2 PH CH 4 FB HO 8 KD MS 16 ST TW 32 HM

ADCP Example CCD C 0 C 1 C 2 C 3 C 4 C 5 C 6 C 7 000 001 010 011 100 101 110 111 Tile colors Encoding CSB {C 0, C 2} {C 0, C 1} {C 0, C 0} {C 0, C 5} {C 6, C 2} {C 6, C 7} {C 0, Cy} {Cx, C 2} {00, 01} {0, 1} {φ} {000, 101} {110, 010} {110, 111} {C 0, Cy} {Cx, C 2} 010 001 000 011 011 111 Framebuffer Compression Using Dynamic Color Palettes 19

FVC Fidelity vs size 0. 9 0. 8 0. 7 0. 6 TW ST S M 64 128 EA N 32 KD HO FB CH 16 HM 8 PH 0. 5 AB FVC relative coverage 1