Exploiting the ProduceConsume Relationship in DMA to Improve

Exploiting the Produce-Consume Relationship in DMA to Improve I/O Performance Institute of Computing Technology, Chinese Academy of Sciences 2009. 2. 15 Workshop on The Influence of I/O on Microprocessor Architecture (IOM-2009) INSTITUTE OF COMPUTING TECHNOLOGY Dan Tang, Yungang Bao, Yunji Chen, Weiwu Hu, Mingyu Chen

An Brief Intro Of ICT, CAS INSTITUTE OF COMPUTING TECHNOLOGY ICT has developed the Loongson CPU ICT has built the Fastest HPC in China – Dawning 5000, which is 233. 5 TFlops and rank 10 th in Top 500.

Overview l l l Background Nature of DMA Mechanism DMA Cache Scheme Research Methodology Evaluations Conclusions and Ongoing Work INSTITUTE OF COMPUTING TECHNOLOGY

Importance of I/O operations l I/O are ubiquitous l l l INSTITUTE OF COMPUTING TECHNOLOGY Load binary files：Disk Memory Brower web, media stream：Network Memory … I/O are important l Many commercial applications are I/O intensive： l Database, Internet applications etc.

State-of-the-Art I/O Technologies l I/O Bues: 20 GB/s l l PCI-Express 2. 0 Hyper. Transport 3. 0 Quick. Path Interconnect I/O Devices l l RAID: 400 MB/s 10 GE: 1. 25 GB/s INSTITUTE OF COMPUTING TECHNOLOGY

A Typical Computer Architecture INSTITUTE OF COMPUTING TECHNOLOGY NIC

Direct Memory Access (DMA) INSTITUTE OF COMPUTING TECHNOLOGY l DMA is an essential feature of I/O operation in all modern computers l DMA allows I/O subsystems to access system memory for reading and/or writing independently of CPU. l Many I/O devices use DMA l Including disk drive controllers, graphics cards, network cards, sound cards and GPUs

Overview l l l Background Nature of DMA Mechanism DMA Cache Scheme Research Methodology Evaluations Conclusions and Ongoing Work INSTITUTE OF COMPUTING TECHNOLOGY

DMA in Computer Architecture INSTITUTE OF COMPUTING TECHNOLOGY NIC

An Example of Disk Read: DMA Receiving Operation INSTITUTE OF COMPUTING TECHNOLOGY CPU ① Descriptor Driver Buffer ④ Kernel Buffer ② ③ Memory ⑤ DMA Engine User Buffer • Cache Access Latency： ~20 Cycles • Memory Access Latency：~200 Cycles

Potential Improvement of DMA INSTITUTE OF COMPUTING TECHNOLOGY CPU ① Descriptor Driver Buffer ④ Kernel Buffer ② ③ Memory ⑤ DMA Engine User Buffer • This is a typical Shared-Cache Scheme

Problems of Shared-Cache Scheme l l l Cache Pollution Cache Thrashing Degrade performance when DMA requests are large (>100 KB) for “Oracle + TPC-H” application INSTITUTE OF COMPUTING TECHNOLOGY

Rethink DMA Mechanism l The Nature of DMA l l l INSTITUTE OF COMPUTING TECHNOLOGY There is a producer-consumer relationship between CPU and DMA engine Memory plays a role of transient place for I/O data transferred between processor and I/O device Corollaries l l Once I/O data is produced, it will be consumed I/O data within DMA buffer will be used only once in most cases (i. e. almost no reuse). Characterizations of I/O data are different from CPU data It may not be appropriate to store I/O data and CPU data together

Overview l l l Background Nature of DMA Mechanism DMA Cache Scheme Research Methodology Evaluations Conclusions and Ongoing Work INSTITUTE OF COMPUTING TECHNOLOGY

DMA Cache Proposal l A Dedicated Cache l Storing I/O data l Capable of exchanging data with processor’s last level cache (LLC) l Reduce overhead of I/O data movement DMA INSTITUTE OF COMPUTING TECHNOLOGY

DMA Cache Design Issues l l l DMA Cache State Diagram is similar to CPU Cache in Uniprocessor system We are researching multiprocessor platform… DMA Cache State Diagram l Cache Coherence Data Path Replacement Policy Write Policy Prefetching CPU Cache State Diagram l INSTITUTE OF COMPUTING TECHNOLOGY

DMA Cache Design Issues l l l Cache Coherence Data Path Replacement Policy Write Policy Prefetching DMA Additional data paths and data access ports for LLC are not required because data migration operations between DMA cache and LLC can share existing data paths and ports of snooping mechanism INSTITUTE OF COMPUTING TECHNOLOGY

Data Path: CPU Read CPU read Cache Ctrl Miss in LLC & Hit in DMA Cache Hit in DMA cache? cmd data Cache Ctrl Last Level Cache DMA Cache Snoop Ctrl System Bus DMA Ctrl Mem Ctrl I/O Device Memory INSTITUTE OF COMPUTING TECHNOLOGY

Data Path: DMA Read Cache Ctrl Last Level Cache DMA Cache Snoop Ctrl Hit in LLC? Snoop Ctrl System Bus DMA read DMA Ctrl Mem Ctrl I/O Device Memory cmd data INSTITUTE OF COMPUTING TECHNOLOGY Miss in DMA Cache & Hit in LLC

DMA Cache Design Issues l l l Cache Coherence Data Path Replacement Policy Write Policy Prefetching l INSTITUTE OF COMPUTING TECHNOLOGY An LRU-like Replace Policy 1. Invalid Block 2. Clean Block 3. Dirty Block

DMA Cache Design Issue l l l INSTITUTE OF COMPUTING TECHNOLOGY Cache Coherence Data Path Replacement Policy Write Policy l Adopt Write-Allocate Policy Prefetching l Both Write-Back or Write Through policies are available

DMA Cache Design Issue l l l Cache Coherence Data Path Replacement Policy Write Policy Prefetching l INSTITUTE OF COMPUTING TECHNOLOGY Adopt straightforward sequential prefetching l Prefetching trigged by cache miss l Fetch 4 blocks one time

Overview l l l Background Nature of DMA Mechanism DMA Cache Scheme Research Methodology Evaluations Conclusions and Ongoing Work INSTITUTE OF COMPUTING TECHNOLOGY

Memory Trace Collection l INSTITUTE OF COMPUTING TECHNOLOGY Hyper Memory Trace Tool (HMTT) l l Capable of Collecting all memory requests Provide APIs for injecting tags into memory trace to identify high-level system operations

FPGA Emulation l l L 2 Cache from Godson-2 F DDR 2 Memory Controller from Godson-2 F DDR 2 DIM model from Micron Technology Xtreme system from Cadence Memory trace DMA Cache L 2 Cache Mem. Ctrl DDR 2 Dram INSTITUTE OF COMPUTING TECHNOLOGY

Overview l l l Background Nature of DMA Mechanism DMA Cache Scheme Research Methodology Evaluations Conclusions and Ongoing Work INSTITUTE OF COMPUTING TECHNOLOGY

Experimental Setup l Machine l l AMD Opteron 2 GB Memory 1 GE NIC IDE disk l Configurations l Snoop Cache (2 MB) l Shared Cache (2 MB) l DMA Cache l l Benchmark l l l File Copy TPC-H SPECWeb 2005 INSTITUTE OF COMPUTING TECHNOLOGY l l l 256 KB + prefetch 256 KB w/o prefetch 128 KB + prefetch 128 KB w/o prefetch 64 KB + prefetch 64 KB w/o prefetch 32 KB + prefetch 32 KB w/o prefetch

Characterization of DMA INSTITUTE OF COMPUTING TECHNOLOGY l l The portions of DMA memory reference varies depending on applications The sizes of DMA requests varies depending on application

Normalized Speedup INSTITUTE OF COMPUTING TECHNOLOGY l l Baseline is snoop cache scheme DMA cache schemes exhibits better performance than others

DMA Write & CPU Read Hit Rate l l Both shared cache and DMA cache exhibit high hit rates Then, where do cycle go for shared cache scheme? INSTITUTE OF COMPUTING TECHNOLOGY

Breakdown of Normalized Total Cycles INSTITUTE OF COMPUTING TECHNOLOGY

% of DMA Writes causing Dirty Block Replacement INSTITUTE OF COMPUTING TECHNOLOGY l l Those DMA writes cause cache pollution and thrashing problem The 256 KB DMA cache is able to significantly eliminate these phenomena

% of Valid Prefetched Blocks INSTITUTE OF COMPUTING TECHNOLOGY l l DMA caches can exhibit an impressive high prefetching accuracy This is because I/O data has very regular access pattern.

Overview l l l Background Nature of DMA Mechanism DMA Cache Scheme Research Methodology Evaluations Conclusions and Ongoing Work INSTITUTE OF COMPUTING TECHNOLOGY

Conclusions and Ongoing Work INSTITUTE OF COMPUTING TECHNOLOGY l The Nature of DMA l l There is a producer-consumer relationship between CPU and DMA engine Memory plays a role of transient place for I/O data transferred between processor and I/O device l We propose a DMA cache scheme and its design issues. l Experimental results show that DMA cache can significantly improve I/O performance. l Ongoing Work l l l The impact of multiprocessor, multiple DMA channels for DMA cache In theory, a shared cache with an intelligent replacement policy can achieve the effect of DMA cache scheme. Godson-3 has integrated an dedicate cache management policy for I/O data.

INSTITUTE OF COMPUTING TECHNOLOGY THANKS！ Q&A?