Lecture 4 1 Memory Hierarchy Introduction Learning Objectives

Lecture 4. 1 Memory Hierarchy: Introduction

Learning Objectives n n Outline the memory hierarchy Explain the principle of locality n n n Understand the abstract view of cache in computer organization n n Spatial locality Temporal locality Cache is transparent to processor Calculate hit rate and miss rate 2

Coverage n Textbook Chapter 5. 1 3

Processor-Memory Performance Gap µProc 55%/year (2 X/1. 5 yr) “Moore’s Law” Processor-Memory Performance Gap (grows 50%/year) DRAM 7%/year (2 X/10 yrs)

The “Memory Wall” q Clocks per DRAM access Processor vs DRAM speed disparity continues to grow Clocks per instruction n Good memory hierarchy (cache) design is increasingly important to overall performance

Memory Technology n Static RAM (SRAM) n n Dynamic RAM (DRAM) n n 50 ns – 70 ns, $20 – $75 per GB Magnetic disk n n 0. 5 ns – 2. 5 ns, $2000 – $5000 per GB 5 ms – 20 ms, $0. 20 – $2 per GB Ideal memory n n Access time of SRAM Capacity and cost/GB of disk 6

Memory Hierarchy n The Pyramid 7

A Typical Memory Hierarchy q Take advantage of the principle of locality to present the user with as much memory as is available in the cheapest technology while at the speed offered by the fastest technology On-Chip Components Control Instr Data Cache ITLB DTLB Reg. File Datapath Speed (cycles): ½’s 1’s Size (bytes): 10 K’s Cost: 100’s highest Second Level Cache (SRAM) 10’s Main Memory (DRAM) 100’s G’s Secondary Memory (Disk) 10, 000’s T’s lowest

Inside the Processor n AMD Barcelona: 4 processor cores

Principle of Locality n n Programs access a small proportion of their address space at any time Temporal locality (locality in time) n Items accessed recently are likely to be accessed again soon n e. g. , instructions in a loop, induction variables Keep most recently accessed items into the cache Spatial locality (locality in space) n Items near those accessed recently are likely to be accessed soon n n E. g. , sequential instruction access, array data Move blocks consisting of contiguous words closer to the processor 10

Taking Advantage of Locality n Memory hierarchy n n Store everything on disk Copy recently accessed (and nearby) items from disk to smaller DRAM memory n n Main memory Copy more recently accessed (and nearby) items from DRAM to smaller SRAM memory n Cache memory attached to CPU Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 11

Memory Hierarchy Levels n Block (aka cache line) n n n Unit of copying May be multiple words If accessed data is present in upper level n Hit: access satisfied by upper level n n Hit ratio: hits/accesses Hit time: Time to access the block + Time to determine hit/miss n If accessed data is absent n Miss: data not in upper level n n Miss ratio: misses/accesses = 1 – hit ratio Miss penalty: Time to access the block in the lower level + Time to transmit that block to the level that experienced the miss + Time to insert the block in that level + Time to pass the block to the requestor Chapter 5 — Large and Fast: Exploiting Memory Hierarchy — 12