L 15 Caches II CSE 351 Summer 2018

L 15: Caches II CSE 351 Summer 2018 Instructor: Justin Hsia Teaching Assistants: Josie Lee Natalie Andreeva Teagan Horkan https: //what-if. xkcd. com/111/ CSE 351, Summer 2018

L 15: Caches II CSE 351, Summer 2018 Administrivia v v v Lab 3 due Friday (7/27) Homework 4 released today, due next Wed (8/1) § On Structs, Caches Mid-Quarter Survey Feedback § Pace is “moderate” to “a bit fast” § You talk too fast in lecture (or rush at the end) and I wish there were more peer instruction questions § Canvas quiz answer keys are annoying § Hard to get started on the labs • Videos and office hours! 2

L 15: Caches II CSE 351, Summer 2018 Checking for a Requested Address v Block Number 3

L 15: Caches II CSE 351, Summer 2018 Peer Instruction Question Vote at http: //Poll. Ev. com/justinh v Based on the following behavior, which of the following block sizes is NOT possible for our cache? § Cache starts empty, also known as a cold cache § Access (addr: hit/miss) stream: • A. B. C. D. E. (14: miss), (15: hit), (16: miss) 4 bytes 8 bytes 16 bytes 32 bytes We’re lost… 4

L 15: Caches II CSE 351, Summer 2018 Making memory accesses fast! v v v Cache basics Principle of locality Memory hierarchies Cache organization § Direct-mapped (sets; index + tag) § Associativity (ways) § Replacement policy § Handling writes Program optimizations that consider caches 5

L 15: Caches II CSE 351, Summer 2018 Direct-Mapped Cache Memory Block Num Block Data 00 00 00 01 00 10 00 11 01 00 01 01 01 10 01 11 10 00 10 01 10 10 10 11 11 00 11 01 11 10 11 11 Cache Index 00 01 10 11 v Tag 00 11 01 01 Block Data Hash function: (block number) mod (# of blocks in cache) § Each memory address maps to exactly one index in the cache § Fast (and simpler) to find an address 6

L 15: Caches II CSE 351, Summer 2018 Direct-Mapped Cache Problem Memory Block Num Block Data 00 00 00 01 00 10 00 11 01 00 01 01 01 10 01 11 10 00 10 01 10 10 10 11 11 00 11 01 11 10 11 11 Cache Index 00 01 10 11 v v Tag ? ? Block Data ? ? What happens if we access the following addresses? § 8, 24, 8, …? § Conflict in cache (misses!) § Rest of cache goes unused Solution? 7

L 15: Caches II CSE 351, Summer 2018 Associativity v What if we could store data in any place in the cache? § More complicated hardware = more power consumed, slower v So we combine the two ideas: § Each address maps to exactly one set § Each set can store block in more than one way 1 -way: 8 sets, 1 block each 0 1 2 3 4 5 6 7 Set 0 2 -way: 4 sets, 2 blocks each Set 4 -way: 2 sets, 4 blocks each Set 8 -way: 1 set, 8 blocks 0 1 0 2 1 3 direct mapped fully associative 8

L 15: Caches II Cache Organization (3) CSE 351, Summer 2018 Note: The textbook uses “b” for offset bits v Used for tag comparison Decreasing associativity Direct mapped (only one way) Selects the set Selects the byte from block Increasing associativity Fully associative (only one set) 9

L 15: Caches II CSE 351, Summer 2018 block size: 16 B capacity: 8 blocks address: 16 bits Example Placement v Where would data from address 0 x 1833 be placed? § Binary: 0 b 0001 1000 0011 Direct-mapped Set Tag Data 0 1 2 3 4 5 6 7 2 -way set associative Set Tag Data 0 1 2 3 4 -way set associative Set Tag Data 0 1 10

L 15: Caches II CSE 351, Summer 2018 Block Replacement v v Any empty block in the correct set may be used to store block If there are no empty blocks, which one should we replace? § No choice for direct-mapped caches § Caches typically use something close to least recently used (LRU) 2 -way set“not associative Direct-mapped 4 -way set associative (hardware usually implements most recently used”) Set Tag 0 1 2 3 4 5 6 7 Data Set Tag 0 1 2 3 Data Set Tag Data 0 1 11

L 15: Caches II CSE 351, Summer 2018 Peer Instruction Question v Our cache is of size 2 Ki. B with block size of 128 B. If our cache has 2 sets, what is its associativity? § Vote at http: //Poll. Ev. com/justinh A. B. C. D. E. v 2 4 8 16 We’re lost… If addresses are 16 bits wide, how wide is the Tag field? 12

L 15: Caches II CSE 351, Summer 2018 set “line” (block plus management bits) V Tag 0 1 2 K-1 valid bit 13

L 15: Caches II CSE 351, Summer 2018 Notation Review v We just introduced a lot of new variable names! § Please be mindful of block size notation when you look at past exam questions or are watching videos Variable This Quarter Formulas Block size Cache size Associativity Number of Sets Address space Address width Tag field width Index field width Offset field width 14

L 15: Caches II CSE 351, Summer 2018 1) Locate set 2) Check if any line in set is valid and has matching tag: hit 3) Locate data starting at offset Cache Read Address of byte in memory: tag set block index offset data begins at this offset v tag 0 1 2 K-1 valid bit 15

L 15: Caches II CSE 351, Summer 2018 v tag 0 1 2 3 4 5 6 7 Address of int: 0… 01 100 find set 16

L 15: Caches II CSE 351, Summer 2018 Address of int: valid? + match? : yes = hit v tag 0 1 2 3 0… 01 4 5 6 100 7 block offset 17

L 15: Caches II CSE 351, Summer 2018 Address of int: valid? + match? : yes = hit v tag 0 1 2 3 0… 01 4 5 6 100 7 block offset int (4 B) is here This is why we want alignment! No match? Then old line gets evicted and replaced 18

L 15: Caches II CSE 351, Summer 2018 Address of short int: 0… 01 v tag 0 1 2 3 4 5 6 7 v tag 0 1 2 3 4 5 6 7 100 find set 19

L 15: Caches II CSE 351, Summer 2018 Address of short int: 0… 01 compare both 100 valid? + match: yes = hit v tag 0 1 2 3 4 5 6 7 block offset 20

L 15: Caches II CSE 351, Summer 2018 Address of short int: 0… 01 compare both 100 valid? + match: yes = hit v tag 0 1 2 3 4 5 6 7 block offset short int (2 B) is here No match? • One line in set is selected for eviction and replacement • Replacement policies: random, least recently used (LRU), … 21

L 15: Caches II CSE 351, Summer 2018 Types of Cache Misses: 3 C’s! v 22

L 15: Caches II CSE 351, Summer 2018 What about writes? v Multiple copies of data exist: § L 1, L 2, possibly L 3, main memory v What to do on a write-hit? § Write-through: write immediately to next level § Write-back: defer write to next level until line is evicted (replaced) • v Must track which cache lines have been modified (“dirty bit”) What to do on a write-miss? § Write-allocate: (“fetch on write”) load into cache, then update line • Good if more writes or reads to the location follow § No-write-allocate: (“write around”) just write immediately to memory v Typical caches: § Write-back + Write-allocate, usually § Write-through + No-write-allocate, occasionally 23

L 15: Caches II CSE 351, Summer 2018 351 Cache Simulator v v We’ve built a new cache simulator! § To help you visualize what the cache parameters mean § To help you see the effects of cache accesses § Please report any bugs you find! https: //courses. cs. washington. edu/courses/cse 351/cachesim/ § Demo [if time] 24