Snoop cache AMANO Hideharu Keio University hungaamicskeioacjp Textbookpp

Snoop cache AMANO, Hideharu, Keio University hunga@am．ics．keio．ac．jp Textbook　pp. 40 -60

Cache memory n n n A small high speed memory for storing frequently accessed data/instructions. Essential for recent microprocessors. Basis knowledge for uni-processor’s cache is reviewed first.

CPU Memory Hierarchy Locality is used. Small high speed Large low speed Managed by Operating System L 1　Cache Transparent from Software On-Chip cache ～ 64 KB　1 -2 clock L 2 Cache ～ 256 KB 3 -10 clock L 3 Cache SRAM 2 M～ 4 MB 10 -20 clock Main memory DRAM 4～ 16 GB 50 -100 clock Secondary Memory μ-msec TB

Controlling cache n Mapping q q q n Write policy q q n direct map n-way set associative map full-map write through write back Replace policy q LRU(Least Recently Used)

Direct Map From CPU 0011010 0011 010 100 … … Main Memory (1 KB=128 blocks) 010 = Yes：Hit Data 010 0011 Cache (64 B=8 blocks) Cache Directory (Tag Memory) 8 entries X (4 bit ) Simple Directory structure　

Direct Map (Conflict Miss) From CPU 0000010 0000 010 100 … Main Memory 010 = … No: Miss Hit 010 0011 0000 Cache Conflict Miss occurs between two blocks with the same index Cache Directory (Tag Memory)

2 -way set associative Map From CPU 0011010 00110 10 100 … … Main Memory (1 KB=128 blocks) 10 Yes: Hit = Data 10 00110 Cache (64 B=8 blocks) = 00000 No Cache Directory (Tag Memory) 4 entries X 5 bit X 2

2 -way set associative Map From CPU 0000010 00000 10 100 0011010 … … 10 Main Memory (1 KB=128 blocks) No = 10 00110 Cache (64 B=8 blocks) = 00000 Yes: Hit Cache Directory (Tag Memory) 4 entries X 5 bit X 2 Data Conflict　Miss is reduced

Write　Through　（Hit） 0011010 … From CPU … Main Memory (1 KB=128 blocks) 0011 010 100 The main memory is updated 0011 Hit Cache Directory (Tag Memory) 8 entries X (4 bit ) Cache (64 B=8 blocks) Write Data

Write　Through　（Miss：Write non-allocate(Direct write) ) 0000010 0011010 … … From CPU Main Memory (1 KB=128 blocks) 0000 010 100 Only main memory is updated 0011 Miss Cache Directory (Tag Memory) 8 entries X (4 bit ) Cache (64 B=8 blocks) Write Data

Write　Through　（Miss：Write allocate(Fetch-onwrite) ) 0000010 0011010 … From CPU … Main Memory (1 KB=128 blocks) 0000 010 100 0011 0000 Miss Cache Directory (Tag Memory) 8 entries X (4 bit ) Cache (64 B=8 blocks) Write Data

Write Back　（Hit） 0011010 … … From CPU Main Memory (1 KB=128 blocks) 0011 010 100 Dirty 0011 0 1 Hit Cache Directory (Tag Memory) 8 entries X (4 bit+1 bit ) Cache (64 B=8 blocks) Write Data

Write　Back　（Replace） 0000010 0011010 … … From CPU Write Back 0000 010 100 Dirty 0011 1 Miss Cache Directory (Tag Memory) 8 entries X (4 bit+1 bit ) Main Memory (1 KB=128 blocks) Cache (64 B=8 blocks)

Write　Back　（Replace） 0000010 0011010 … … From CPU 0000 010 100 Replace Main Memory (1 KB=128 blocks) Clean 0011 0 0000 Cache (64 B=8 blocks) Cache Directory (Tag Memory) 8 entries X (4 bit+1 bit )

Shared memory connected to the bus n Cache is required q Shared cache n q Often difficult to implement even in on-chip multiprocessors Private cache n Consistency problem → Snoop cache

Shared Cache PE PE n 1 port shared cache q n Shared Cache Severe access conflict 4 -port shared cache q A large multi-port memory is hard to implement Bus Interface Main　Memory Shared cache is often used for L 2 cache of on-chip multiprocessors

Ｐｒｉｖａｔｅ（Snoop）　Cache Main　Memory (L 2 Cache) A　large　bandwidth　shared　bus Snoop Cache PU PU Each PU provides its own private cache

Bus as a broadcast media n n A single module can send (write) data to the media All modules can receive (read) the same data →　Broadcasting Tree Crossbar + Bus Network on Chip (No. C) n Here, I show as a shape of classic bus but remember that it is just a logical image.

Cache coherence problem Main　Memory (L 2 Cache) A　large　bandwidth　shared　bus A PU PU PU The same block is cached in two cache modules

Cache coherence (consistency) problem Main　Memory (L 2 Cache) A　large　bandwidth　shared　bus A PU A’ A PU PU Data in each cache is not the same PU

Coherence vs. Consistency Coherence and consistency are complementary： n Coherence defines the behavior of reads and writes to the same memory location, while n Consistency defines the behavior of reads and writes with respect to accesses to other memory location. Hennessy & Patterson “Computer Architecture the 5 th edition” pp. 353 n

Cache　Consistency　Protocol n Each cache keeps consistency by monitoring (snooping) bus transactions. Write　Through：Every written data updates the shared memory. Frequent access of bus will degrade performance Write　Back: Basis（Synapse) Invalidate Ilinois Berkeley Update Firefly (Broadcast) Dragon

Ｗｒｉｔｅ　Ｔｈｒｏｕｇｈ　Ｃａｃｈｅ（Invalidation type：Data read out） Main　Memory (L 2 Cache) Ｉ：Invalidated Ｖ：Valid A　large　bandwidth　shared　bus ＲｅａｄＶ PU PU PU

Ｗｒｉｔｅ　Ｔｈｒｏｕｇｈ　Ｃａｃｈｅ（Invalidate type：Data write into） Main　Memory (L 2 Cache) Ｉ：Invalidate Ｖ：Valid A　large　bandwidth　shared　bus Monitoring (Snooping) Ｖ→ I ＶＷｒｉｔｅ PU PU

Ｗｒｉｔｅ　Ｔｈｒｏｕｇｈ　Ｃａｃｈｅ（Invalidate type　Write-non-allocate) The target cache block is not existing in the cache Main　Memory Ｉ：Invalidated Ｖ：Valid A　large　bandwidth　shared　bus Monitoring (Snooping) V→ I Ｗｒｉｔｅ PU PU

Ｗｒｉｔｅ　Ｔｈｒｏｕｇｈ　Ｃａｃｈｅ（Invalidate type　Write-allocate) Cache block is not existing in the target cache Main　Memory Ｉ：Invalidated Ｖ：Valid A　large　bandwidth　shared　bus ＶＷｒｉｔｅ PU PU

Ｗｒｉｔｅ　Ｔｈｒｏｕｇｈ　Ｃａｃｈｅ（Invalidate type　Write-allocate) Main　Memory Ｉ：Invalidated Ｖ：Valid First, Fetch A　large　bandwidth　shared　bus ＶＶＷｒｉｔｅ PU PU Fetch and write PU PU

Ｗｒｉｔｅ　Ｔｈｒｏｕｇｈ　Ｃａｃｈｅ（Invalidate type　Write-allocate) Main　Memory Ｉ：Invalidated Ｖ：Valid A　large　bandwidth　shared　bus Monitoring (Snoop) Ｖ→ ＩＶＷｒｉｔｅ PU PU Fetch and write PU PU

Ｗｒｉｔｅ　Ｔｈｒｏｕｇｈ　Ｃａｃｈｅ（Update type）Ｉ：Invalidated Ｖ：Valid Main　Memory A　large　bandwidth　shared　bus Monitoring (Snoop) Ｖ V Ｗｒｉｔｅ PU PU PU Data is updated PU

The structure of Snoop cache Shared bus Directory can be accessed simultaneously from both sides. Directory The same Directory (Dual　Port) Cache　Memory Entity Directory CPU The bus transaction can be checked without caring the access from CPU.

Quiz n Following accesses are done sequentially into the same cache block of Write through　Write nonallocate protocol. How the state of each cache block is changed ? q q q PU A: Read PU B: Read PU A: Write PU B: Read PU B: Write PU A: Write

The Problem of Write　Through　 Cache n n In uniprocessors, the performance of the write through cache with well designed write buffers is comparable to that of write back cache. However, in bus connected multiprocessors, the write through cache has a problem of bus congestion.

Basic Protocol States attached to each block Main　Memory C：Clean (Consistent to shared memory) D: Dirty I：Invalidate A　large　bandwidth　shared　bus C C Read PU PU

Basic Protocol（A PU writes the data） Main　Memory A　large　bandwidth　shared　bus Invalidation signal C→ I C→ D Write PU PU Invalidation signal: address only transaction

Basic Protocol (A PU reads out) Main　Memory Read request A　large　bandwidth　shared　bus Snoop D I Read PU PU

Basic Protocol (A PU reads out) Main　Memory A　large　bandwidth　shared　bus D→ C C Read PU PU

Basic Protocol (A PU writes into again) Main　Memory Write request A　large　bandwidth　shared　bus Snoop I Snoop Cache D Snoop Cache PU PU PU W PU

Basic Protocol (A PU writes into again) Main　Memory A　large　bandwidth　shared　bus D→ I D W PU PU

State Transition Diagram of the Basic Protocol I write Replace read write miss for the block I Write back & Replace write hit D Invalidate write miss Replace D C C read miss Write back & Replace read miss for the block read miss Replace CPU request Bus snoop request

States for each block Illinois’s Protocol(MESI)CE：Clean　Exclusive Main　Memory CS：Clean　Sharable DE：Dirty　Exclusive I：Invalidate A　large　bandwidth　shared　bus CE PU The first PU reads Snoop Cache PU PU

States for each block Illinois’s Protocol CE：Clean　Exclusive CS：Clean　Sharable DE：Dirty　Exclusive I：Invalidate Main　Memory A　large　bandwidth　shared　bus Snoop CE →CS PU Snoop Cache CS Snoop Cache PU PU PU The second PU reads

Illinois’s Protocol (The role of CE) Main　Memory CE：Clean　Exclusive CS：Clean　Sharable DE：Dirty　Exclusive I：Invalidate A　large　bandwidth　shared　bus CE →DE Snoop Cache PU PU PU W PU CE is changed into DE without using the bus

Berkeley’s protocol (MOSI) Main　Memory A　large　bandwidth　shared　bus US Snoop Cache PU PU Snoop Cache R PU PU Ownership→responsibility of write back OS: Owned　Sharable　OE: Owned　Exclusive US: Unowned　Sharable　I：Invalidated

Berkeley’s protocol (MOSI protocol) Main　Memory A　large　bandwidth　shared　bus US Snoop Cache PU PU R PU PU Ownership→responsibility of write back OS: Owned　Sharable　OE: Owned　Exclusive US: Unowned　Sharable　I：Invalidated

Berkeley’s protocol (A PU writes into) Main　Memory A　large　bandwidth　shared　bus snoop US →OE Snoop Cache US PU PU →I Snoop Cache W PU PU Invalidation is done like the basic protocol

Berkeley’s protocol The block with US is Main　Memory not required to be written back A　large　bandwidth　shared　bus snoop OE Snoop Cache I Snoop Cache PU PU R PU A PU reads a block owned by the other PU PU

Berkeley’s protocol The block with US is Main　Memory not required to be written back A　large　bandwidth　shared　bus OE →OS Snoop Cache I →　US Snoop Cache R PU PU Inter-cache transfer occurs! In this case, the block with US is not consistent with the shared memory.

Firefly protocol (MES) Main　Memory The usage of CE is the same as that in Illinois. A　large　bandwidth　shared　bus CE →CS Snoop Cache R PU PU First read: 　CE Second read: 　CS CE：Clean　Exclusive　CS: Clean　Sharable DE: Dirty　Exclusive I：　Invalidate is not used!

Firefly protocol (Writes into the CS block) Main　Memory A　large　bandwidth　shared　bus CS Snoop Cache PU PU W All caches and shared memory are updated → Like update type Write　Through Cache

Firefly protocol (The role of CE) Main　Memory A　large　bandwidth　shared　bus CE →DE Snoop Cache PU PU PU W PU Like Illinoi’s, writing CE does not require bus transactions

Dragon protocol (MOES) Main　Memory A　large　bandwidth　shared　bus UE Snoop Cache PU PU Snoop Cache R PU PU Ownership→Resposibility of write back OS: Owned　Sharable　OE: Owned　Exclusive US: Unowned　Sharable　UE: Unowned　Exclusive

Dragon protocol Main　Memory A　large　bandwidth　shared　bus snoop UE →US Snoop Cache PU PU R PU PU Ownership→Resposibility of write back OS: Owned　Sharable　OE: Owned　Exclusive US: Unowned　Sharable　UE: Unowned　Exclusive

Dragon protocol Main　Memory A　large　bandwidth　shared　bus update US →OS Snoop Cache US Snoop Cache PU PU PU W PU Only corresponding cache block is updated. The block with US is not required to be written back.

Dragon protocol Main　Memory A　large　bandwidth　shared　bus Miss hit OE Snoop Cache R PU PU A PU reads a block owned by the other PU.

Dragon protocol Main　Memory A　large　bandwidth　shared　bus OE →OS Snoop Cache → US Snoop Cache R PU PU Direct inter-cache data transfer like Berkeley’s protocol

Dragon protocol (The role of the UE) Main　Memory A　large　bandwidth　shared　bus UE →OE Snoop Cache PU PU PU W PU No bus transaction is needed like CE is Illinois’

MOESI Protocol class Valid Owned O： Owned S：Sharable Exclusive M: Modified E： Exclusive I： Invalid

MOESI protocol class n n n Basic：MSI Illinois：MESI Berkeley: MOSI Firefly：MES Dragon: MOES Theoretically well defined model. Detail of cache is not characterized in the model.

Invalidate　ｖｓ．Update n The drawback of Invalidate protocol q n The drawback of Update protocol q n Frequent data writing to shared data makes bus congestion　→ ping-pong effect Once a block shared, every writing data must use shared bus. Improvement q q Competitive　Snooping Variable Protocol Cache

Ping-pong effect（A PU writes into） Main　Memory A　large　bandwidth　shared　bus Invalidation C →　I Snoop Cache C →D Ｗ PU PU

Ping-pong effect （The other reads out） Main　Memory A　large　bandwidth　shared　bus I →C Snoop Cache D →C Snoop Cache R PU PU

Ping-pong effect （The other writes again） Main　Memory A　large　bandwidth　shared　bus Invalidation Snoop Cache C →D C Snoop Cache PU PU →　I Ｗ PU PU

Ping-pong effect（A PU reads again） Main　Memory A　large　bandwidth　shared　bus D →C PU Snoop Cache I Snoop Cache PU PU →C R PU A cache block goes and returns iteratively →Ping-pong effect

The drawback of update protocol（ Firefly protocol） Main　Memory A　large　bandwidth　shared　bus CS Snoop Cache PU PU W B Once a block becomes CS, a block is sent even if B the block is not used any more. False　Sharing causes unnecessary bus transaction.

Summary n n n Snoop　Cache is the most successful technique for parallel architectures. In order to use multiple buses, a single block for sending control signals is used. Sophisticated techniques do not improve the performance so much. Variable structures can be considerable for on-chip multiprocessors. Recently, snoop protocols using No. C(Network-onchip) are researched.

Exercise n Following accesses are done sequentially into the same cache block of Illinois protocol and Firefly protocol. How the state of each cache block is changed ? q q q PU A: Read PU B: Read PU A: Write PU B: Read PU B: Write PU A: Write

Private FIQ blocks MPCore (ARM+NEC) It uses MESI　Protocol … Interrupt Distributor Timer CPU Wdog interface IRQ CPU/VFP L 1 Memory Private Peripheral Bus Snoop Control Unit (SCU) Duplicated L 1 Tag Private AXI R/W 64 bit Bus L 2 Cache Coherence Control Bus

Competitive　Snooping Main　Memory A　large　bandwidth　shared　bus CS Snoop Cache CS →I Snoop Cache W PU PU Update n times, and then invalidates The performance is degraded in some cases.

Write　Once　(Goodman　Protocol） Main　Memory A　large　bandwidth　shared　bus Invalidation C →　I PU Snoop Cache C →D →CE PU Ｗ PU Main memory is updated with invalidation. Only the first written data is transferred to the main memory.

Read　Broadcast（Berkeley) Main　Memory A　large　bandwidth　shared　bus US →OE Snoop Cache US PU PU →I US →I W PU PU Invalidation is the same as the basic protocol.