Using Speculation to Simplify Multiprocessor Design Daniel J

Using Speculation to Simplify Multiprocessor Design Daniel J. Sorin 1, Milo M. K. Martin 2, Mark D. Hill 3, David A. Wood 3 1 Dept. of Electrical & Computer Engineering, Duke University 2 Dept. of Computer & Information Science, Univ. of Pennsylvania 3 Computer Sciences Dept. , University of Wisconsin-Madison (C) 2004 Daniel Sorin Duke Architecture

My Talk in One Slide • Shared memory multiprocessors are complicated – Difficult to design for every possible corner case • Proposal: Use speculation to target the common case – Speculate that corner cases won’t happen – Detect if they do occur and recover system – Ensure forward progress • Case studies – Simplify cache coherence protocols – Simplify the interconnection network 2 IPDPS 2004 – Daniel Sorin

Speculation for Simplicity • Why we want to avoid complexity – Time and money for design and verification • Design for the common case – But we have to make ALL cases work correctly • Examples of this philosophy in uniprocessors – Trapping to software for infrequent/obsolescent instructions – Pentium 4 recovers from edge case scheduler deadlocks • But this idea hadn’t been used for multiprocessors – Key: we now have efficient multiprocessor recovery 3 IPDPS 2004 – Daniel Sorin

Framework for Speculation • Four keys to design simplification with speculation 1) 2) 3) 4) Ensure that mis-speculations are rare Detect all mis-speculations Recover from mis-speculations Ensure forward progress even for worst-case 4 IPDPS 2004 – Daniel Sorin

Safety. Net Checkpoint/Recovery • • • We use Safety. Net [ISCA 2002] for system recovery All-hardware checkpoint/recovery for shared memory multiprocessors Periodically, takes logical checkpoints of system – Including caches, coherence state, memory, directory state – Implements checkpointing with incremental logging – Consistent checkpoints using logical time coordination • • Can recover 100, 000+ cycles Negligible performance impact – Incremental logging performed off critical path • 5 Small log buffers (512 KB) at caches & memories IPDPS 2004 – Daniel Sorin

The Need for Multiprocessor Recovery • Assumption: multiprocessors will have system-wide recovery mechanisms for purposes of availability – As fault rates keep increasing, recovery is crucial • Will be all-hardware (like Safety. Net) for performance – But many alternative designs are possible • 6 We leverage this recovery mechanism for recovering from mis-speculations IPDPS 2004 – Daniel Sorin

Outline • • • 7 A Framework for Speculation Simplifying Cache Coherence Protocols Simplifying the Interconnection Network Evaluation Conclusions IPDPS 2004 – Daniel Sorin

Directory Protocol Complexity • We want adaptive routing in interconnection network – Better performance and availability – But adaptive routing precludes point-to-point ordering • So what? – Point-to-point ordering simplifies protocol design – Eliminates several potential corner case races 8 IPDPS 2004 – Daniel Sorin

Race Case in Directory Protocol • Example race if no point-to-point ordering in network Forwarded Request. Read. Write Dir Request. Read. Writeback P 1 P 2 Request. Read. Write arrives first at Dir, gets forwarded to P 1 9 IPDPS 2004 – Daniel Sorin

Race Case in Directory Protocol Forwarded Request. Read. Write Dir Request. Read. Writeback P 1 Writeback Ack P 2 Forwarded Request. Read. Write arrives after Writeback Ack 10 IPDPS 2004 – Daniel Sorin

Race Case in Directory Protocol • Problem: P 1 sees Forwarded Request in state Invalid Forwarded Request. Read. Write Dir Request. Read. Writeback P 1 Writeback Ack P 2 Not possible if point-to-point order in interconnection network 11 IPDPS 2004 – Daniel Sorin

Simplifying a Directory Protocol • Speculate that adaptive network provides ordering 1) Why is mis-speculation rare? – Not many re-orderings – Most re-orderings don’t matter! 2) How do we detect all mis-speculations? – If we get a Forwarded Request. Read. Write in state Invalid 3) How do we recover? – Safety. Net 4) How do we ensure forward progress? – Slow-start operation for a while after recovery – Guarantees that this race can’t keep recurring 12 IPDPS 2004 – Daniel Sorin

Simplifying a Snooping Coherence Protocol • During design, we missed a corner case Request Read. Writeback State M • • 13 State trans 1 Request Read. Write State trans 2 ? ? ? Solution: it’s rare, treat it as mis-speculation Detect by seeing Request. Read. Write in state trans 2 Recovery with Safety. Net Forward progress with slow-start after recovery IPDPS 2004 – Daniel Sorin

Outline • A Framework for Speculation • Simplifying Cache Coherence Protocols • Simplifying the Interconnection Network – Deadlock – Avoiding deadlock • Evaluation • Conclusions 14 IPDPS 2004 – Daniel Sorin

Two Causes of Deadlock full of requests P 1 Endpoint Deadlock Switch Response full of requests P 2 switch 1 full of messages Deadlock Message M 1 switch 2 Message M 2 15 Response full of messages IPDPS 2004 – Daniel Sorin

Avoiding Deadlock • Simple but wasteful solution: full buffering – But it’s rare that we ever need full buffering • More efficient solution: virtual channels (networks) • For endpoint deadlock – Need a virtual network per type of message • For switch deadlock – Need some number of virtual channels per virtual network – Depends on network topology and routing scheme • A major source of design complexity 16 IPDPS 2004 – Daniel Sorin

Simplifying Deadlock Avoidance • Speculate that deadlock won’t occur, despite using less than full buffering and no virtual channels 1) Why is mis-speculation rare? – Can usually avoid deadlock with reasonable buffering 2) How do we detect all mis-speculations? – Timeout mechanism for cache coherence transactions 3) How do we recover? – Safety. Net 4) How do we ensure forward progress? – Slow-start operation for a while after recovery – Guarantees that deadlock can’t keep recurring 17 IPDPS 2004 – Daniel Sorin

Outline • • A Framework for Speculation Simplifying Cache Coherence Protocols Simplifying the Interconnection Network Evaluation – Goals – Methodology – Results • Conclusions 18 IPDPS 2004 – Daniel Sorin

Goals • Discover the point at which mis-speculation recoveries impact performance – Determines whether our simplified snooping protocol and our simplified interconnection network are viable • Determine whether our simplified directory protocol can usefully speculate on point-to-point ordering 19 IPDPS 2004 – Daniel Sorin

Methodology • Full-system simulation – Simics provides full-system functionality – We added detailed timing model for memory system • Workloads – – – 20 Online transaction processing (OLTP) with DB 2 SPECjbb 2000 java middleware Apache static web serving Slashcode dynamic web serving Barnes-Hut scientific simulation IPDPS 2004 – Daniel Sorin

How Rare Must Mis-speculation Be? We can tolerate high mis-speculation rates – these rates are much higher than what our simplified designs incur 21 IPDPS 2004 – Daniel Sorin

Adaptive Routing with Speculative Ordering Adaptive routing can provide better performance by routing around congestion, even with mis-speculations 22 IPDPS 2004 – Daniel Sorin

Conclusions • Simplify multiprocessor design with speculation – Treat corner cases as mis-speculations & recover from them • Must be able to ensure that – – Mis-speculations are sufficiently rare Can detect all mis-speculations Can recover from mis-speculations Can provide forward progress in all cases • Showed how to simplify – Cache coherence protocols – Interconnection network deadlock avoidance • Applicable to other complicated designs 23 IPDPS 2004 – Daniel Sorin