EECS 470 Lecture 7 Branches Address prediction and

EECS 470 Lecture 7 Branches: Address prediction and recovery (And interrupt recovery too. )

Warning: Crazy times coming • Project handout and group formation today – Help me to end class 12 minutes early… • P 3 is due on Sunday 2/9 – It’s a lot of work (20 hours? ) • Proposal is due on Monday (2/10) – It’s not a lot of work (1 hour? ) to do the write-up, but you’ll need to meet with your group and discuss things. • Don’t worry too much about getting this right. You’ll be allowed to change (we’ll meet the following Friday). Just a line in the sand. • HW 3 is due on Wednesday 2/12 – It’s a fair bit of work (3 hours? ) • 20 minute group meetings on Friday 2/14 (rather than inlab) • Midterm is on Monday 2/17 in the evening (6 -8 pm) – Exam Q&A on Saturday (2/15 from 6 -8 pm) – Q&A in class 2/17 – Best way to study is look at old exams (posted on-line!)

Last time: • Covered branch predictors – Direction – Address

General speculation • Control speculation – “I think this branch will go to address 90004” • Data speculation – “I’ll guess the result of the load will be zero” • Memory conflict speculation – “I don’t think this load conflicts with any proceeding store. ” • Error speculation – “I don’t think there were any errors in this calculation”

Speculation in general • Need to be 100% sure on final correctness! – So need a recovery mechanism – Must make forward progress! • Want to speed up overall performance – So recovery cost should be low or expected rate of occurrence should be low. – There can be a real trade-off on accuracy, cost of recovery, and speedup when correct. • Should keep the worst case in mind…

BTB (Chapter 3. 9) • Branch Target Buffer – Addresses predictor – Lots of variations • Keep the target of “likely taken” branches in a buffer – With each branch, associate the expected target.

• BTB indexed by current PC – If entry is in BTB fetch target address next • Generally set associative (too slow as FA) • Often qualified by branch taken predictor Branch PC Target address 0 x 05360 AF 0 0 x 05360000 … … … … …

So… • BTB lets you predict target address during the fetch of the branch! • If BTB gets a miss, pretty much stuck with nottaken as a prediction – So limits prediction accuracy. • Can use BTB as a predictor. – If it is there, predict taken. • Replacement is an issue – LRU seems reasonable, but only really want branches that are taken at least a fair amount.

Pipeline recovery is pretty simple • Squash and restart fetch with right address – Just have to be sure that nothing has “committed” its state yet. • In our 5 -stage pipe, state is only committed during MEM (for stores) and WB (for registers)

Tomasulo’s • So far we’ve said “just don’t speculate past unresolved branches” – By that we mean, don’t even dispatch instructions after an unresolved branch. – We are worried that an instruction that wasn’t supposed to happen will modify architectural state. • What are our other options? –. –. – Speculate past unresolved branches and create a recovery mechanism.

What we need is: • Some way to not commit instructions until all branches before it are committed. – Just like in the pipeline, something could have finished execution, but not updated anything “real” yet.

Interrupts • These have a similar problem. – If we can execute out-of-order a “slower” instruction might not generate an interrupt until an instruction in front of it has finished. • This sounds like the end of out-of-order execution – I mean, if we can’t finish out-of-order, isn’t this pointless?

Exceptions and Interrupts Exception Type Sync/Async Maskable? Restartable? I/O request Async Yes System call Sync No Yes Breakpoint Sync Yes Overflow Sync Yes Page fault Sync No Yes Misaligned access Sync No Yes Memory Protect Sync No Yes Machine Check Async/Sync No No Async No No Power failure

Precise Interrupts Instructions Completely Finished • Implementation approaches Precise State • E. g. , Cray-1 PC No Instruction Has Executed At All – Don’t Speculative State – Buffer speculative results • E. g. , P 4, Alpha 21264 • History buffer • Future file/Reorder buffer

Precise Interrupts and branches via the Reorder Buffer • @ Alloc – Allocate result storage at Tail Any order IF ID MEM EX Alloc. Sched CT In-order PC Dst reg. ID Dst value Except? • @ Sched In-order ROB Head ARF Tail • Reorder Buffer (ROB) – Circular queue of spec state – May contain multiple definitions of same register – Get inputs (ROB T-to-H then ARF) – Wait until all inputs ready • @ WB – Write results/fault to ROB – Indicate result is ready • @ CT – – Wait until inst @ Head is done If fault, initiate handler Else, write results to ARF Deallocate entry from ROB

Reorder Buffer Example ROB Code Sequence reg. ID: r 8 reg. ID: f 1 result: 2 result: ? Except: n. Except: ? f 1 = f 2 / f 3 r 3 = r 2 + r 3 r 4 = r 3 – r 2 Time Initial Conditions H - reorder buffer empty - f 2 = 3. 0 - f 3 = 2. 0 - r 2 = 6 - r 3 = 5 T reg. ID: r 8 reg. ID: f 1 reg. ID: r 3 result: 2 result: ? Except: n. Except: ? H T reg. ID: r 8 reg. ID: f 1 reg. ID: r 3 reg. ID: r 4 result: 2 result: ? result: 11 result: ? Except: n. Except: ? Except: N Except: ? H r 3 T

Reorder Buffer Example ROB Code Sequence reg. ID: r 8 reg. ID: f 1 reg. ID: r 3 reg. ID: r 4 result: 2 result: ? result: 11 result: 5 Except: n. Except: ? Except: n f 1 = f 2 / f 3 r 3 = r 2 + r 3 r 4 = r 3 – r 2 Time Initial Conditions H - reorder buffer empty - f 2 = 3. 0 - f 3 = 2. 0 - r 2 = 6 - r 3 = 5 T reg. ID: r 8 reg. ID: f 1 reg. ID: r 3 reg. ID: r 4 result: 2 result: ? result: 11 result: 5 Except: n. Except: y. Except: n H T reg. ID: f 1 reg. ID: r 3 reg. ID: r 4 result: ? result: 11 result: 5 Except: y. Except: n H T

Reorder Buffer Example ROB Code Sequence f 1 = f 2 / f 3 r 3 = r 2 + r 3 r 4 = r 3 – r 2 Time Initial Conditions HT - reorder buffer empty - f 2 = 3. 0 - f 3 = 2. 0 - r 2 = 6 - r 3 = 5 first inst of fault handler H T

There is more complexity here • Rename table needs to be cleared – Everything is in the ARF – Really do need to finish everything which was before the faulting instruction in program order. • What about branches? – Would need to drain everything before the branch. • Why not just squash everything that follows it?

And while we’re at it… • Does the ROB replace the RS? – Is this a good thing? Bad thing?

ROB • ROB – ROB is an in-order queue where instructions are placed. – Instructions complete (retire) in-order – Instructions still execute out-of-order – Still use RS • Instructions are issued to RS and ROB at the same time • Rename is to ROB entry, not RS. • When execute done instruction leaves RS – Only when all instructions in before it in program order are done does the instruction retire.

Adding a Reorder Buffer

CDB T Tomasulo Data Structures (Timing Free Example, “P 6 scheme”) Map Table Reg Tag Reservation Stations (RS) T FU busy op R r 0 r 1 r 2 r 3 r 4 1 2 3 4 5 Instruction r 0=r 1*r 2 r 1=r 2*r 3 Branch if r 1=0 r 0=r 1+r 1 r 2=r 2+1 T 2 V 1 V 2 V ARF Reg V r 0 r 1 r 2 r 3 r 4 Reorder Buffer (Ro. B) Ro. B Number 0 1 Dest. Reg. Value 2 3 4 5 6

Review Questions • Could we make this work without a RS? – If so, why do we use it? • Why is it important to retire in order? • Why must branches wait until retirement before they announce their mispredict? – Any other ways to do this?

More review questions 1. What is the purpose of the Ro. B? 2. Why do we have both a Ro. B and a RS? – Yes, that was pretty much on the last page… 3. Misprediction a) When to we resolve a mis-prediction? b) What happens to the main structures (RS, Ro. B, ARF, Rename Table) when we mispredict? 4. What is the whole purpose of Oo. O execution?

Topic change • Why on earth are we doing this? – Why do we think it helps? • Homework 2 problems 5 and 6 made the argument. – Only need to obey true data dependencies. • Huge speedup potential.

Optimizing CPU Performance • Golden Rule: t. CPU = Ninst*CPI*t. CLK • Given this, what are our options – Reduce the number of instructions executed – Reduce the cycles to execute an instruction – Reduce the clock period • Our first focus: Reducing CPI – Approach: Instruction Level Parallelism (ILP)

Why ILP? • Requirements Vs. – Parallelism – Large window – Limited control deps – Eliminate “false” deps – Find run-time deps

How Much ILP is There? (Chapter 3. 10)

How Large Must the “Window” Be?

ALU Operation GOOD, Branch BAD Expected Number of Branches Between Mispredicts E(X) ~ 1/(1 -p) E. g. , p = 95%, E(X) ~ 20 brs, 100 -ish insts

How Accurate are Branch Predictors?

Impact of Physical Storage Limitations • Each instruction “in flight” must have storage for its result – Really worse than this because of mispeculation…

Registers GOOD, Memory BAD • Benefits of registers – Well described deps – Fast access – Finite resource • Memory loses these benefits for flexibility *p = … *q = … ? … = *p

“Bottom Line” for an Ambitious Design