CS 152 Computer Architecture and Engineering Lecture 15

CS 152 Computer Architecture and Engineering Lecture 15 - Advanced Superscalars Krste Asanovic Electrical Engineering and Computer Sciences University of California at Berkeley http: //www. eecs. berkeley. edu/~krste http: //inst. eecs. berkeley. edu/~cs 152 4/1/2008 CS 152 -Spring’ 08

Last time in Lecture 14 • Control hazards are serious impediment to superscalar performance • Dynamic branch predictors can be quite accurate (>95%) and avoid most control hazards • Branch History Tables (BHTs) just predict direction (later in pipeline) – Just need a few bits per entry (2 bits gives hysteresis) – Need to decode instruction bits to determine whether this is a branch and what the target address is • Branch Target Buffer (BTB) predicts whether a branch, and target address – Needs PC tag, predicted Next-PC, and direction – Just needs PC of instruction to predict target of branch (if any) • Return address stack: special form of BTB used to predict subroutine return addresses 4/1/2008 CS 152 -Spring’ 08 2

“Data in ROB” Design (HP PA 8000, Pentium Pro, Core 2 Duo) Register File holds only committed state Ins# use exec op p 1 src 1 p 2 src 2 pd dest data Reorder buffer Load Unit FU FU FU Store Unit t 1 t 2. . tn Commit < t, result > • On dispatch into ROB, ready sources can be in regfile or in ROB dest (copied into src 1/src 2 if ready before dispatch) • On completion, write to dest field and broadcast to src fields. • On issue, read from ROB src fields 4/1/2008 CS 152 -Spring’ 08 3

Unified Physical Register File (MIPS R 10 K, Alpha 21264, Pentium 4) r 1 r 2 ti tj Rename Table t 1 t 2. tn Snapshots for mispredict recovery Load Unit FU FU FU (ROB not shown) Reg File FU Store Unit < t, result > • One regfile for both committed and speculative values (no data in ROB) • During decode, instruction result allocated new physical register, source regs translated to physical regs through rename table • Instruction reads data from regfile at start of execute (not in decode) • Write-back updates reg. busy bits on instructions in ROB (assoc. search) • Snapshots of rename table taken at every branch to recover mispredicts • On exception, renaming undone in reverse order of issue (MIPS R 10000) 4/1/2008 CS 152 -Spring’ 08 4

Pipeline Design with Physical Regfile Branch Resolution kill Branch Prediction PC Fetch kill Decode & Rename Update predictors kill Out-of-Order Reorder Buffer In-Order Commit In-Order Physical Reg. File Branch ALU MEM Unit Store Buffer D$ Execute 4/1/2008 CS 152 -Spring’ 08 5

Lifetime of Physical Registers • Physical regfile holds committed and speculative values • Physical registers decoupled from ROB entries (no data in ROB) ld r 1, (r 3) add r 3, r 1, #4 sub r 6, r 7, r 9 add r 3, r 6 ld r 6, (r 1) add r 6, r 3 st r 6, (r 1) ld r 6, (r 11) Rename ld P 1, (Px) add P 2, P 1, #4 sub P 3, Py, Pz add P 4, P 2, P 3 ld P 5, (P 1) add P 6, P 5, P 4 st P 6, (P 1) ld P 7, (Pw) When can we reuse a physical register? When next write of same architectural register commits 4/1/2008 CS 152 -Spring’ 08 6

Physical Register Management R 0 R 1 R 2 R 3 R 4 R 5 R 6 R 7 Rename Table P 8 P 7 P 5 P 6 4/1/2008 P 0 P 1 P 2 P 3 P 4 P 5 P 6 P 7 P 8 <R 6> <R 7> <R 3> <R 1> p p p 2 Rd Free List P 0 P 1 P 3 P 2 P 4 ld r 1, 0(r 3) add r 3, r 1, #4 sub r 6, r 7, r 6 add r 3, r 6 ld r 6, 0(r 1) Pn ROB use ex Physical Regs op p 1 PR 2 CS 152 -Spring’ 08 LPRd (LPRd requires third read port on Rename Table for each instruction) 7

Physical Register Management R 0 R 1 R 2 R 3 R 4 R 5 R 6 R 7 Rename Table P 8 P 0 P 7 P 5 P 6 4/1/2008 P 0 P 1 P 2 P 3 P 4 P 5 P 6 P 7 P 8 <R 6> <R 7> <R 3> <R 1> p p p 2 Rd r 1 Free List P 0 P 1 P 3 P 2 P 4 ld r 1, 0(r 3) add r 3, r 1, #4 sub r 6, r 7, r 6 add r 3, r 6 ld r 6, 0(r 1) Pn ROB use ex x Physical Regs op ld p 1 p PR 1 P 7 PR 2 CS 152 -Spring’ 08 LPRd P 8 PRd P 0 8

Physical Register Management R 0 R 1 R 2 R 3 R 4 R 5 R 6 R 7 Rename Table P 8 P 0 P 7 P 1 P 5 P 6 4/1/2008 P 0 P 1 P 2 P 3 P 4 P 5 P 6 P 7 P 8 <R 6> <R 7> <R 3> <R 1> p p p 2 Rd r 1 r 3 Free List P 0 P 1 P 3 P 2 P 4 ld r 1, 0(r 3) add r 3, r 1, #4 sub r 6, r 7, r 6 add r 3, r 6 ld r 6, 0(r 1) Pn ROB use ex x x Physical Regs op p 1 ld p add PR 1 P 7 P 0 PR 2 CS 152 -Spring’ 08 LPRd P 8 P 7 PRd P 0 P 1 9

Physical Register Management R 0 R 1 R 2 R 3 R 4 R 5 R 6 R 7 Rename Table P 8 P 0 P 7 P 1 P 5 P 3 P 6 4/1/2008 P 0 P 1 P 2 P 3 P 4 P 5 P 6 P 7 P 8 <R 6> <R 7> <R 3> <R 1> p p p 2 PR 2 p P 5 Rd r 1 r 3 r 6 Free List P 0 P 1 P 3 P 2 P 4 ld r 1, 0(r 3) add r 3, r 1, #4 sub r 6, r 7, r 6 add r 3, r 6 ld r 6, 0(r 1) Pn ROB use ex x Physical Regs op p 1 ld p add sub p PR 1 P 7 P 0 P 6 CS 152 -Spring’ 08 LPRd P 8 P 7 P 5 PRd P 0 P 1 P 3 10

Physical Register Management R 0 R 1 R 2 R 3 R 4 R 5 R 6 R 7 Rename Table P 8 P 0 P 7 P 1 P 2 P 5 P 3 P 6 x x 4/1/2008 P 0 P 1 P 2 P 3 P 4 P 5 P 6 P 7 P 8 <R 6> <R 7> <R 3> <R 1> p p p 2 PR 2 Rd p P 5 P 3 Free List P 0 P 1 P 3 P 2 P 4 ld r 1, 0(r 3) add r 3, r 1, #4 sub r 6, r 7, r 6 add r 3, r 6 ld r 6, 0(r 1) Pn ROB use ex Physical Regs op p 1 ld p add sub p add PR 1 P 7 P 0 P 6 P 1 r 3 r 6 r 3 CS 152 -Spring’ 08 LPRd P 8 P 7 P 5 P 1 PRd P 0 P 1 P 3 P 2 11

Physical Register Management R 0 R 1 R 2 R 3 R 4 R 5 R 6 R 7 Rename Table P 8 P 0 P 7 P 1 P 2 P 5 P 3 P 4 P 6 4/1/2008 P 0 P 1 P 2 P 3 P 4 P 5 P 6 P 7 P 8 <R 6> <R 7> <R 3> <R 1> p p p 2 PR 2 p P 5 P 3 Rd r 1 r 3 r 6 Free List P 0 P 1 P 3 P 2 P 4 ld r 1, 0(r 3) add r 3, r 1, #4 sub r 6, r 7, r 6 add r 3, r 6 ld r 6, 0(r 1) Pn ROB use ex x x Physical Regs op p 1 ld p add sub p add ld PR 1 P 7 P 0 P 6 P 1 P 0 CS 152 -Spring’ 08 LPRd P 8 P 7 P 5 P 1 P 3 PRd P 0 P 1 P 3 P 2 P 4 12

Physical Register Management R 0 R 1 R 2 R 3 R 4 R 5 R 6 R 7 Rename Table P 8 P 0 P 7 P 1 P 2 P 5 P 3 P 4 P 6 x x x 4/1/2008 P 0 P 1 P 2 P 3 P 4 P 5 P 6 P 7 P 8 <R 1> Free List p <R 6> <R 7> <R 3> <R 1> p p p 2 PR 2 Rd p P 5 P 3 P 0 P 1 P 3 P 2 P 4 ld r 1, 0(r 3) add r 3, r 1, #4 sub r 6, r 7, r 6 add r 3, r 6 ld r 6, 0(r 1) P 8 Pn ROB use ex Physical Regs op p 1 ld p add p sub p add ld p PR 1 P 7 P 0 P 6 P 1 P 0 r 1 r 3 r 6 CS 152 -Spring’ 08 LPRd P 8 P 7 P 5 P 1 P 3 PRd P 0 P 1 P 3 P 2 P 4 Execute & Commit 13

Physical Register Management R 0 R 1 R 2 R 3 R 4 R 5 R 6 R 7 Rename Table P 8 P 0 P 7 P 1 P 2 P 5 P 3 P 4 P 6 x x x x 4/1/2008 P 0 P 1 P 2 P 3 P 4 P 5 P 6 P 7 P 8 Free List <R 1> <R 3> p p <R 6> <R 7> <R 3> p p 2 PR 2 Rd p P 5 P 3 P 0 P 1 P 3 P 2 P 4 ld r 1, 0(r 3) add r 3, r 1, #4 sub r 6, r 7, r 6 add r 3, r 6 ld r 6, 0(r 1) P 8 P 7 Pn ROB use ex Physical Regs op p 1 ld p add p sub p add p ld p PR 1 P 7 P 0 P 6 P 1 P 0 r 1 r 3 r 6 CS 152 -Spring’ 08 LPRd P 8 P 7 P 5 P 1 P 3 PRd P 0 P 1 P 3 P 2 P 4 Execute & Commit 14

CS 152 Administrivia • New shifted schedule - see website for details • Lab 4, PS 4, due Tuesday April 8 – PRIZE (TBD) for winners in both unlimited and realistic categories of branch predictor contest • Quiz 4, Thursday April 10 • Quiz 5, Thursday April 24 • Quiz 6, Thursday May 8 (last day of class) 4/1/2008 CS 152 -Spring’ 08 15

Reorder Buffer Holds Active Instruction Window … (Older instructions) ld r 1, (r 3) add r 3, r 1, r 2 sub r 6, r 7, r 9 add r 3, r 6 ld r 6, (r 1) add r 6, r 3 st r 6, (r 1) ld r 6, (r 1) … (Newer instructions) Commit Execute Fetch Cycle t 4/1/2008 … ld r 1, (r 3) add r 3, r 1, sub r 6, r 7, add r 3, ld r 6, (r 1) add r 6, st r 6, (r 1) ld r 6, (r 1) … r 2 r 9 r 6 r 3 Cycle t + 1 CS 152 -Spring’ 08 16

Superscalar Register Renaming • During decode, instructions allocated new physical destination register • Source operands renamed to physical register with newest value • Execution unit only sees physical register numbers Update Mapping Op Dest Src 1 Src 2 Write Ports Inst 1 Read Addresses Register Free List Rename Table Read Data Op PDest PSrc 1 PSrc 2 4/1/2008 Op Dest Src 1 Src 2 Inst 2 Op PDest PSrc 1 PSrc 2 Does this work? CS 152 -Spring’ 08 17

Superscalar Register Renaming Update Mapping Write Ports Inst 1 Op Dest Src 1 Src 2 Read Addresses =? Rename Table Read Data Must check for RAW hazards between instructions issuing in same cycle. Can be done in parallel with rename Op PDest PSrc 1 PSrc 2 lookup. Inst 2 Op Dest Src 1 Src 2 =? Register Free List Op PDest PSrc 1 PSrc 2 MIPS R 10 K renames 4 serially-RAW-dependent insts/cycle 4/1/2008 CS 152 -Spring’ 08 18

Memory Dependencies st r 1, (r 2) ld r 3, (r 4) When can we execute the load? 4/1/2008 CS 152 -Spring’ 08 19

In-Order Memory Queue • Execute all loads and stores in program order => Load and store cannot leave ROB for execution until all previous loads and stores have completed execution • Can still execute loads and stores speculatively, and out-of-order with respect to other instructions 4/1/2008 CS 152 -Spring’ 08 20

Conservative O-o-O Load Execution st r 1, (r 2) ld r 3, (r 4) • Split execution of store instruction into two phases: address calculation and data write • Can execute load before store, if addresses known and r 4 != r 2 • Each load address compared with addresses of all previous uncommitted stores (can use partial conservative check i. e. , bottom 12 bits of address) • Don’t execute load if any previous store address not known (MIPS R 10 K, 16 entry address queue) 4/1/2008 CS 152 -Spring’ 08 21

Address Speculation st r 1, (r 2) ld r 3, (r 4) • Guess that r 4 != r 2 • Execute load before store address known • Need to hold all completed but uncommitted load/store addresses in program order • If subsequently find r 4==r 2, squash load and all following instructions => Large penalty for inaccurate address speculation 4/1/2008 CS 152 -Spring’ 08 22

Memory Dependence Prediction (Alpha 21264) st r 1, (r 2) ld r 3, (r 4) • Guess that r 4 != r 2 and execute load before store • If later find r 4==r 2, squash load and all following instructions, but mark load instruction as store-wait • Subsequent executions of the same load instruction will wait for all previous stores to complete • Periodically clear store-wait bits 4/1/2008 CS 152 -Spring’ 08 23

Speculative Loads / Stores Just like register updates, stores should not modify the memory until after the instruction is committed - A speculative store buffer is a structure introduced to hold speculative store data. 4/1/2008 CS 152 -Spring’ 08 24

Speculative Store Buffer V V V S S S Load Address Tag Tag Tag Data Data L 1 Data Cache Tags Store Commit Path Data Load Data • On store execute: – mark entry valid and speculative, and save data and tag of instruction. • On store commit: – clear speculative bit and eventually move data to cache • On store abort: – clear valid bit 4/1/2008 CS 152 -Spring’ 08 25

Speculative Store Buffer V V V • • S S S Tag Tag Tag Load Address Data Data L 1 Data Cache Tags Data Store Commit Path Load Data If data in both store buffer and cache, which should we use? Speculative store buffer If same address in store buffer twice, which should we use? Youngest store older than load 4/1/2008 CS 152 -Spring’ 08 26

Datapath: Branch Prediction and Speculative Execution Update predictors Branch Prediction kill Branch Resolution kill PC Fetch Decode & Rename Reorder Buffer Commit Reg. File 4/1/2008 Branch ALU MEM Unit Execute CS 152 -Spring’ 08 Store Buffer D$ 27

Acknowledgements • These slides contain material developed and copyright by: – – – Arvind (MIT) Krste Asanovic (MIT/UCB) Joel Emer (Intel/MIT) James Hoe (CMU) John Kubiatowicz (UCB) David Patterson (UCB) • MIT material derived from course 6. 823 • UCB material derived from course CS 252 4/1/2008 CS 152 -Spring’ 08 28