Evolution of Processor Performance So far we examined

  • Slides: 45
Download presentation
Evolution of Processor Performance So far we examined static & dynamic techniques to improve

Evolution of Processor Performance So far we examined static & dynamic techniques to improve the performance of scalar (singleissue) pipelined CPU designs including: static & dynamic scheduling, static & dynamic branch predication. Even with these improvements, the restriction of issuing a single instruction per cycle still limits the ideal CPI = 1 Pipelined (single issue) Multi-cycle CPI Source: John P. Chen, Intel Labs (Ch 3. 6, 3. 7, 4. 3, 4. 5) > 10 1. 1 -10 0. 5 - 1. 1 Multiple Issue (CPI <1) Superscalar/VLIW/SMT . 35 -. 5 (? ) We examine next the two approaches to achieve a CPI < 1 by issuing multiple instructions per cycle: • Superscalar CPUs • Very Long Instruction Word (VLIW) CPUs. EECC 551 - Shaaban #1 lec # 6 Spring 2004 3 -29 -2004

Parallelism in Microprocessor VLSI Generations Multiple micro-operations per cycle Single-issue Pipelined CPI =1 Superscalar

Parallelism in Microprocessor VLSI Generations Multiple micro-operations per cycle Single-issue Pipelined CPI =1 Superscalar /VLIW CPI <1 Simultaneous Multithreading SMT: e. g. Intel’s Hyper-threading Chip-Multiprocessors (CMPs) e. g IBM Power 4 Not Pipelined CPI >> 1 Thread Level Parallelism (TLP) Instruction Level Parallelism (ILP) Pipelining single issue Multiple Instruction Issue EECC 551 - Shaaban #2 lec # 6 Spring 2004 3 -29 -2004

Multiple Instruction Issue: CPI < 1 • To improve a pipeline’s CPI to be

Multiple Instruction Issue: CPI < 1 • To improve a pipeline’s CPI to be better [less] than one, and to utilize Instruction Level Parallelism (ILP) better, a number of instructions have to be issued in the same pipeline cycle. • Multiple instruction issue processors are of two types: – Superscalar: A number of instructions (2 -8) is issued in the same cycle, scheduled statically by the compiler or -more commonlydynamically (Tomasulo). • Power. PC, Sun Ultra. Sparc, Alpha, HP 8000, Intel PII, III, 4. . . – VLIW (Very Long Instruction Word): A fixed number of instructions (3 -6) are formatted as one long instruction word or packet (statically scheduled by the compiler). – Example: Explicitly Parallel Instruction Computer (EPIC) • Originally a joint HP/Intel effort. • ISA: Intel Architecture-64 (IA-64) 64 -bit address: • First CPU: Itanium, Q 1 2001. • Limitations of the approaches: – Available ILP in the program (both). – Specific hardware implementation difficulties (superscalar). – VLIW optimal compiler design issues. CPI < 1 or Instructions Per Cycle (IPC) > 1 EECC 551 - Shaaban #3 lec # 6 Spring 2004 3 -29 -2004

Multiple Instruction Issue: Superscalar Vs. VLIW • Usually dynamically scheduled (Tomasulo) • Statically scheduled.

Multiple Instruction Issue: Superscalar Vs. VLIW • Usually dynamically scheduled (Tomasulo) • Statically scheduled. • Complex compiler design. • Complex scheduling hardware. • Simplified Hardware for decoding, issuing instructions. • Smaller code size. • No Interlock Hardware (compiler checks? ) • Binary compatibility across generations of hardware. • More ISA registers (no register renaming) , but simplified hardware for register ports. EECC 551 - Shaaban #4 lec # 6 Spring 2004 3 -29 -2004

Simple Statically Scheduled Superscalar Pipeline • • Two instructions can be issued per cycle

Simple Statically Scheduled Superscalar Pipeline • • Two instructions can be issued per cycle (two-issue superscalar). One of the instructions is integer (including load/store, branch). The other instruction is a floating-point operation. – This restriction reduces the complexity of hazard checking. Hardware must fetch and decode two instructions per cycle. Then it determines whether zero (a stall), one or two instructions can be issued (in decode stage) per cycle. Instruction Type 1 2 3 Integer Instruction IF IF ID ID IF IF EX EX ID ID IF IF FP Instruction Integer Instruction FP Instruction 4 MEM EX EX EX ID ID IF IF 5 WB EX MEM EX EX EX ID ID Two-issue statically scheduled pipeline in operation FP instructions assumed to be adds (EX takes 3 cycles) (Ch. 3. 6) 6 Ideal CPI = 0. 5 Ideal Instructions Per Cycle (IPC) = 2 7 WB WB EX MEM EX EX EX WB WB EX MEM EX 8 WB WB EX EECC 551 - Shaaban #5 lec # 6 Spring 2004 3 -29 -2004

Intel IA-64: VLIW “Explicitly Parallel Instruction Computing (EPIC)” • Three 41 -bit instructions in

Intel IA-64: VLIW “Explicitly Parallel Instruction Computing (EPIC)” • Three 41 -bit instructions in 128 bit “Groups” or bundles; an instruction bundle template field (5 -bits) determines if instructions are dependent or independent and statically specifies the functional units to used by the instructions: – Smaller code size than old VLIW, larger than x 86/RISC – Groups can be linked to show dependencies of more than three instructions. • 128 integer registers + 128 floating point registers • Hardware checks dependencies (interlocks Þ binary compatibility over time) • Predicated execution: An implementation of conditional instructions used to reduce the number of conditional branches used in the generated code Þ larger basic block size • IA-64 : Name given to instruction set architecture (ISA). • Itanium : Name of the first implementation (2001). EECC 551 - Shaaban #6 lec # 6 Spring 2004 3 -29 -2004

Intel/HP EPIC VLIW Approach original source code Sequential Code Expose Instruction Parallelism compiler Instruction

Intel/HP EPIC VLIW Approach original source code Sequential Code Expose Instruction Parallelism compiler Instruction Dependency Analysis Dependency Graph Exploit Instruction Parallelism: Generate VLIWs Optimize 128 -bit bundle 127 0 Instruction 2 Instruction 1 Instruction 0 Template 41 bits 5 bits EECC 551 - Shaaban #7 lec # 6 Spring 2004 3 -29 -2004

IA-64 Instruction Types Instruction Type Description Execution Unit Type A Integer ALU I-unit or

IA-64 Instruction Types Instruction Type Description Execution Unit Type A Integer ALU I-unit or M-unit I Non-integer ALU I-unit M Memory M-unit F Floating Point F-unit B Branch B-unit L+X Extended I-unit/B-unit EECC 551 - Shaaban #8 lec # 6 Spring 2004 3 -29 -2004

IA-64 Template Use • The template specifies the functional units for the three operations

IA-64 Template Use • The template specifies the functional units for the three operations in the instruction. – Part of static scheduling • Possible instruction combinations: – – – – – M-unit, I-unit M-unit, L-unit, X-unit M-unit, I-unit M-unit, F-unit, I-unit M-unit, F-unit M-unit, I-unit, B-unit M-unit, B-unit, B-unit M-unit, F-unit, B-unit EECC 551 - Shaaban #9 lec # 6 Spring 2004 3 -29 -2004

IA-64 Instruction Format Example: A-Unit Instruction Format Register-register format: 40 37 36 8 4

IA-64 Instruction Format Example: A-Unit Instruction Format Register-register format: 40 37 36 8 4 1 35 34 33 32 29 28 27 26 20 19 13 12 6 5 0 X 2 a Ve X 4 X 2 b R 3 R 2 R 1 qp 2 1 4 2 7 7 7 6 § 8 is the major opcode for this instruction type. §X 2 a, X 2 b, Ve, and X 4 are opcode extensions. §qp is the predicate register assigned to this operation (64 such flags) EECC 551 - Shaaban #10 lec # 6 Spring 2004 3 -29 -2004

IA-64 Instruction Format Example: A-Unit Instruction Format 14 -bit Immediate format: 40 37 36

IA-64 Instruction Format Example: A-Unit Instruction Format 14 -bit Immediate format: 40 37 36 35 34 33 32 27 26 20 19 13 12 6 5 0 8 s X 2 a Ve imm 6 d R 3 imm 7 b R 1 qp 4 1 2 1 6 7 7 7 6 §X 2 a and Ve are opcode extensions. §qp is the predicate register assigned to this operation. §The immediate value is made up of s, imm 6 d, and imm 7 b. §Other formats available for 8 -bit and 22 -bit immediates. EECC 551 - Shaaban #11 lec # 6 Spring 2004 3 -29 -2004

IA-64 Instruction Format Example: M-Unit Instruction Format Integer Load/Store operations: 40 37 36 35

IA-64 Instruction Format Example: M-Unit Instruction Format Integer Load/Store operations: 40 37 36 35 30 29 28 27 26 20 4 m X 6 hint x R 3 4 1 2 2 1 7 19 13 7 12 6 5 0 R 1 qp 7 6 § m and x determine the subset of load/store operations. §Bits 35: 32 give special directives for LAT, cache, etc. §Bits 31: 30 tell the size of the load (1, 2, 4, or 8 bytes). §Hint allows for system pre-fetching of data by specifying if temporal locality exists for this data. EECC 551 - Shaaban #12 lec # 6 Spring 2004 3 -29 -2004

IA-64 Instruction Format Example: B-Unit Instruction Format IP-relative branch: 40 37 36 35 opcode

IA-64 Instruction Format Example: B-Unit Instruction Format IP-relative branch: 40 37 36 35 opcode s d 4 1 1 34 33 26 20 12 wh imm 20 b p 2 20 1 11 9 3 8 6 5 0 btype qp 3 6 §Opcode 0 - indirect branch, 1 - indirect call, 4 - IP-relative branch, 5 - IP-relative call §btype sub-category of branch type, qp- branch condition §d- cache de-allocation, wh- branch hint, s & imm 20 b give offset §p pre-fetch 0 -few, or 1 -many inst. following the target EECC 551 - Shaaban #13 lec # 6 Spring 2004 3 -29 -2004

Unrolled Loop Example for Scalar (single-issue) Pipeline 1 Loop: 2 3 4 5 6

Unrolled Loop Example for Scalar (single-issue) Pipeline 1 Loop: 2 3 4 5 6 7 8 9 10 11 12 13 14 L. D ADD. D S. D DADDUI S. D BNE S. D F 0, 0(R 1) F 6, -8(R 1) F 10, -16(R 1) F 14, -24(R 1) F 4, F 0, F 2 F 8, F 6, F 2 F 12, F 10, F 2 F 16, F 14, F 2 F 4, 0(R 1) F 8, -8(R 1) R 1, #-32 F 12, 16(R 1) R 1, R 2, LOOP F 16, 8(R 1) L. D to ADD. D: 1 Cycle ADD. D to S. D: 2 Cycles ; 8 -32 = -24 14 clock cycles, or 3. 5 per original iteration (result) (unrolled four times) EECC 551 - Shaaban #14 lec # 6 Spring 2004 3 -29 -2004

Loop Unrolling in 2 -way Superscalar Pipeline: (1 Integer, 1 FP/Cycle) Integer instruction FP

Loop Unrolling in 2 -way Superscalar Pipeline: (1 Integer, 1 FP/Cycle) Integer instruction FP instruction Clock cycle Loop: L. D F 0, 0(R 1) 1 L. D F 6, -8(R 1) 2 L. D F 10, -16(R 1) ADD. D F 4, F 0, F 2 3 L. D F 14, -24(R 1) ADD. D F 8, F 6, F 2 4 L. D F 18, -32(R 1) ADD. D F 12, F 10, F 2 5 S. D F 4, 0(R 1) ADD. D F 16, F 14, F 2 6 S. D F 8, -8(R 1) ADD. D F 20, F 18, F 2 7 S. D F 12, -16(R 1) 8 DADDUI R 1, #-40 9 S. D F 16, -24(R 1) 10 BNE R 1, R 2, LOOP 11 SD -32(R 1), F 20 12 • Unrolled 5 times to avoid delays and expose more ILP (unrolled one more time) • 12 cycles, or 12/5 = 2. 4 cycles per iteration (3. 5/2. 4= 1. 5 X faster than scalar) • CPI = 12/ 17 =. 7 worse than ideal CPI =. 5 because 7 issue slots are wasted EECC 551 - Shaaban #15 lec # 6 Spring 2004 3 -29 -2004

Loop Unrolling in VLIW Pipeline (2 Memory, 2 FP, 1 Integer / Cycle) Memory

Loop Unrolling in VLIW Pipeline (2 Memory, 2 FP, 1 Integer / Cycle) Memory FP FP reference 1 reference 2 L. D F 0, 0(R 1) L. D F 6, -8(R 1) L. D F 10, -16(R 1) L. D F 14, -24(R 1) L. D F 18, -32(R 1) L. D F 22, -40(R 1) L. D F 26, -48(R 1) Int. op/ Clock operation 1 op. 2 branch 1 2 ADD. D F 4, F 0, F 2 ADD. D F 12, F 10, F 2 ADD. D F 8, F 6, F 2 ADD. D F 16, F 14, F 2 3 4 5 ADD. D F 20, F 18, F 2 ADD. D F 24, F 22, F 2 S. D F 4, 0(R 1) S. D F 8, -8(R 1) ADD. D F 28, F 26, F 2 S. D F 12, -16(R 1) S. D F 16, -24(R 1) DADDUI R 1, #-56 S. D F 24, 16(R 1) 8 9 6 7 S. D F 20, 24(R 1) S. D F 28, 8(R 1) BNE R 1, R 2, LOOP Unrolled 7 times to avoid delays and expose more ILP 7 results in 9 cycles, or 1. 3 cycles per iteration (2. 4/1. 3 =1. 8 X faster than 2 -issue superscalar, 3. 5/1. 3 = 2. 7 X faster than scalar) Average: about 23/9 = 2. 55 IPC (instructions per clock cycle) Ideal IPC =5, CPI =. 39 Ideal CPI =. 2 thus about 50% efficiency, 22 issue slots are wasted Note: Needs more registers in VLIW (15 vs. 6 in Superscalar) EECC 551 - Shaaban (In chapter 4. 3 pages 317 -318) EECC 551 - Shaaban #16 lec # 6 Spring 2004 3 -29 -2004

Superscalar Dynamic Scheduling • • The Tomasulo dynamic scheduling algorithm is extended to issue

Superscalar Dynamic Scheduling • • The Tomasulo dynamic scheduling algorithm is extended to issue more than one instruction per cycle. However the restriction that instructions must issue in program order still holds to avoid violating instruction dependencies. – The result of issuing multiple instructions in one cycle should be the same as if they were single-issued, one instruction per cycle. • • • How to issue two instructions and keep in-order instruction issue for Tomasulo? Simplest Method: Restrict Type of Instructions Issued Per Cycle To simplify the issue logic, issue one integer + one floating-point instruction per cycle (for a 2 -way superscalar). – 1 Tomasulo control for integer, 1 for floating point. – Only FP loads might cause a dependency between integer and FP issue: • Replace load reservation station with a load queue; operands must be read in the order they are fetched. • Load checks addresses in Store Queue to avoid RAW violation • Store checks addresses in Load Queue to avoid WAR, WAW. (the above load/store queue checking is also applicable to single-issue Tomasulo to take care of memory RAW, WAR, WAW). EECC 551 - Shaaban #17 lec # 6 Spring 2004 3 -29 -2004

Superscalar Dynamic Scheduling Three techniques can be used to support multiple instruction issue in

Superscalar Dynamic Scheduling Three techniques can be used to support multiple instruction issue in Tomasulo without putting restrictions on the type of instructions issued per cycle: 1 Issue at a higher clock rate so that issue remains in order. – For example for a 2 -Issue supercalar issue at 2 X Clock Rate. Issue First Instruction Issue Second Instruction One Cycle 2 Widen the issue logic to handle multiple instruction issue – All possible dependencies between instructions to be issues are detected at once and the result of the multiple issue matches in-order issue Check Instruction Dependecies Issue Both Instructions 2 -Issue supercalar 0, 1 or 2 instructions issued per cycle for either method One Cycle EECC 551 - Shaaban #18 lec # 6 Spring 2004 3 -29 -2004

Superscalar Dynamic Scheduling 3 To avoid increasing the CPU clock cycle time in the

Superscalar Dynamic Scheduling 3 To avoid increasing the CPU clock cycle time in the last two approaches, multiple instruction issue can be spilt into two pipelined issue stages: – Issue Stage One: Decide how many instructions can issue simultaneously checking dependencies within the group of instructions to be issued, ignoring instructions already issued. – Issue Stage Two: Examine hazards among the selected instructions from the group and the those already issued. • • This approach is usually used in dynamically-scheduled wide superscalars that can issue four or more instructions per cycle. Splitting the issue into two pipelined staged increases the CPU pipeline depth and increases branch penalties – This increases the importance of accurate dynamic branch prediction methods. Further pipelining of issue stages beyond two stages may be necessary as CPU clock rates are increased. The dynamic scheduling control logic for superscalars is generally very complex growing at least quadratically with issue width. – E. g 4 wide superscalar -> 4 x 4 = 16 times complexity EECC 551 - Shaaban #19 lec # 6 Spring 2004 3 -29 -2004

Multiple Instruction Issue with Dynamic Scheduling Example Assumptions: Restricted 2 -way superscalar: 1 integer,

Multiple Instruction Issue with Dynamic Scheduling Example Assumptions: Restricted 2 -way superscalar: 1 integer, 1 FP Issue Per Cycle A sufficient number of reservation stations is available. One integer unit (for ALU, effective address) One integer unit for branch condition 2 CDBs Execution cycles: Integer: 1 cycle Load: 2 cycles (1 ex + 1 mem) FP add: 3 cycles Any instruction following a branch cannot start execution until after branch condition is evaluated in EX Branches are single issued, no delayed branch, perfect branch prediction Example on page 221 EECC 551 - Shaaban #20 lec # 6 Spring 2004 3 -29 -2004

Multiple Instruction Issue with Dynamic Scheduling Example EECC 551 - Shaaban #21 lec #

Multiple Instruction Issue with Dynamic Scheduling Example EECC 551 - Shaaban #21 lec # 6 Spring 2004 3 -29 -2004

Three Loop Iterations on Restricted 2 -way Superscalar Tomasulo (Start) Only one CDB is

Three Loop Iterations on Restricted 2 -way Superscalar Tomasulo (Start) Only one CDB is actually needed in this case. EECC 551 - Shaaban #22 lec # 6 Spring 2004 3 -29 -2004

Resource Usage Table for Example: EECC 551 - Shaaban #23 lec # 6 Spring

Resource Usage Table for Example: EECC 551 - Shaaban #23 lec # 6 Spring 2004 3 -29 -2004

Multiple Instruction Issue with Dynamic Scheduling Example Assumptions: The same loop in previous example

Multiple Instruction Issue with Dynamic Scheduling Example Assumptions: The same loop in previous example On restricted 2 -way superscalar: 1 integer, 1 FP Issue Per Cycle A sufficient number of reservation stations is available. Two integer units one for ALU, one for effective address One integer unit for branch condition 2 CDBs Execution cycles: Integer: 1 cycle Load: 2 cycles (1 ex + 1 mem) FP add: 3 cycles Any instruction following a branch cannot start execution until after branch condition is evaluated Branches are single issued, no delayed branch, perfect branch prediction Example on page 223 EECC 551 - Shaaban #24 lec # 6 Spring 2004 3 -29 -2004

Same three loop Iterations on Restricted 2 -way Superscalar Tomasulo but with Two integer

Same three loop Iterations on Restricted 2 -way Superscalar Tomasulo but with Two integer units (one for ALU, one for effective address) (Start) (page 224) EECC 551 - Shaaban #25 lec # 6 Spring 2004 3 -29 -2004

Resource Usage Table for Example: (page 225) EECC 551 - Shaaban #26 lec #

Resource Usage Table for Example: (page 225) EECC 551 - Shaaban #26 lec # 6 Spring 2004 3 -29 -2004

Multiple Instruction Issue Challenges • While a two-issue single Integer/FP split is simple in

Multiple Instruction Issue Challenges • While a two-issue single Integer/FP split is simple in hardware, we get a CPI of 0. 5 only for programs with: – Exactly 50% FP operations – No hazards of any type. • If more instructions issue at the same time, greater difficulty of decode and issue operations arise: – Even for a 2 -issue superscalar machine, we have to examine 2 opcodes, 6 register specifiers, and decide if 0, 1 or 2 instructions can issue. • VLIW: tradeoff instruction space for simple decoding – The long instruction word has room for many operations. – By definition, all the operations the compiler puts in the long instruction word are independent => execute in parallel – E. g. 2 integer operations, 2 FP ops, 2 Memory refs, 1 branch • 16 to 24 bits per field => 7*16 or 112 bits to 7*24 or 168 bits wide – Need compiling technique that schedules across several branches. EECC 551 - Shaaban #27 lec # 6 Spring 2004 3 -29 -2004

Limits to Multiple Instruction Issue Machines • Inherent limitations of ILP: – If 1

Limits to Multiple Instruction Issue Machines • Inherent limitations of ILP: – If 1 branch exist for every 5 instruction : How to keep a 5 -way VLIW busy? – Latencies of unit adds complexity to the many operations that must be scheduled every cycle. – For maximum performance multiple instruction issue requires about: Pipeline Depth x No. Functional Units independent instructions per cycle. • Hardware implementation complexities: – Duplicate FUs for parallel execution are needed, more CDBs. – More instruction bandwidth is essential. – Increased number of ports to Register File (datapath bandwidth): • VLIW example needs 7 read and 3 write for Int. Reg. & 5 read and 3 write for FP reg – Increased ports to memory (to improve memory bandwidth). – Superscalar decoding complexity may impact pipeline clock rate, depth. EECC 551 - Shaaban #28 lec # 6 Spring 2004 3 -29 -2004

Superscalar Architectures: Issue Slot Waste Classification • Empty or wasted issue slots can be

Superscalar Architectures: Issue Slot Waste Classification • Empty or wasted issue slots can be defined as either vertical waste or horizontal waste: – Vertical waste is introduced when the processor issues no instructions in a cycle. – Horizontal waste occurs when not all issue slots can be filled in a cycle. Ex: 4 -Issue Superscalar Ideal IPC =4 Ideal CPI =. 25 Instructions Per Cycle = IPC = 1/CPI EECC 551 - Shaaban #29 lec # 6 Spring 2004 3 -29 -2004

Sources of Unused Issue Cycles in an 8 -issue Superscalar Processor. Ideal Instructions Per

Sources of Unused Issue Cycles in an 8 -issue Superscalar Processor. Ideal Instructions Per Cycle, IPC = 8 Here real IPC about 1. 5 Processor busy represents the utilized issue slots; all others represent wasted issue slots. 61% of the wasted cycles are vertical waste, the remainder are horizontal waste. Workload: SPEC 92 benchmark suite. Source: Simultaneous Multithreading: Maximizing On-Chip Parallelism Dean Tullsen et al. , Proceedings of the 22 rd Annual International Symposium on Computer Architecture, June 1995, pages 392 -403. EECC 551 - Shaaban #30 lec # 6 Spring 2004 3 -29 -2004

Superscalar Architectures: All possible causes of wasted issue slots, and latency-hiding or latency reducing

Superscalar Architectures: All possible causes of wasted issue slots, and latency-hiding or latency reducing techniques that can reduce the number of cycles wasted by each cause. Source: Simultaneous Multithreading: Maximizing On-Chip Parallelism Dean Tullsen et al. , Proceedings of the 22 rd Annual International Symposium on Computer Architecture, June 1995, pages 392 -403. EECC 551 - Shaaban #31 lec # 6 Spring 2004 3 -29 -2004

Hardware Support for Extracting More Parallelism • Compiler ILP techniques (loop-unrolling, software Pipelining etc.

Hardware Support for Extracting More Parallelism • Compiler ILP techniques (loop-unrolling, software Pipelining etc. ) are not effective to uncover maximum ILP when branch behavior is not well known at compile time. • Hardware ILP techniques: – Conditional or Predicted Instructions: An extension to the instruction set with instructions that turn into no-ops if a condition is not valid at run time. – Speculation: An instruction is executed before the processor knows that the instruction should execute to avoid control dependence stalls: • Static Speculation by the compiler with hardware support: – The compiler labels an instruction as speculative and the hardware helps by ignoring the outcome of incorrectly speculated instructions. – Conditional instructions provide limited speculation. • Dynamic Hardware-based Speculation: (Ch. 3. 7) – Uses dynamic branch-prediction to guide the speculation process. – Dynamic scheduling and execution continued passed a conditional branch in the predicted branch direction. EECC 551 - Shaaban #32 lec # 6 Spring 2004 3 -29 -2004

Conditional or Predicted Instructions • Avoid branch prediction by turning branches into conditionally-executed instructions

Conditional or Predicted Instructions • Avoid branch prediction by turning branches into conditionally-executed instructions (helps increase average size of basic blocks): if (x) then (A = B op C) else NOP – If false, then neither store result nor cause exception: instruction is annulled (turned into NOP). – Expanded ISA of Alpha, MIPS, Power. PC, SPARC have conditional move. – HP PA-RISC can annul any following instruction. x – IA-64: 64 1 -bit condition fields (flags) selected so conditional execution of any instruction (Predication). • Drawbacks of conditional instructions – Still takes a clock cycle even if “annulled”. – Must stall if condition is evaluated late. – Complex conditions reduce effectiveness; condition becomes known late in pipeline. (Ch. 4. 5) NOP A= B op C EECC 551 - Shaaban #33 lec # 6 Spring 2004 3 -29 -2004

IA-64 Predication Example Predication turns control dependence into data dependence on predication flags Flag

IA-64 Predication Example Predication turns control dependence into data dependence on predication flags Flag evaluated Taken Direction Not Taken Direction EECC 551 - Shaaban #34 lec # 6 Spring 2004 3 -29 -2004

Dynamic Hardware-Based Speculation • Combines: – Dynamic hardware-based branch prediction – Dynamic Scheduling: issue

Dynamic Hardware-Based Speculation • Combines: – Dynamic hardware-based branch prediction – Dynamic Scheduling: issue multiple instructions in order and execute out of order. (Tomasulo) • Continue to dynamically issue, and execute instructions passed a conditional branch in the dynamically predicted branch direction, before control dependencies are resolved. – This overcomes the ILP limitations of the basic block size. – Creates dynamically speculated instructions at run-time with no compiler support at all. – If a branch turns out as mispredicted all such dynamically speculated instructions must be prevented from changing the state of the machine (registers, memory). • Addition of commit (retire, completion, or re-ordering) stage and forcing instructions to commit in their order in the code (i. e to write results to registers or memory). • Precise exceptions are possible since instructions must commit in order. (Ch. 3. 7) EECC 551 - Shaaban #35 lec # 6 Spring 2004 3 -29 -2004

Hardware-Based Speculation Speculative Execution + Tomasulo’s Algorithm EECC 551 - Shaaban #36 lec #

Hardware-Based Speculation Speculative Execution + Tomasulo’s Algorithm EECC 551 - Shaaban #36 lec # 6 Spring 2004 3 -29 -2004

Four Steps of Speculative Tomasulo Algorithm 1. Issue — (In-order) Get an instruction from

Four Steps of Speculative Tomasulo Algorithm 1. Issue — (In-order) Get an instruction from FP Op Queue If a reservation station and a reorder buffer slot are free, issue instruction & send operands & reorder buffer number for destination (this stage is sometimes called “dispatch”) 2. Execution — (out-of-order) Operate on operands (EX) When both operands are ready then execute; if not ready, watch CDB for result; when both operands are in reservation station, execute; checks RAW (sometimes called “issue”) 3. Write result — (out-of-order) Finish execution (WB) Write on Common Data Bus (CDB) to all awaiting FUs & reorder buffer; mark reservation station available. 4. Commit — (In-order) Update registers, memory with reorder buffer result – When an instruction is at head of reorder buffer & the result is present, update register with result (or store to memory) and remove instruction from reorder buffer. – A mispredicted branch at the head of the reorder buffer flushes the reorder buffer (sometimes called “graduation”) Þ Instructions issue in order, execute (EX), write result (WB) out of order, but must commit in order. EECC 551 - Shaaban #37 lec # 6 Spring 2004 3 -29 -2004

Hardware-Based Speculation Example Show speculated Tomasulo status when MUL. D is ready to commit

Hardware-Based Speculation Example Show speculated Tomasulo status when MUL. D is ready to commit Example on page 229 EECC 551 - Shaaban #38 lec # 6 Spring 2004 3 -29 -2004

speculated Tomasulo status when MUL. D is ready to commit EECC 551 - Shaaban

speculated Tomasulo status when MUL. D is ready to commit EECC 551 - Shaaban #39 lec # 6 Spring 2004 3 -29 -2004

Hardware-Based Speculation Example on page 231 EECC 551 - Shaaban #40 lec # 6

Hardware-Based Speculation Example on page 231 EECC 551 - Shaaban #40 lec # 6 Spring 2004 3 -29 -2004

L. D. and MUL. D of first iteration have committed, other instructions completed execution

L. D. and MUL. D of first iteration have committed, other instructions completed execution Next to commit EECC 551 - Shaaban #41 lec # 6 Spring 2004 3 -29 -2004

Multiple Issue with Speculation Example A sufficient number of reservation stations and reorder (commit)

Multiple Issue with Speculation Example A sufficient number of reservation stations and reorder (commit) buffer entries are available. Branches still single issue Example on page 235 EECC 551 - Shaaban #42 lec # 6 Spring 2004 3 -29 -2004

Answer: Without Speculation EECC 551 - Shaaban #43 lec # 6 Spring 2004 3

Answer: Without Speculation EECC 551 - Shaaban #43 lec # 6 Spring 2004 3 -29 -2004

Answer: 2 -way Superscalar Tomasulo With Speculation Branches Still Single Issue EECC 551 -

Answer: 2 -way Superscalar Tomasulo With Speculation Branches Still Single Issue EECC 551 - Shaaban #44 lec # 6 Spring 2004 3 -29 -2004

Advantages of HW (Tomasulo) vs. SW (VLIW) Speculation • • • HW determines address

Advantages of HW (Tomasulo) vs. SW (VLIW) Speculation • • • HW determines address conflicts. HW provides better branch prediction. HW maintains precise exception model. HW does not execute bookkeeping instructions. Works across multiple implementations SW speculation is much easier for HW design. EECC 551 - Shaaban #45 lec # 6 Spring 2004 3 -29 -2004