Lecture 6 Dynamic Scheduling with Scoreboarding and Tomasulo

Lecture 6: Dynamic Scheduling with Scoreboarding and Tomasulo Algorithm (Section 2. 4) Nov. 9, 2004 1

Scoreboard Implications • Out-of-order completion => WAR, WAW hazards • Solutions for WAR – CDC 6600: Stall Write to allow Reads to take place; Read registers only during Read Operands stage. • For WAW, must detect hazard: stall in the Issue stage until other completes • Need to have multiple instructions in execution phase => multiple execution units or pipelined execution units • Scoreboard replaces ID with 2 stages (Issue and RO) • Scoreboard keeps track of dependencies, state or operations – Monitors every change in the hardware. – Determines when to read ops, when can execute, when can wb. – Hazard detection and resolution is centralized. 2

Four Stages of Scoreboard Control 1. Issue—decode instructions & check for structural hazards (ID 1) If a functional unit for the instruction is free and no other active instruction has the same destination register (WAW), the scoreboard issues the instruction to the functional unit and updates its internal data structure. If a structural or WAW hazard exists, then the instruction issue stalls, and no further instructions will issue until these hazards are cleared. 2. Read operands—wait until no data hazards, then read operands (ID 2) A source operand is available if no earlier issued active instruction is going to write it, or if the register containing the operand is being written by a currently active functional unit. When the source operands are available, the scoreboard tells the functional unit to proceed to read the operands from the registers and begin execution. The scoreboard resolves RAW hazards dynamically in this step, and instructions may be sent into execution out of order. 3

Four Stages of Scoreboard Control 3. Execution—operate on operands (EX) The functional unit begins execution upon receiving operands. When the result is ready, it notifies the scoreboard that it has completed execution. 4. Write result—finish execution (WB) Once the scoreboard is aware that the functional unit has completed execution, the scoreboard checks for WAR hazards. If none, it writes results. If WAR, then it stalls the instruction. Example: DIVD F 0, F 2, F 4 ADDD F 10, F 8 SUBD F 8, F 14 CDC 6600 scoreboard would stall SUBD until ADDD reads operands CDC 6600 has one integer, 2 FP multipliers, 1 FP divide, 1 FP add units. 4 See Fig. A. 50.

Three Parts of the Scoreboard 1. Instruction status—which of 4 steps the instruction is in 2. Functional unit status—Indicates the state of the functional unit (FU). 9 fields for each functional unit Busy—Indicates whether the unit is busy or not Op—Operation to perform in the unit (e. g. , + or –) Fi—Destination register Fj, Fk—Source-register numbers Qj, Qk—Functional units producing source registers Fj, Fk Rj, Rk—Flags indicating when Fj, Fk are ready and not yet read. Set to No after operand are read. 3. Register result status—Indicates which functional unit will write each register, if one exists. Blank when no pending instructions will write that register 5

Scoreboard Example Cycle 7 Note: (1) In-order Issue (2) I 2 could not be issued at cycle 2 due to structural hazard (3) I 3 issued in cycle 6, but stalled at read because I 2 isn’t complete 6

27

Review: Scoreboard • Limitations of 6600 scoreboard – – – No forwarding Limited to instructions in basic block (small window) Large number of functional units (structural hazards) Stall on WAR hazards Stall on WAW hazards DIV. D F 0, F 2, F 4 ADD. D F 6, F 0, F 8 WAR S. D F 6, 0(R 1) WAW SUB. D F 8, F 10, F 14 Output dependence Antidependence MUL. D F 6, F 10, F 8 Name dependence Nov. 2, 2004 Lec. 7 8

Dynamic Scheduling: Tomasulo Algorithm • For IBM 360/91 about 3 years after CDC 6600 that proposed scoreboarding • Goal: High Performance without special compilers • Differences between Tomasulo Algorithm & Scoreboard – Control & buffers distributed with Function Units vs. centralized in scoreboard; called “reservation stations” – Registers in instructions replaced by pointers to reservation station buffer – HW renaming of registers to avoid WAW hazards – Buffer operand values to avoid WAR hazards – Common Data Bus broadcasts results to all FUs – Load and Stores treated as FUs as well • Why study? Lead to Alpha 21264, HP 8000, MIPS Nov. 2, 2004 Lec. 7 10000, Pentium II, Power PC 604 … 9

FP unit and load-store unit using Tomasulo’s alg. Nov. 2, 2004 Lec. 7 10

Dynamic Algorithm: Tomasulo Algorithm DIV. D F 0, F 2, F 4 ADD. D S, F 0, F 8 S. D S, 0(R 1) renaming SUB. D T, F 10, F 14 MUL. D F 6, F 10, T register • Implemented through reservation stations (rs) per functional unit – Buffers an operand as soon as it is available – avoids WAR hazards. – Pending instr. designate rs that will provide their inputs – avoids WAW hazards. – The last write in a sequence of same-register-writing actually updates the register – Decentralize hazard detection and execution control Nov. – 2, 2004 Lec. 7 directly to the FU from rs rather 11 than Instruction results are passed from registers

Three Stages of Tomasulo Algorithm 1. Issue—get instruction from FP Op Queue Stall if structural hazard, ie. no space in the rs. If reservation station (rs) is free, the issue logic issues instr to rs & read operands into rs if ready (Register renaming => Solves WAR). Make status of destination register waiting for this latest instn even if the previous instn writing to this register hasn’t completed => Solves WAW hazards. 2. Execution—operate on operands (EX) When both operands are ready then execute; if not ready, watch CDB for result – Solves RAW 3. Write result—finish execution (WB) Write on Common Data Bus to all awaiting units; mark reservation station available. Write result into dest. reg. if its status is r. => Solves WAW. • Normal data bus: • CDB: data + destination data + source (“go to” bus) (“come from” bus) – 64 bits of data + 4 bits of Functional Unit source address – Write if matches expected Functional Unit (produces result) – Does broadcast Nov. 2, 2004 Lec. 7 12

Reservation Station Components Op—Operation to perform in the unit (e. g. , + or –) Vj, Vk— Value of the source operand. Qj, Qk— Name of the RS that would provide the source operands. Value zero means the source operands already available in Vj or Vk, or is not necessary. Busy—Indicates reservation station or FU is busy Register File Status Qi: Qi —Indicates which functional unit will write each register, if one exists. Blank (0) when no pending instructions that will write that register meaning that the value is already available. Nov. 2, 2004 Lec. 7 13