Chapter 14 Instruction Level Parallelism and Superscalar Processors

Chapter 14 Instruction Level Parallelism and Superscalar Processors

What is Superscalar? A Superscalar machine executes multiple independent instructions in parallel. • Common instructions (arithmetic, load/store, conditional branch) can be executed independently. • Equally applicable to RISC & CISC, but more straightforward in RISC machines. • The order of execution is usually determined by the compiler.

Example Superscalar Organization

Superpipelined Machines Superpiplined machines overlap pipe stages - rely on stages being able to begin operations before the last is complete.

Superscalar v Superpipeline

Limitations of Superscalar Dependent upon: • Instruction level parallelism • Compiler based optimization • Hardware support • Limited by —True data dependency —Procedural dependency —Resource conflicts —Output dependency —Antidependency (one instruction can overwrite a value that an “earlier instruction” has not yet read)

True Data Dependency ADD r 1, r 2 (r 1 : = r 1+r 2) MOVE r 3, r 1 (r 3 : = r 1) • Can fetch and decode second instruction in parallel with first • Can NOT execute second instruction until first is finished Compare with the following? LOAD r 1, X (r 1 : = X) MOVE r 3, r 1 (r 3 : = r 1) What additional problem do we have here?

Procedural Dependency • Can not execute instructions after a branch in parallel with instructions before a branch, because? • Also, if instruction length is not fixed, instructions have to be decoded to find out how many fetches are needed

Resource Conflict • Two or more instructions requiring access to the same resource at the same time —e. g. two arithmetic instructions • Solution - Can possibly duplicate resources —e. g. have two arithmetic units

Antidependancy ADD R 4, R 3, 1 ADD R 3, R 5, 1 • Cannot complete the second instruction before the first has read R 3

Effect of Dependencies

Instruction-level Parallelism • Consider: LOAD R 1, R 2 ADD R 3, 1 ADD R 4, R 2 These can be handles in parallel • Consider: ADD R 3, 1 ADD R 4, R 3 STO (R 4), R 0 These cannot

Instruction Issue Policies • Order in which instructions are fetched • Order in which instructions are executed • Order in which instructions change registers and memory

In-Order Issue In-Order Completion Issue instructions in the order they occur: • Not very efficient • May fetch >1 instruction • Instructions must stall if necessary

In-Order Issue In-Order Completion (Diagram) Assume: • I 1 requires 2 cycles to execute • I 3 & I 4 conflict for the same functional unit • I 5 depends upon value produced by I 4 • I 5 & I 6 conflict for a functional unit

In-Order Issue Out-of-Order Completion How does this effect interrupts?

Out-of-Order Issue Out-of-Order Completion • Decouple decode pipeline from execution pipeline • Can continue to fetch and decode until this pipeline is full • When a functional unit becomes available an instruction can be executed • Since instructions have been decoded, processor can look ahead

Out-of-Order Issue Out-of-Order Completion (Diagram) Note: I 5 depends upon I 4, but I 6 does not

Register Renaming Output and antidependencies occur because register contents may not reflect the correct ordering from the program • Could result in a pipeline stall • One solution: Registers allocated dynamically

Register Renaming example • • • R 3 b: =R 3 a + R 5 a (I 1) R 4 b: =R 3 b + 1 (I 2) R 3 c: =R 5 a + 1 (I 3) R 7 b: =R 3 c + R 4 b (I 4) Without subscript refers to logical register in instruction • With subscript is hardware register allocated • Note R 3 a R 3 b R 3 c

Machine Parallelism Support • Duplication of Resources • Out of order issue • Renaming • Windowing

Speedups of Machine Organizations Without Procedural Dependencies

Study Conclusions • Not worth duplication functional units without register renaming • Need instruction window large enough (more than 8, probably not more than 32)

Branch Prediction in Superscalar Machines • Delayed branch not used much. Why? • Branch prediction used - Branch history may still be useful

Superscalar Execution

Committing or Retiring Instructions • Sometimes results must be held in temporary storage until it is certain they can be placed in “permanent” storage. This temporary storage needs to be regularly cleaned up.

Superscalar Hardware Support • Simultaneously fetch multiple instructions • Logic to determine true dependencies involving register values and Mechanisms to communicate these values • Mechanisms to initiate multiple instructions in parallel • Resources for parallel execution of multiple instructions • Mechanisms for committing process state in correct order