ILP cont Maintaining Sequential Appearance Precise Interrupts RUU

ILP, cont. • Maintaining Sequential Appearance – Precise Interrupts – RUU approach to Oo. O Scheduling © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002

Superscalar Processors: The Big Picture Program Form Processing Phase Static program dynamic inst. Stream (trace) Fetch and CT prediction Dispatch/ dataflow execution window inst. Issue inst execution completed instructions inst. Reorder & commit © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002

Rename Dispatch buffer scheduler RF FUs Reorder buffer I-CACHE Pre-decode A Generic Superscalar OOO Processor Memory Interface © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002

Maintaining Sequential Semantics • What if execution gets interrupted at an arbitrary point? – All insts. before commit – None thereafter • We’ll focus on interrupts • Same mechanisms used today to support SPECULATIVE EXECUTION • “Definition”: Instr. executes speculatively up to complete. We don’t know yet if we should have executed this instr. Verification happens at commit (if ever). © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002

Interrupts • Examples – Power Failing, Arithmetic Overflow – I/O Device Request, OS Call, Page Fault – Invalid Opcode, Breakpoint, Protection Viol. • Aka Faults, Exceptions, or Traps • Requirements – Surprise Jump (to vectored Address) – Linking Return Address – Saving State – Changing State (e. g. , kernel mode) © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002

Classifying Interrupts • 1 a: Synchronous – Function of program state – overflow, page fault, etc. • 1 b. Asynchronous – e. g. , External device or malfunction • 2. Use Request – OS Call • 2 b. Coersed – From OS or hardware – page fault, protection violation • 3 a. User Maskable – Use can disable processing • 3 b. Non-Maskable – Guess!!! © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002

Classifying Interrupts, contd. • 4 a. Between Instructions – Usually Asynchronous • 4 b. Within an Instruction – Usually Synchronous – Harder to deal with, why? ? ? • 5 a. Resume – As if nothing happened as far as the program is concerned • 5 b. Catastrophic – Say, bye, program is leaving us © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002

Restartable Pipelines • Interrupts within an instruction are not catastrophic • Most machines support this – Needed for virtual memory • Some machines did not support this – Cost & Slowdown • PRECISE INTERRUPTS is key – As if the interrupt happened at a well defined point in the original sequential order – First let’s consider a simple DLXstyle pipeline © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002

Precise Interrupts • Sequential Semantics • Complete instructions before the offending instruction • Squash (effects of) instructions after • Save PC • Force trap instruction into FETCH stage – divert execution to interrupt handler © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002

Precise Interrupts • Jim Smith and Andrew Plezkun Paper • Original work was for a “simple” pipeline • Today the same principles are used in virtually all modern microprocessors – Support for SPECULATIVE EXECUTION • executing instruction without knowing whether we should • more on this later – and of course, precise interrupts • We’ll stick to precise interrupts for the time being © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002

Do the Simple Thing First • Modify State only when all preceding insts. are KNOWN to be exception free. • Mechanism: Result Shift Register • Stage = cycle • At FETCH: Reserve all stages for the duration of the instruction © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002

Simple Solution Discussion • Essentially In-Order Completion – Simple • Easy to implement – Performance? • Execution overlap still possible • Writebacks in order • Amplifies latencies • Dependent Instructions wait longer © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002

Allowing out-of-order completes • Add one more state for instruction execution: – COMPLETE & COMMIT • COMPLETE: – Result calculated – Dependent instructions can use – BUT, don’t know if preceding instructions are all OK – I. e. , don’t know if this instruction should have executed now based on the original program order • COMMIT: – All preceding instructions executed with no problems – Can safely commit stage changes © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002

OOO Complete & IO Commit • Want: Out-of-Order Completion – – Allow OOO completion Maintain in-order COMMIT Allow maximum overlap Guarantee precise state if needed Time • How does this improve performance? In-Order Complete OOO Complete DIV R 3, _, _ ADD R 1, _, _ ADD _, R 1, _ © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002 In-order commits

• Result Shift Register: When to commit When to complete res = result v = valid e = result NYA motion Reorder Buffer – Reserve Result Bus – Out-of-Order Completion • Reorder Buffer – Defer Commits and do them in-order – Allow OOO Completes by buffering state © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002

Reorder Buffer Complications • State is kept in the reorder buffer • Have to bypass from every entry – Need to determine latest write w/ respect to the consuming instruction RF RB Essentially: 1. In-Order Commits 2. Buffer speculative state till commit © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002

Speculative State Updates • Two fundamental approaches – Do changes but keep a record of old state – Everything OK? • Just discard record of changes • HISTORY BUFFER – Keep two states: • Architectural and Speculative • On COMPLETE write state to Speculative • On ISSUE read from speculative • On COMMIT write to Architectural • On Error, throw out Speculative state • FUTURE FILE © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002

History Buffer • Allow out-of-order register file updates • At decode record current value of target register in RB – notice that this is the previous value the register had • On Commit? – Do nothing, state is fine • On Exception? – Use History to UNDO changes made results RF Exception Source operands Destination registers HB © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002

History Buffer Discussion • • Simple Mechanism Additional Register File Port Single Source for Input Operands Normal Instruction processing Not changed by much – Control mostly unchanged – Nothing to do on Commit for the common case • Slow response to Interrupts – Need to scan through HB – Complex? © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002

Future File: The Optimist’s View • Two Register Files: – One updated Out-of-Order (FUTURE) • assume no exception will occur – One updated in Order (ARCHITECTURAL) • Advantage: No delay to restore state on exception results FF Source operands RF RB © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002

How These Relate to Register Renaming? • Physical Registers provide sufficient storage for both speculative and architectural storage • It’s the register map table that determines what is the current state • On interrupt we have to restore the map table – Values are there in the physical register file • History and Future approaches still valid – History: keep track of changes to register map table – On interrupt undo them one by one – Future: keep two tables • (ECE, Speculative: © A. Moshovos Toronto) ECE 1773 updated – Spring 2002 at decode • Architectural: updated at commit

RUU • Sohi’s Paper • Common Mechanism for Precise Interrupts and OOO Execution • Register Update Unit – A collection of Reservation stations – Organized as a FIFO queue – Instructions Enter In-order at FETCH – They Exit In-Order at COMMIT • Register File updates happen at this point. • Simplescalar follows this model – Well, mostly – Cut’s corners on when Completes become visible © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002

RUU: OOO Execution • Decode: – Check RUU for most recent write to register – If none found, read value from RF • Do it in parallel really – If found, link to producer with a TAG • RUU number is the TAG • Issue – Wait till all input operands are ready • Complete – Broadcast value and RUU ID • Waiting instructions will pick value up • Commit – Head and Tail pointer for FIFO operation © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002 – Only when everyone before has

Where is the Rename Table? • It’s the RUU – @ decode insts scan for the most recent update to register – If none found, then register in register file – Otherwise, get RUU entry # as tag • Interrupts? – Simply flush RUU • Pros/Cons: – Associative lookup for decode – RUU ports limit when consumers can read a value © A. Moshovos (ECE, Toronto) ECE 1773 – Spring 2002