Optical Interconnection Networks Design Analysis and Simulation Study

Optical Interconnection Networks Design, Analysis, and Simulation Study of Optical Interconnection Networks M. S. Thesis Defense Presentation by Ch’ng Shi Baw Advisor: Prof. Mark A. Franklin 20 April 1999 1

Optical Interconnection Networks Presentation Organization • Introduction – Background information & related work • Thesis Contribution – Interconnection Network Simulator Framework – Improving the Gemini interconnect architecture • Conclusion – Summary and future work 2

Optical Interconnection Networks Thesis Contributions • Interconnection Network Simulator (ICNS) framework – Design of the ICNS framework – Simulator Verification • Study of the Gemini network – Performance analysis – Improving Gemini’s throughput – Adding fair-scheduling capability to Gemini 3

Optical Interconnection Networks Introduction • • Interconnection network in generic terms Motivation: why optics Overview of enabling technologies The Gemini interconnect architecture and related works • Simulation tool to aid design and study of interconnection networks 4

Optical Interconnection Networks Interconnection Network • Terminals generate and/or consume data messages – e. g. : CPU, sensor banks, disks, other I/O devices • Links and switches transport data 5

Optical Interconnection Networks Motivation • Want to solve large problems fast. • Target problems that are compute-, data-, and communications-intensive: – need multiple processors and high speed networks to connect processors. – want high bandwidth and low latency • Use optics to build interconnection networks 6

Optical Interconnection Networks Motivation: Why Optics • Strengths of optics: – very high bandwidth (tens of Tb/s in one fiber) – low electro-magnetic interference – virtually no transmission line effects at high speed • Weaknesses of optics (current technology): – unsuitable to implement logical functions – optical components are generally costly 7

Optical Interconnection Networks Optics: Technology Overview • Guided-wave optics • Free-space optics • “Smart Pixel Array” Arrays of VCSELs and detectors 8

Optical Interconnection Networks Optics: Technology Overview • Free-space optics and “Smart Pixel Array” – Potential to provide physically clutter-free interconnect – Limited distance spanning capability – Insufficient reliability study 9

Optical Interconnection Networks “Smart Pixel Array” ring architecture. [Chen 98, Gourlay 98, Lacroix 98, Franklin] 10

Optical Interconnection Networks Optics: Technology Overview • Guided-wave Optics – Fiber optics (mature, widely deployed) – Polymer wave guides • Recent developments: – polymer wave guides layout technology [Eldada 96] – efficient fiber-to-polymer wave guide coupling [Barry 97] – electro-optical switching elements [Sneh 96, Lucent 97] 11

Optical Interconnection Networks Electro-optical Switching • The y-branch: – Refraction index of Li. Nb. O 3 changes in the presense of electric field. 12

Optical Interconnection Networks Building on the y-branch • A 2 x 2 electro-optical switch – Issues: power loss and crosstalk. • Circuit-switched. No Buffering. 13

Optical Interconnection Networks Reducing Crosstalk • Time-Dilation [Qiao 96] – Reduce crosstalk using scheduling technique • Space-Dilation – Add hardware 14

Optical Interconnection Networks – Space-dilation technique used by Lucent Technologies [Lucent 97]. Next: Gemini Network Overview 15

Optical Interconnection Networks The Gemini Network • Use two networks – one optically switched (Banyan topology to reduce power loss) – one electronically switched – as proposed, the two networks have identical topology [Chamberlain 97] • Main idea: – off-load bulk data to high-bandwidth optical network – maintain low-latency in lightly-loaded electrical network to cater for control messages • Goals (not related to performance): – easily manufacturable (low cost) – forward compatibility 16

Optical Interconnection Networks The Gemini Network 17

Optical Interconnection Networks The Gemini Network – Layout polymer wave guides using wire-printing techniques [Eldada 96] – Board level connection employs polymer-to-fiber coupling technique [Barry 97] – Assume space-dilation (use Lucent switch [Lucent 97]) 18

Optical Interconnection Networks Related Work • Pan, Qiao, and Yang [Pan 99] – Use 2 x 2 electro-optical switches – Banyan topology – Rely on time-dilation technique 19

Optical Interconnection Networks Time-Dilation • Construct Contention-and-Conflict Free (CF) mappings – enforce switch element disjoint (SED) condition – schedule connections so that no two connections share a switch element • Assumes that a laser source can be completely turned off. 20

Optical Interconnection Networks • Example A set of CF-mappings for a 4 x 4 network. • • • Challenge is to find an optimal set of mappings for a given (arbitrary) set of connections Need 8 mappings for a 16 connections in a 4 x 4 network Need about 50 for 1000 connections (32 x 32 network) Polynomial time algorithm by Qiao to construct optimal set of CF-mappings [Qiao 96] Assume the existence of a centralized controller Next: The Need of a Simulation Tool 21

Optical Interconnection Networks Simulation Tool • Simulation tools are generally helpful in the study of queueing systems • Need an extensible simulator – vast interconnection network design space – want the ability to easily extend simulator to simulate future optical network components • Tune network design to specific applications – want the ability to incorporate application models into simulation Next: Thesis Contribution 22

Optical Interconnection Networks Thesis Contributions • Interconnection Network Simulator (ICNS) framework – Design of the ICNS framework – Simulator and Simulation Verification • Study of the Gemini network – Performance analysis – Improving Gemini’s throughput – Adding fair-scheduling capability to Gemini 23

Optical Interconnection Networks Interconnection Network Simulator (ICNS) • Process-based discrete event simulation engine. • Object-oriented design. • Implementation environment: – Uses the MODSIM III language developed by CACI Products Company. – MODSIM III C++ executable. – Simulation engine and MODSIM III to C++ compiler by CACI. – C++ to executable compiler is gcc. – Developed in Solaris 2. 5 environment. First Major Contribution of Thesis 24

Optical Interconnection Networks ICNS Framework: Base Classes • Message. Obj – models messages – provides uniform interface to Node. Obj • Node. Obj – abstract base class to be subclassed to model links, switches, terminals, etc. – provides uniform interface to Message. Obj • Network. Obj – container of Node. Obj’s – provide identifier-to-object-reference translation service 25

Optical Interconnection Networks ICNS Class Hierarchy 26

Optical Interconnection Networks Progression of a Simulation • Interactions between Message. Obj and Node. Obj drive simulation forward. • Message. Obj’s operation: – Engaging a Node. Obj: • • • Ask if Node. Obj is busy If it is, ask to be queued and terminate Ask to be processed otherwise Wait for the processing to finish (elapse simulation time) Terminate 27

Optical Interconnection Networks Progression of a Simulation • Node. Obj’s operations: – When asked if it is busy: • answer yes or no – When asked to queue a Message. Obj • action depends on queueing policy (subclass-specific) – When asked to process a Message. Obj • action depends on which object is being simulated (subclass-specific) • tell Message. Obj how long to wait 28

Optical Interconnection Networks Description of Selected Objects • Message • Link • Terminal – Message Generator – Buffer – Central Processing Unit (CPU) • Switch 29

Optical Interconnection Networks The Message Object 30

Optical Interconnection Networks The Link Object • A Simple Link • A Multichannel Link 31

Optical Interconnection Networks The Terminal Object • Processing Node Model 32

Optical Interconnection Networks The Switch Object • The 2 x 2 Switch Model 33

Optical Interconnection Networks An ICNS Application: sim • Separates topology from other network parameters – Topology descriptor file (text file) – Parameter descriptor file (text file) • Param. Obj – parses parameter descriptor file – keeps track of all network parameters • Build. GNetwork Procedure – procedure parses topology descriptor file, instantiates and initializes objects accordingly 34

Optical Interconnection Networks sim: User Interface • hand edit parameter file and topology file – or use Java-based GUI tools to generate files • to invoke the sim program, type sim param_file topology_file 35

Optical Interconnection Networks Simulator Verification • Examine event trace (for small simulations) • Use visualization tool (for medium size simulations) – Visualization tool driven by event trace – Visualization developed by Wrighton [WUCCRC-99 -02] • Simulate systems with known analytical results – Compare simulation results to analytical results Next: Visual Demonstration 36

Optical Interconnection Networks Visualization Tool Demo. . . 37

Optical Interconnection Networks M/M/1 and M/D/1 Simulations • Verification Example: – Simulate parallel M/M/1 and M/D/1 systems 38

Optical Interconnection Networks M/M/1 and M/D/1 Simulations • Within 3% of analytical results for loads up to about 92% 39

Optical Interconnection Networks Simulation Verification • Simulate for long enough time to get valid statistics • Wait out transient states to get valid steadystate statistics • Demonstrated that simulator produced valid results for a wide range of loads – within 3% of analytical results for loads up to 92% Next: The Gemini Network 40

Optical Interconnection Networks The Gemini Network [Chamberlain 97] • Architecture Overview – Network Model – Terminal Model – Switch Model • • Basic Protocol Performance Limits Simulation Results Improvements. . . 41

Optical Interconnection Networks Gemini Architecture Overview • Network Model – Banyan topology – Bufferless, optically circuitswitched network – Buffered, electronically packet-switched network 42

Optical Interconnection Networks Gemini Architecture Overview • Terminal Model – CPU module models applications – One pair of optical output port – One pair of electrical output port 43

Optical Interconnection Networks Gemini Architecture Overview • Switch Model – Electrical switch controls optical switch 44

Optical Interconnection Networks Gemini Network Protocol • Original Protocol – Rely on Negative ACK – Fire-on-timeout mechanism – Issue: • How to set timeout parameter? 45

Optical Interconnection Networks Evaluating Protocols. . . • Space Complexity – How much state information to keep (in switches) – Original protocol: O(1) per switch • Time Complexity – How many computational steps needed to (for a switch) process a control signal – Original protocol: O(1) • Performance measures – Throughput, latency, utilization, etc. 46

Optical Interconnection Networks The setup-teardown Protocol • Similar to original protocol – But use positive ACK – Fire upon ACK • Signals – – setup(S, D, blocked) ack. Setup(S, D) block(S, D) teardown(S, D) • Space Complexity O(1) per switch • Time Complexity O(1) 47

Optical Interconnection Networks Switch Operation • Each switch keeps a list and a one bit optical switch state variable (= or x) • Processing a setup(S, D, blocked) signal: – – determine output port if setup already blocked, forward to output port. Determine requested state (= or x) if requested state conflict with current state and list is not empty • set blocked and forward to output port – else set state to requested state, add S to list, forward to output port • Processing a teardown(S, D) signal: – determine output port – if S in list, remove S from list – forward to output port 48

Optical Interconnection Networks Switch Operation Complexity • Space Complexity O(1) – list size at most 2 • Time Complexity O(1) Next: Performance Analysis 49

Optical Interconnection Networks Performance Limits numerator Send teardown Send setup … delay. . . Receive ack. Setup Send optical message denominator • Optical network utilization efficiency limited by • Define , 50

Optical Interconnection Networks Performance Limits Next: Simulations 51

Optical Interconnection Networks Performance Limits Next: Simulations 52

Optical Interconnection Networks setup-teardown Simulations • Poisson arrival process • Fixed and Exponentially distributed message lengthes • Choose G=16384, Q=12, asig=1. 25 • Network sizes from 4 x 4 to 32 x 32 53

Optical Interconnection Networks setup-teardown Simulation Results 54

Optical Interconnection Networks setup-teardown Simulation Results 55

Optical Interconnection Networks setup-teardown Simulation Results Next: Blocking, the cause of low utilization 56

Optical Interconnection Networks Problem: Blocking • Lose throughput due to blocking. 57

Optical Interconnection Networks Solution: Virtual Output Queues • Terminals queue outgoing messages according to their destinations Second Major Contribution of Thesis 58

Optical Interconnection Networks VOQ Protocol • Terminals allowed to send one setup request for each non-empty VOQ – Get around Head-of-Line blocking by exploring all possible optical paths in parallel • Switch processes setup, teardown signals as before – Add/delete source-destination pair to/from list instead – Have N 2 source-destination pairs – but can bound list size to 2 N • Space Complexity O(N) per switch • Time Complexity O(1) 59

Optical Interconnection Networks VOQ Simulation Results 60

Optical Interconnection Networks VOQ Simulation Results 61

Optical Interconnection Networks VOQ Simulation Results 62

Optical Interconnection Networks VOQ Simulation Results 63

Optical Interconnection Networks VOQ Simulation Results • Load on the electrical network Network Size 4 x 4 8 x 8 16 x 16 32 x 32 Load < 0. 6% < 1. 2% < 2. 4% < 4. 6% • VOQ imposes minimal load on the electrical network – lightly loaded electrical network can maintain low latency for application control messages – variations of VOQ to further reduce electrical network load 64

Optical Interconnection Networks VOQ Complexity • Even though there are N 2 source-destination pairs, can implement list using bitmap and/or perfect hashing to make switch operations’ time complexities O(1). • Space complexity is – N 2 bits per switch if list is implemented as a bitmap – 2 N bits per switch if list is implemented using perfect hashing as well • Can exploit regularity of Banyan network to construct simple perfect hash functions 65

Optical Interconnection Networks VOQ Implementation Complexity 66

Optical Interconnection Networks VOQ Merits and Demerits • VOQ is good because: – VOQ significantly increases throughput – VOQ adds only minimal complexity to the system • But. . . – VOQ may lead to starvation under very high load. . . Prevent starvation using fair scheduling techniques. Next: Fair Scheduling in Gemini 67

Optical Interconnection Networks Fair Scheduling in Gemini • Starvation – How and when it occurs – what is the tradeoff • Use fair scheduling to prevent starvation – concept of fairness in Gemini • fairness granularity • quantitative fairness measure • Gemini fair scheduler design – what are the desirable characteristics – the Distributed Deficit Round Robin (d. DRR) fair scheduler • Fair scheduler evaluation Third Major Contribution of Thesis 68

Optical Interconnection Networks Starvation • How and when it occurs • Problem: – Switch reinforced to stay in the same state – Need to induce switch to change state 69

Optical Interconnection Networks Starvation: When does it occur? • 4 x 4 network, 16 flows (i. e. , source-destination pairs) • Load is 0. 8 • Plot cumulative number of bits sent versus flow number – snapshots taken at 500 and 20 message time intervals. 70

Optical Interconnection Networks Starvation: When does it occur? • 4 x 4 network, 16 flows (i. e. , source-destination pairs) • Load is 1. 0 • Plot cumulative number of bits sent versus flow number – snapshots taken at 500 and 20 message time intervals. 71

Optical Interconnection Networks Starvation: When does it occur? • 4 x 4 network, 16 flows (i. e. , source-destination pairs) • Load is 1. 2. Observe starvation on right plot. • Plot cumulative number of bits sent versus flow number – snapshots taken at 500 and 20 message time intervals. 72

Optical Interconnection Networks Starvation • The tradeoff – Under variable message size assumption, inducing a switch to change state means stopping a connection from sending data – The more frequent we induce a change in switch states, the more throughput we lose – Fairness granularity directly related to how often we induce a change in switch state Tradeoff between fairness granularity and throughput Next: The Concept of Fairness 73

Optical Interconnection Networks Concept of Fairness • Fairness granularity – no smaller than maximum message size – no smaller than scheduler’s resolution • e. g. , scheduler may keep tab using 1 KB chunk as basic unit, thus fairness granularity cannot be finer than 1 KB. • Quantitative Fairness Measure – make sense to measure fairness for a time interval iif • all flows are actively contending throughout the interval • there is a bound on message size 74

Optical Interconnection Networks Fairness Measure • Fairness. Measure (modified from [SV 95]) 75

Optical Interconnection Networks Fairness Measure • Ideally fair system – FM(I) = 0 for all I • Worst case – one flow monopolize access, all other flows starve – FM(I) = |F| • Worst case for Gemini – assume all connections actively contending for access – FM(I) = N for an Nx. N network Next: Scheduler Design Considerations 76

Optical Interconnection Networks Gemini Fair Scheduler Design Considerations • Existing fair schedulers assume: – many-to-one contention: multiple flows contending for one link (RR, WFQ, WF 2 Q, DRR, SCFQ, VCFQ, etc. ) – many-to-many contention, but in a non-blocking network (crossbar) [e. g. , Prabhakar 97, Mc. Keown 95] – slotted time [Lu 97, Prabhakar 97, Mc. Keown 97] – intermediate buffering available • Gemini violates all the above assumptions. 77

Optical Interconnection Networks Gemini Fair Scheduler Design Considerations • Where to put the scheduler – centralized scheduler – distributed schedulers in terminals – distributed schedulers in switches • Scheduler complexity – Space Complexity (SC) • How much storage to keep track of flow states – Time Complexity (TC) • How many computational steps needed to make a scheduling decision. 78

Optical Interconnection Networks Gemini Fair Scheduler Design Considerations • Desirable Characteristics – – distributed in switches leverage underlying VOQ protocol low space and time complexities tunable fairness granularity (scheduler resolution) • Modify DRR [SV 95] to work in Gemini Distributed DRR (d. DRR) 79

Optical Interconnection Networks DRR Description • Scheduler resolution (fairness granularity) determined by quota (quanta) assigned to flows • Keeps track of flow’s unused quotas • d. DRR uses similar ideas 80

Optical Interconnection Networks Switch to DRR slide show. . . 81

Optical Interconnection Networks d. DRR Switch Controller Structure • Each 2 x 2 electrical switch controller contains a partial d. DRR scheduler. • The d. DRR module selectively blocks setup requests • Blocking is resolved by the VOQ module 82

Optical Interconnection Networks Differences Between DRR and d. DRR’s Assumed Environments DRR • scheduler co-locates with queues, queue state information readily available • visits queues one by one in round robin order d. DRR • queue state information needs to be explicitly conveyed to scheduler • receives (setup) requests in no particular order Next: Modify Signals to Pass Queue State Information 83

Optical Interconnection Networks Passing Queue and Flow State Information to d. DRR Schedulers VOQ • setup(S, D, blocked) • teardown(S, D) d. DRR • setup(flow. ID, blocked, amount) • teardown(flow. ID, amount, more) terminal-S terminal-D Next: d. DRR Data Structure 84

Optical Interconnection Networks Switch to d. DRR slide show. . . 85

Optical Interconnection Networks d. DRR Data Structure in a Switch qi dci spi nri morei q dc sp nr more qj dcj spj nrj morej qk dck spk nrk morek a d. DRR entry for a flow q quantum is the amount by which a flow’s quota is replenished at each round dc deficit counter keeps track of a flow’s available quota (initialized to q) sp suspension flag indicates if a flow has exhausted its quota (initially set) nr new round flag indicates if a flow has contended in the current round (initially set) more flag indicates if a flow’s queue is empty (initially unset) Next: d. DRR Space Complexity 86

Optical Interconnection Networks d. DRR Complexity • Space Complexity O(N) per switch – – Each flow has one entry in each switch it passes through In an Nx. N network (N 2 flows), each switch handles 2 N flows Space Complexity is 2 N entries per switch Can easily hash flow ID from N 2 space to 2 N space • given a flow ID, can directly hash into an entry in O(1) time. • Time Comlexity O(1) Next: d. DRR operations (psuedo-code) 87

Optical Interconnection Networks Processing a setup(i, amount) signal Next: Condition to set nri to be explained later 88

Optical Interconnection Networks Processing a teardown(i, amount, more) signal Note that the processing of setup and teardown signals is done in O(1) time. Next: Determining Round Boundary 89

Optical Interconnection Networks Determine New Round Boundary • A round ends if all flows have – either exhausted their quota – or stopped contending (queues become empty) • Begin a new round if all flows suspension flags become set • Can check for New Round condition in O(1) time – test all suspension flags in parallel in hardware 90

Optical Interconnection Networks The New Round Operation • Note that all operations take O(1) time • d. DRR complexity for an Nx. N network: – Space Complexity O(N) per switch – Time Complexity O(1) Next: d. DRR Feature and Functionality 91

Optical Interconnection Networks d. DRR Feature and Functionality • Properties inherited from DRR: – Tunable Fairness Granularity • scheduler resolution determined by quanta assignment • assign larger quanta to get coarser grained fairness – can trade off fairness granularity with throughput – Weighted Fair Scheduling • assign different quanta to different flows to perform weighted fair scheduling • tradeoff not well understood Next: d. DRR Simulation Results 92

Optical Interconnection Networks Recall old results: • 4 x 4 network, 16 flows (i. e. , source-destination pairs) • Load is 1. 2. Observe starvation on right plot. • Plot cumulative number of bits sent versus flow number – snapshots taken at 500 and 20 message time intervals. 93

Optical Interconnection Networks Same Simulation with d. DRR: • 4 x 4 network, 16 flows, quantum is eight times average message size. • Load is 1. 2. Starvation eradicated. • Plot cumulative number of bits sent versus flow number – snapshots taken at 500 and 20 message time intervals. 94

Optical Interconnection Networks More d. DRR Simulation Results • • Pre-saturated queues, snapshots taken at 20 message time intervals No scheduler case: Fairness. Measure = 4. 00, Throughput = 1. 00 Left plot quantum = 4 MAX. Fairness. Measure =. 33, Throughput =. 86 Right plot quantum = MAX. Fairness. Measure =. 11, Throughput =. 71 95

Optical Interconnection Networks Fairness Granularity vs. Throughput Tradeoff • Pre-saturated queues, variable-size messages • Plots of normalized throughput vs. quantum size (normalized to MAX) Next: Weighted Fair Scheduling 96

Optical Interconnection Networks Fairness Granularity vs. Throughput Tradeoff • • Pre-saturated queues. Plots of normalized throughput vs. quantum size. Left: fixed-size messages. Quantum size normalized to message size. Right: variable-size messages. Quantum size normalized to MAX. Next: Weighted Fair Scheduling 97

Optical Interconnection Networks d. DRR Weighted Fair Scheduling • Assign q to all flows, except 2 q to flow 4 and 3 q to flow 13. • Results: – flow 4 sent 1. 9984 times the average traffic sent by others (except flow 13) – flow 13 sent 3. 0029 times the average traffic sent by others (except flow 4) • Throughput is 81% of the case where all flows are assigned q Next: Prioritized Scheduling 98

Optical Interconnection Networks Prioritized Fair Scheduling in Gemini qi dci spi nri morei qj dcj spj nrj morej qk dck spk nrk morek class 1 qi dci spi nri morei qj dcj spj nrj morej qk dck spk nrk morek class 2 qi dci spi nri morei qj dcj spj nrj morej qk dck spk nrk morek class 3 qi dci spi nri morei qj dcj spj nrj morej qk dck spk nrk morek class K Next: Conclusion 99

Optical Interconnection Networks Conclusion • Summary of Thesis Contributions – Extensible interconnection network simulator framework – VOQ improves Gemini throughput • O(1) time complexity • O(N) per switch space complexity – d. DRR scheduler adds fair scheduling capability to Gemini • O(1) time complexity • O(N) per switch space complexity • Tunable fairness granularity Next: Future Work 100

Optical Interconnection Networks Conclusion • Future Work – Better understanding of the throughput vs. fairness granularity tradeoff • better understanding of many-to-many fair scheduling in general. – Gemini network interface design • need high speed network interface to take advantage of high bandwidth optical network – Gemini is only half optical • need to study the electrical half as well Next Slide: Many-to-Many Fiar Scheduling 101

Optical Interconnection Networks Many-to-Many Weighted Fair Scheduling (Future Work) • Fundamental tradeoff: – Example 1: 4 -flow, 4 -parallel system, assign weights 1: 1: 1: 2 • lose 3/8 throughput – Example 2: 4 -flow, 4 -parallel system, assign weights 1: 1: 1: 3 • lose 1/2 throughput – Gemini (N 2 flow, N-parallel system) • Nature of “fairness” and throughput tradeoff not yet understood. 102

Optical Interconnection Networks Many-to-Many Weighted Fair Scheduling (Future Work) No reason to do fair scheduling here since resources are not under contention. • Fundamental tradeoff: – Example 1: 4 -flow, 4 -parallel system, assign weights 1: 1: 1: 2 • lose 3/8 throughput – Example 2: 4 -flow, 4 -parallel system, assign weights 1: 1: 1: 3 • lose 1/2 throughput – Gemini (N 2 flow, N-parallel system) • Nature of “fairness” and throughput tradeoff not yet understood. 103

Optical Interconnection Networks Conclusion • Future Work – Other optical technologies • “Smart Pixel Array” and free-space optics • Extend ICNS framework to include – Predicate interconnection network study on target applications • demonstrate that applications indeed benefit from using an optical network 104

Optical Interconnection Networks Acknowledgement • Advisors: – Dr. Mark A. Franklin, Dr. Roger D. Chamberlain • Committee Members: – Dr. Jonathan S. Turner, Dr. George Varghese • NSF and DARPA for financial support 105