CSL 718 Multiprocessors Interconnection Mechanisms Performance Models 20

CSL 718 : Multiprocessors Interconnection Mechanisms Performance Models 20 th April, 2006 Anshul Kumar, CSE IITD

Connecting Processors and Memories • Shared Buses • Interconnection Networks – Static Networks – Dynamic Networks M M P P M P Interconnection Network M M M P M P Interconnection Network M M Global Interconnection Network M Anshul Kumar, CSE IITD M M 2

Shared Bus each processor sees this picture: processing bus access prob of a processor using the bus = prob of a processor not using the bus = 1 – prob of none of the n processors using the bus = (1 – )n prob of at least one processor using the bus = 1 – (1 – )n achieved BW on a relative scale = 1 – (1 – )n required BW = n available BW = 1 Anshul Kumar, CSE IITD 3

Effect of re-submitted requests (1 -PA ) 1 - + PA A prob = q. A Anshul Kumar, CSE IITD 1 -PA W PA prob = q. W 4

Anshul Kumar, CSE IITD 5

Anshul Kumar, CSE IITD 6

Waiting time Anshul Kumar, CSE IITD 7

Switched Networks • • BUS Shared media Lower Cost Lower throughput Scalability poor Anshul Kumar, CSE IITD • • Switched Network Switched paths Higher cost Higher throughput Scalability better 8

Interconnection Networks • • Topology : who is connected to whom Direct / Indirect : where is switching done Static / Dynamic : when is switching done Circuit switching / packet switching : how are connections established • Store & forward / worm hole routing : how is the path determined • Centralized / distributed : how is switching controlled • Synchronous/asynchronous : mode of operation Anshul Kumar, CSE IITD 9

P node M S link S node M P link P node M S link S node M P DIRECT Anshul Kumar, CSE IITD P node M link SWITCH Direct and Indirect Networks INDIRECT 10

Static and Dynamic Networks • Static Networks – fixed point to point connections – usually direct – each node pair may not have a direct connection – routing through nodes • Dynamic Networks – connections established as per need – usually indirect – path can be established between any pair of nodes – routing through switches Anshul Kumar, CSE IITD 11

Static Network Topologies Non-uniform connectivity Linear 2 D-Mesh Tree Star Anshul Kumar, CSE IITD 12

Static Networks Topologies- contd. Uniform connectivity Ring Torus Fully Connected Anshul Kumar, CSE IITD 13

Illiac IV Mesh Network 0 1 2 3 4 5 6 7 8 neighbors of node r : (r 1) mod 9 and (r 3) mod 9 Anshul Kumar, CSE IITD 0 8 1 7 2 6 3 5 4 Chordal Ring 14

Fat Tree Network Anshul Kumar, CSE IITD 15

Dynamic Networks k k cross -bar switch building block for multi-stage dynamic networks simplest cross-bar straight exchange Anshul Kumar, CSE IITD 2 2 switch upper broadcast lower broadcast 16

Baseline Network 000 001 010 011 100 101 110 111 blocking can occur Anshul Kumar, CSE IITD 17

Benes Network non-blocking Anshul Kumar, CSE IITD 18

Switching Mechanism • Circuit Switching (connection oriented communication) – A circuit is established between the source and the destination • Packet Switching (connectionless communication) – Information is divided into packets and each packet is sent independently from node to node Anshul Kumar, CSE IITD 19

Routing in Networks incoming message store & forward routing node outgoing message header payload/data time worm hole routing Anshul Kumar, CSE IITD 20

Routing in presence of congestion • Worm hole routing – When message header is blocked, many links get blocked with the message • Solution: cut-through routing – When message header is blocked, tail is allowed to move, compressing the message into a single node Anshul Kumar, CSE IITD 21

Routing Options • Deterministic routing: always same path followed • Adaptive routing: best path selected to minimize congestion • Source based routing: message specifies path to destination • Destination based routing: message specifies only destination address Anshul Kumar, CSE IITD 22

Some Performance Parameters sender overhead Tx time=bytes/BW time of flight receiver Tx time=bytes/BW overhead transport latency total latency time Anshul Kumar, CSE IITD 23

Other Parameters • • • Throughput Bandwidth (no credit for header) Bisection bandwidth = BW across a bisection Node degree Network Diameter Cost Fault Tolerance Anshul Kumar, CSE IITD 24

Multidimensional Grid/Mesh Size =k k …. k (n times) =kn Diameter = (k-1) n without end around connections = k n /2 with end around connections for (Binary) Hypercube : k = 2 k-ary n-cube Anshul Kumar, CSE IITD 25

Grid/Mesh Performance - 1 kd Anshul Kumar, CSE IITD 26

Grid/Mesh Performance - 2 Anshul Kumar, CSE IITD 27

Grid/Mesh Performance - 3 k-ary n-cube Anshul Kumar, CSE IITD 28

Switch Performance k m cross -bar switch Anshul Kumar, CSE IITD 29

Switch Performance – contd. Anshul Kumar, CSE IITD 30

Switch Performance – contd. Anshul Kumar, CSE IITD 31

Effect of re-submitted requests Anshul Kumar, CSE IITD 32

Effect of buffering There are two possibilities • Buffering before switching (k buffers, one at each input port) • Buffering after switching (m buffers, one at each output port) Anshul Kumar, CSE IITD 33

Switch with input buffers Rate of messages at input and output of each queue is same in steady state - r per cycle Service time includes delays due to conflicts, calculated as earlier. This has an exponential distribution – recall the analysis for a shared bus. M/M/1 open queue model can be used to calculate queuing delay. Details are omitted. Anshul Kumar, CSE IITD 34

Switch with output buffers Here we assume that all the messages destined for same output are queued in the same buffer, in some order. That is no rejections and no re-submissions. For each queue, Messages arriving per service cycle = = Prob of a request coming from one of the k sources = p = Apply MB/D/1 model for finding queuing delay Tw Anshul Kumar, CSE IITD 35

References • D. Sima, T. Fountain, P. Kacsuk, "Advanced Computer Architectures : A Design Space Approach", Addison Wesley, 1997. • K. Hwang, "Advanced Computer Architecture : Parallelism, Scalability, Programmability", Mc. Graw Hill, 1993. Anshul Kumar, CSE IITD 36