ECE 526 Network Processing Systems Design Switching Fabric
ECE 526 – Network Processing Systems Design Switching Fabric D. E. Comer: chapter 10
Overview • Computation ─ Single CPU is not fast enough for processing packets ─ Multiple optimized processors (e. g. Network Processors) ─ Multiple ports with each port powered by NPs • Storage ─ Memory is bottleneck (memory wall) ─ On-chip, heterogeneous memories ─ Multi-threading to hide long memory access • Communication ─ Bus: data, control and address ─ Memory: can also used for communication Ning Weng ECE 526 2
Router Data Path Port Network Interface Network Processor Processing Engine Interconnect Processing Engine I/O • How can ports be connected? ─ “System backplane” • Switching Fabric: ─ Hardware mechanism serving as backplane by providing path for data flow among the ports Ning Weng ECE 526 3
Switching Fabric Requirement • Connectivity: ─ interconnection between ports and control processor • High throughput • Low cost • Scalability ─ Scale with data rate on input/output (one port) ─ Scale with packet rate on input/output (one port) ─ Scale with number of input/output ports • Feature ─ Support transfer of unicast, multicast, and broadcast packets Conflicting requirements: tradeoffs necessary Ning Weng ECE 526 4
Switching Fabric Types • Synchronous vs. asynchronous ─ Sync: regular traffic (e. g. , fixed cells); async: variable size (e. g. , packets) ─ In practice synchronous fabrics are used ─ Packets are divided into fixed-sized cells • Multiplexing: ─ Time Division: single or few paths shared among many ports ─ Space Division: many paths used for less delay • Stage: ─ Single or multiple Ning Weng ECE 526 5
Switching Fabric Design Spectrum • Design spectrum • Evaluation criteria ─ Low cost ─ Performance ─ Scalability • Design option ─ Data transferred granularity ─ Queuing for contention Ning Weng ECE 526 6
Shared Bus Fabric • Disadvantage: lower aggregate data rate–Hardware needs to operate at N times faster than ports • What granularity is best for access: packet, cells? Ning Weng ECE 526 7
Granularity for Bus • Packet ─ Pro. : Simple hardware ─ Con. : delay- small packet suffers waiting for large pkt • Fixed size block ─ Pro. : multiple transfer concurrently ─ Con. : overhead from switching senders Ning Weng ECE 526 8
Shared Memory Fabric • Input ports deposit packets in memory • Problem: cost of memory interfaces Ning Weng ECE 526 9
Fully Interconnected Fabric • Separate physical data path between each port pair • Advantage and disadvantage? Ning Weng ECE 526 10
Fully Interconnected Fabric • Advantage ─ possible N concurrently transfers • Disadvantage ─ Port contention: two input port to the same output simultaneously • worst case: only one transfer ─ Expensive: • N*M wires • Output port needs contention circuitry • Question: is it possible to N concurrently transfers but not N*M wires ─ Observation: for each output port, only one of N wires is used at anytime Ning Weng ECE 526 11
Crossbar Fabric • Switched interconnect • Allows multiple Simultaneous connections • Controller decides on assignment of active connections • Pros and cons? Ning Weng ECE 526 12
Crossbar Fabric • Pros: ─ Using N + M wires to achieve maximum simultaneous transfers up to Min(N, M) • High aggregate data rate • Cons: N * M switches Ning Weng ECE 526 13
Contention • What happens if multiple input ports send to the same output port? ─ Causes “port contention” ─ Is this a problem of the fabric architecture? • How can contention be addressed? ─ Queues that buffer packets Ning Weng ECE 526 14
Input Queueing • Incoming packet traffic is burst • Head of line blocking ─ The first packet in the queue to output port s, which is busy ─ The second packet in the queue to output port t, which is idle • Solution for head of line blocking ─ Multiple queues for each port ─ Change queue from FCFS to randomly access ─ Virtual queueing: • based on estimate of processing time to reserve packet position in the queue • requires complex coordination • What is right size of input queue ─ Upper bound: port rate Ning Weng ECE 526 15
Output Queueing • Bursts from inputs to one output are buffered on output side • Queue size • Too small -> packet dropped • Too large -> keeping packet are longer valid – Voice packet arrived to destination too late – TCP packet already retransmitted Ning Weng ECE 526 16
Queuing Summary • Variety of queuing designs possible • Input queuing: ─ Packets are buffered on input side until output is available ─ Problem: head of line blocking • Output queuing: ─ Bursts from inputs to one output are buffered on output side ─ Problem: queue size, switch fabric speedup • Virtual queuing: ─ Partial output queues are maintained on input size ─ Each port gets rate or virtual time assigned for sending ─ Problem: requires complex coordination Ning Weng ECE 526 17
Multistage Fabrics • None of the fabrics so far really scales ─ ─ Full interconnect: squared cost with port number Crossbar: limitations in controller Bus: interconnect needs to be N times faster than port Shared Memory: memory interfaces are costly and not scalable • Multistage fabrics: ─ Multiple “steps” between input and output ─ Provides multiple data paths, but not as many as fully connected ─ Packets can be queued or not on each stage • Nice properties: ─ Scalability ─ Self routing Ning Weng ECE 526 18
Switching Elements • Switching element is basic component: ─ Two inputs, two outputs ─ Two different settings • Straight through • Crossover ─ Setting is determined by “label” • Multiple switching elements make up larger fabric • Various topologies possible: ─ ─ Banyan Benes Delta Etc. Ning Weng ECE 526 19
Banyan Switch • From 2 -port switch to 4 -port switch: • Can be applied recursively Ning Weng ECE 526 20
Banyan Switch • 8 -port: ─ Recursive extension • Self-routing: ─ Based on output port “label” each stage can make local decision ─ Recursive structure will lead to correct destination Ning Weng ECE 526 21
Nice Feature of Banyan • Scalability ─ Can be operated at lower rate than a shared bus ─ Applying to arbitrary number of input ports ─ Modular design: cheap 2 -input switch • Self-routing ─ Each output port including internal port is identified by a unique binary label ─ Sender prefixes the appropriate label on the packet ─ At each stage, switch extract the next bit and use it to determines the path • Internal packet queues allowed but not required ─ With queues will increase throughput, packet will move forward as long as it allowed ─ Without queues is cheaper, path established first, then send it Ning Weng ECE 526 22
Banyan Switch • Scalability due to recursive structure • Contention can occur ─ How? ─ Usually due to overloading of one output • Other multistage switch fabrics: ─ Different recursive rules ─ Additional stages: distribution to avoid blocking • Practical note: ─ Standards define Switch Fabric Interfaces Ning Weng ECE 526 23
Exercises 1 • Draw a banyan network for 8 input with full details, and label intermediate connection with label prefix to reach to the connection. Ning Weng ECE 526 24
Exercise 2 • Derive the number of stage required in a Banyan switch with 2 n inputs. How many of 2 -input switch required? How many queues required if queues allowed? Ning Weng ECE 526 25
Exercises 3 • What is the maximum number of transfers that can occur in a 8 -input Banyan switch in the best case? Does the answer changes with or without buffers? (transfers defines as a connection from input of switch to its output) • What is the maximum number of transfers that can occur in a 8 -input Banyan switch in the worst case? Does the answer changes with or without buffers? Ning Weng ECE 526 26
Summary • Switching fabric provides connections inside single network system • Two basic approaches – Time-division has lowest cost – Space-division has highest performance • Multistage designs compromise between two • Banyan fabric is example of multistage 27
Ning Weng ECE 526 28
- Slides: 28