High Performance Routers Slides originally by Nick Mc

High Performance Routers Slides originally by Nick Mc. Keown Professor of Electrical Engineering and Computer Science, Stanford University nickm@stanford. edu www. stanford. edu/~nickm Stanford High Performance Networking group: Nick Mc. Keown IEE, October 2001 http: //klamath. stanford. edu 1

Outline Background v v v What is a router? Why do we need faster routers? Why are they hard to build? Architectures and techniques v v The evolution of router architecture. IP address lookup. Packet buffering. Switching. The Future Nick Mc. Keown IEE, October 2001 2

What is Routing Forwarding? R 3 R 1 A R 4 B D C Nick Mc. Keown D E D R 2 Destination Next Hop D R 3 E R 3 F R 5 IEE, October 2001 R 5 F 3

What is Routing? R 3 R 1 A 4 16 D 1 D Ver HLen T. Service 20 bytes B Fragment ID TTL C Flags Fragment Offset Header Checksum Source Address R 2 Destination Address Destination Next Hop D Options (if any)R 3 F Nick Mc. Keown D D D 32 Total Packet Length Protocol E R 4 Data R 5 E F R 3 R 5 IEE, October 2001 4

What is Routing? R 3 A R 1 R 4 B C Nick Mc. Keown D E R 2 R 5 IEE, October 2001 F 5

Points of Presence (POPs) POP 2 A POP 1 POP 4 B C Nick Mc. Keown POP 3 E POP 5 POP 6 POP 7 IEE, October 2001 D POP 8 F 6

Where High Performance Routers are Used (2. 5 Gb/s) R 1 R 2 R 5 R 4 R 3 R 8 R 9 R 10 Nick Mc. Keown R 7 R 11 R 14 R 13 (2. 5 Gb/s) R 6 R 15 (2. 5 Gb/s) R 12 R 16 (2. 5 Gb/s) IEE, October 2001 7

What a Router Looks Like Cisco GSR 12416 Juniper M 160 19” Capacity: 160 Gb/s Power: 4. 2 k. W 6 ft 3 ft 2 ft Nick Mc. Keown Capacity: 80 Gb/s Power: 2. 6 k. W 2. 5 ft IEE, October 2001 8

Generic Router Architecture Header Processing Data Hdr Lookup Update IP Address Header IP Address 1 M prefixes Off-chip DRAM Nick Mc. Keown Queue Packet Data Hdr Next Hop Address Table Buffer Memory IEE, October 2001 1 M packets Off-chip DRAM 9

Generic Router Architecture Data Hdr Header Processing Lookup IP Address Buffer Manager Update Header Buffer Memory Address Table Data Hdr Header Processing Lookup IP Address Buffer Manager Update Header Data Memory. Hdr Header Processing Lookup IP Address Buffer Manager Update Header Buffer Memory Address Table Nick Mc. Keown Data Hdr Buffer Address Table Data Hdr IEE, October 2001 10

Why do we Need Faster Routers? 1. To prevent routers becoming the bottleneck in the Internet. 2. To increase POP capacity, and to reduce cost, size and power. Nick Mc. Keown IEE, October 2001 11

Why we Need Faster Routers 1: To prevent routers from being the bottleneck Packet processing Power Link Speed 10000 2 x / 7 months 100 10 1 1985 1990 1995 Fiber Capacity (Gbit/s) 2 x / 18 months 1000 2000 0, 1 TDM DWDM Source: SPEC 95 Int & David Miller, Stanford. Nick Mc. Keown IEE, October 2001 12

Why we Need Faster Routers 2: To reduce cost, power & complexity of POPs POP with smaller routers POP with large routers v v Nick Mc. Keown Ports: Price >$100 k, Power > 400 W. It is common for 50 -60% of ports to be for interconnection. IEE, October 2001 13

Why are Fast Routers Difficult to Make? 1. It’s hard to keep up with Moore’s Law: v The bottleneck is memory speed. v Memory speed is not keeping up with Moore’s Law. Nick Mc. Keown IEE, October 2001 14

Why are Fast Routers Difficult to Make? Speed of Commercial DRAM 1. It’s hard to keep up with Moore’s Law: v The bottleneck is memory speed. 1. 1 x /is 18 not monthskeeping up with v Memory speed Moore’s Law 2 x / 18 months Nick Mc. Keown IEE, October 2001 15

Why are Fast Routers Difficult to Make? 1. It’s hard to keep up with Moore’s Law: v The bottleneck is memory speed. v Memory speed is not keeping up with Moore’s Law. 2. Moore’s Law is too slow: v Routers need to improve faster than Moore’s Law. Nick Mc. Keown IEE, October 2001 16

Router Performance Exceeds Moore’s Law Growth in capacity of commercial routers: v v v Capacity 1992 ~ 2 Gb/s Capacity 1995 ~ 10 Gb/s Capacity 1998 ~ 40 Gb/s Capacity 2001 ~ 160 Gb/s Capacity 2003 ~ 640 Gb/s Average growth rate: 2. 2 x / 18 months. Nick Mc. Keown IEE, October 2001 17

Outline Background v v v What is a router? Why do we need faster routers? Why are they hard to build? Architectures and techniques v v The evolution of router architecture. IP address lookup. Packet buffering. Switching. The Future Nick Mc. Keown IEE, October 2001 18

First Generation Routers Shared Backplane Li CP n I U nt e er fa ce M em or y CPU Route Table Buffer Memory Line Interface MAC MAC Typically <0. 5 Gb/s aggregate capacity Nick Mc. Keown IEE, October 2001 19

Second Generation Routers CPU Route Table Buffer Memory Line Card Buffer Memory Fwding Cache MAC MAC Typically <5 Gb/s aggregate capacity Nick Mc. Keown IEE, October 2001 20

Third Generation Routers Switched Backplane Li I CP n ne Ute rf ac M e em or y Line Card CPU Card Line Card Local Buffer Memory Routing Table Local Buffer Memory Fwding Table MAC Typically <50 Gb/s aggregate capacity Nick Mc. Keown IEE, October 2001 21

Fourth Generation Routers/Switches Optics inside a router for the first time Optical links 100 s of metres Switch Core Nick Mc. Keown Linecards 0. 3 - 10 Tb/s routers in development IEE, October 2001 22

Outline Background v v v What is a router? Why do we need faster routers? Why are they hard to build? Architectures and techniques v v The evolution of router architecture. IP address lookup. Packet buffering. Switching. The Future Nick Mc. Keown IEE, October 2001 23

Generic Router Architecture Header Processing Lookup IP Address Buffer Manager Update Header Address Table Buffer Memory Header Processing Lookup IP Address Buffer Manager Update Header Buffer Memory Address Table Nick Mc. Keown IEE, October 2001 24

IP Address Lookup Why it’s thought to be hard: 1. It’s not an exact match: it’s a longest prefix match. 2. The table is large: about 120, 000 entries today, and growing. 3. The lookup must be fast: about 30 ns for a 10 Gb/s line. Nick Mc. Keown IEE, October 2001 25

IP Lookups find Longest Prefixes 128. 9. 176. 0/24 128. 9. 16. 0/21 128. 9. 172. 0/21 65. 0. 0. 0/8 0 128. 9. 0. 0/16 128. 9. 16. 14 142. 12. 0. 0/19 232 -1 Routing lookup: Find the longest matching prefix (aka the most specific route) among all prefixes that match the destination address. Nick Mc. Keown IEE, October 2001 26

IP Address Lookup Why it’s thought to be hard: 1. It’s not an exact match: it’s a longest prefix match. 2. The table is large: about 120, 000 entries today, and growing. 3. The lookup must be fast: about 30 ns for a 10 Gb/s line. Nick Mc. Keown IEE, October 2001 27

Address Tables are Large Source: Geoff Huston, Oct 2001 Nick Mc. Keown IEE, October 2001 28

IP Address Lookup Why it’s thought to be hard: 1. It’s not an exact match: it’s a longest prefix match. 2. The table is large: about 120, 000 entries today, and growing. 3. The lookup must be fast: about 30 ns for a 10 Gb/s line. Nick Mc. Keown IEE, October 2001 29

Lookups Must be Fast Nick Mc. Keown Year Line 40 B packets (Mpkt/s) 1997 1999 2001 2003 622 Mb/s 2. 5 Gb/s 10 Gb/s 40 Gb/s 1. 94 7. 81 31. 25 125 IEE, October 2001 30

IP Address Lookup Binary tries 0 1 f d e g h i abc Example a) b) c) d) e) f) g) h) i) j) Prefixes: 000010 00011 001 011 100 1010 1100 11110000 j Nick Mc. Keown IEE, October 2001 31

Prefix Length Distribution Source: Geoff Huston, Oct 2001 99. 5% prefixes are 24 -bits or shorter Nick Mc. Keown IEE, October 2001 32

24 -8 Direct Lookup Trie 0000…… 0000 24 bits 1111…… 1111 0 224 -1 8 bits 0 28 -1 v When pipelined, allows one lookup per memory access. v Although inefficient use of memory, total memory cost < $20. Nick Mc. Keown IEE, October 2001 33

IP Address Lookup Summary v Lookup limited by memory bandwidth. v Lookup uses high-degree trie. v State of the art: 10 Gb/s line rate. v Scales to: 40 Gb/s line rate. v By 2008, entire IPv 4 address space will fit on one $20 DRAM! Nick Mc. Keown IEE, October 2001 34

Outline Background v v v What is a router? Why do we need faster routers? Why are they hard to build? Architectures and techniques v v The evolution of router architecture. IP address lookup. Packet buffering. Switching. The Future Nick Mc. Keown IEE, October 2001 35

Generic Router Architecture Header Processing Lookup IP Address Buffer Queue Manager Packet Update Header Buffer Memory Address Table Header Processing Lookup IP Address Queue Buffer Packet Manager Update Header Buffer Memory Address Table Nick Mc. Keown IEE, October 2001 36

Fast Packet Buffers Example: 40 Gb/s packet buffer Size = RTT*BW = 10 Gb; 40 byte packets Write Rate, R 1 packet every 8 ns Buffer Manager Read Rate, R 1 packet every 8 ns Buffer Memory Use SRAM? Use DRAM? + fast enough random access time, but - too low density to store 10 Gb of data. + high density means we can store data, but - too slow (50 ns random access time). Nick Mc. Keown IEE, October 2001 37

Packet Caches Small ingress SRAM cache of FIFO heads Small ingress SRAM cache of FIFO tails Arriving Packets 60 59 58 57 56 55 1 97 96 2 Buffer Manager Q SRAM 91 90 89 88 87 4 3 2 1 5 4 3 2 1 6 5 4 3 2 1 2 Departing Packets Q b>>1 packets at a time DRAM Buffer Memory 54 53 52 51 50 10 9 8 7 6 5 8 7 6 11 10 9 8 7 DRAM Buffer Memory 95 94 93 92 91 90 89 88 87 86 1 Nick Mc. Keown 15 14 13 12 11 10 9 86 85 84 83 82 IEE, October 2001 1 2 Q 38

Packet Buffers Summary Packet buffers limited by memory bandwidth. v Packet buffer caches use hybrid SRAM+DRAM. v State of the art: 10 Gb/s line rate. v Scales to: 40 Gb/s line rate. v Nick Mc. Keown IEE, October 2001 39

Outline Background v v v What is a router? Why do we need faster routers? Why are they hard to build? Architectures and techniques v v The evolution of router architecture. IP address lookup. Packet buffering. Switching. The Future Nick Mc. Keown IEE, October 2001 40

Generic Router Architecture Data Hdr Header Processing Lookup IP Address Update Header 1 1 Buffer Memory Address Table Data Hdr Header Processing Lookup IP Address Queue Packet Update Header 2 2 NQueue times line rate Packet Buffer Memory Address Table N times line rate Data Hdr Header Processing Lookup IP Address Update Header N N Buffer Memory Address Table Nick Mc. Keown Queue Packet IEE, October 2001 41

Generic Router Architecture Data Hdr Header Processing Lookup IP Address Update Header Address Table Data Hdr Update Header Address Table Nick Mc. Keown Queue Packet 2 2 Data Hdr Buffer Memory Header Processing Lookup IP Address 1 Buffer Address Table Data Hdr 1 Data Memory. Hdr Header Processing Lookup IP Address Queue Packet Update Header Queue Packet Scheduler N N Buffer Data Memory. Hdr IEE, October 2001 42

Output Buffered Switches The best that any queueing system can achieve. Nick Mc. Keown IEE, October 2001 43

Input Buffered Switches Head of Line Blocking The best that any queueing system can achieve. Nick Mc. Keown IEE, October 2001 44

Head of Line Blocking Nick Mc. Keown IEE, October 2001 45

Virtual Output Queues Nick Mc. Keown IEE, October 2001 46

A Router with Virtual Output Queues The best that any queueing system can achieve. Nick Mc. Keown IEE, October 2001 47

Maximum Weight Matching A 1(n) A 11(n) S*(n) L 11(n) 1 1 D 1(n) A 1 N(n) AN(n) DN(n) AN 1(n) N ANN(n) N LNN(n) L 11(n) Maximum Weight Match LN 1(n) “Request” Graph Nick Mc. Keown IEE, October 2001 Bipartite Match 48

The Evolution of Switching Different weight functions, incomplete information, pipelining. Theory: Input Queueing (IQ) IQ + VOQ, Maximum weight matching 58% [Karol, 1987] 100% [M et al. , 1995] 100% [Various] Randomized algorithms 100% [Tassiulas, 1998] IQ + VOQ, Maximal size matching, Speedup of two. 100% [Dai & Prabhakar, 2000] Practice: Input Queueing (IQ) Nick Mc. Keown IQ + VOQ, Sub-maximal size matching e. g. PIM, i. SLIP. IEE, October 2001 Various heuristics, distributed algorithms, and amounts of speedup 49

Switching Summary v Routers use virtual output queues, and a centralized scheduler. State of the art: 2. 5 Tb/s. v Scales to: ~10 Tb/s. v v 100% throughput will be standard soon. Nick Mc. Keown IEE, October 2001 50

Current Internet Router Technology Summary v There are three potential bottlenecks: Address lookup, v Packet buffering, and v Switching. v v Techniques exist today for: 10+Tb/s Internet routers, with v 40 Gb/s linecards. v v But what comes next…? Nick Mc. Keown IEE, October 2001 51

Outline Background v v v What is a router? Why do we need faster routers? Why are they hard to build? Architectures and techniques v v The evolution of router architecture. IP address lookup. Packet buffering. Switching. The Future v v Nick Mc. Keown More parallelism. Eliminating schedulers. Introducing optics into routers. Natural evolution to circuit switching? IEE, October 2001 52

External Parallelism: Multiple Parallel Routers What we’d like: IP Router capacity 100 s of Tb/s R R Nx. N R R The building blocks we’d like to use: R R R Nick Mc. Keown R IEE, October 2001 53

Multiple parallel routers Load Balancing R R R R/k 1 2 … R/k … R R R k Nick Mc. Keown IEE, October 2001 54

Intelligent Packet Load-balancing Parallel Packet Switching Router R/k rate, R 1 R/k 1 1 rate, R 2 rate, R N N rate, R k Nick Mc. Keown Bufferless IEE, October 2001 55

Parallel Packet Switching Advantages v Single-stage of buffering v No excess link capacity v kh a power per subsystem i v kh a memory bandwidth i v kh a lookup rate i Nick Mc. Keown IEE, October 2001 56

Parallel Packet Switch Theorem If S > 2 k/(k+2) @ 2 then a parallel packet switch can precisely emulate a single big router. Nick Mc. Keown IEE, October 2001 57

Outline Background v v v What is a router? Why do we need faster routers? Why are they hard to build? Architectures and techniques v v The evolution of router architecture. IP address lookup. Packet buffering. Switching. The Future v v Nick Mc. Keown More parallelism. Eliminating schedulers. Introducing optics into routers. Natural evolution to circuit switching? IEE, October 2001 58

Eliminating schedulers Two-Stage Switch [Chang et al. , 2001] External Inputs Internal Inputs External Outputs 1 1 N N 1 N Load Balancing First Round-Robin Nick Mc. Keown IEE, October 2001 Second Round-Robin 59

Outline Background v v v What is a router? Why do we need faster routers? Why are they hard to build? Architectures and techniques v v The evolution of router architecture. IP address lookup. Packet buffering. Switching. The Future v v Nick Mc. Keown More parallelism. Eliminating schedulers. Introducing optics into routers. Natural evolution to circuit switching? IEE, October 2001 60

Do optics belong in routers? They are already there. v Connecting linecards to switches. Optical processing doesn’t belong on the linecard. v You can’t buffer light. v Minimal processing capability. Optical switching can reduce power. Nick Mc. Keown IEE, October 2001 61

Optics in routers Optical links Switch Core Nick Mc. Keown Linecards IEE, October 2001 62

Complex linecards Typical IP Router Linecard Optics Physical Layer Framing & Maintenance Lookup Tables Buffer & State Memory Packet Processing Buffer Mgmt & Scheduling Buffer & State Memory Switch Fabric Arbitration 10 Gb/s linecard: Number of gates: 30 M v Amount of memory: 2 Gbits v Cost: >$20 k v Power: 300 W v Nick Mc. Keown IEE, October 2001 63

Replacing the switch fabric with optics Typical IP Router Linecard Lookup Tables Packet Processing Optics Physical Layer Typical IP Router Linecard Buffer & State Memory electrical Buffer Mgmt & Scheduling Switch Fabric Framing & Maintenance Buffer Mgmt & Scheduling Buffer & State Memory 1. 2. 3. 4. MEMs. Packet Processing Optics Physical Layer Optics Framing & Maintenance Physical Layer Buffer Mgmt & Scheduling Typical IP Router Linecard optical Switch Fabric Framing & Maintenance Nick Mc. Keown Packet Processing Fast tunable lasers + passive optical couplers. Diffraction waveguides. Electroholographic materials. Buffer Mgmt & Scheduling Buffer & State Memory Buffer Mgmt & Scheduling Lookup Tables Arbitration Candidate technologies Typical IP Router Linecard Lookup Tables Buffer & State Memory Req/Grant Arbitration Buffer & State Memory Lookup Tables Buffer Mgmt & Scheduling Packet Processing Optics Framing & Maintenance Req/Grant IEE, October 2001 Physical Layer Buffer Mgmt & Scheduling Buffer & State Memory 64

The Stanford Phicticious Optical Router Li 160 Gb/s ar c e n 1 d# on ati essing n i erm proc ng t ne et ri • Li pack uffe b • IP cket • Pa 160320 Gb/s Optical 2 -stage Switch Line c 160320 Gb/s ard #62 5 • Li n • IP e term • Pa packe inatio n t cke t bu proce s ffe ring sing 40 Gb/s 100 Tb/s IP Router, 625 linecards, each operating at 160 Gb/s. Nick Mc. Keown IEE, October 2001 65

Outline Background v v v What is a router? Why do we need faster routers? Why are they hard to build? Architectures and techniques v v The evolution of router architecture. IP address lookup. Packet buffering. Switching. The Future v v Nick Mc. Keown More parallelism. Eliminating schedulers. Introducing optics into routers. Natural evolution to circuit switching? IEE, October 2001 66

Evolution to circuit switching • Optics enables simple, low-power, very high capacity circuit switches. • The Internet was packet switched for two reasons: v Expensive links: statistical multiplexing. v Resilience: soft-state routing. • Neither reason holds today. Nick Mc. Keown IEE, October 2001 67

Fast Links, Slow Routers Processing Power 2 x / 2 years Link Speed (Fiber) 2 x / 7 months Source: SPEC 95 Int; Prof. Miller, Stanford Univ. Nick Mc. Keown IEE, October 2001 68

Fewer Instructions per packet since 1996 Nick Mc. Keown IEE, October 2001 69

Outline Background v v v What is a router? Why do we need faster routers? Why are they hard to build? Architectures and techniques v v The evolution of router architecture. IP address lookup. Packet buffering. Switching. The Future v v Nick Mc. Keown More parallelism. Eliminating schedulers. Introducing optics into routers. Natural evolution to circuit switching? IEE, October 2001 70

References General 1. 2. 3. J. S. Turner “Design of a Broadcast packet switching network”, IEEE Trans Comm, June 1988, pp. 734 -743. C. Partridge et al. “A Fifty Gigabit per second IP Router”, IEEE Trans Networking, 1998. N. Mc. Keown, M. Izzard, A. Mekkittikul, W. Ellersick, M. Horowitz, “The Tiny Tera: A Packet Switch Core”, IEEE Micro Magazine, Jan-Feb 1997. Fast Packet Buffers 1. Nick Mc. Keown Sundar Iyer, Ramana Rao, Nick Mc. Keown “Design of a fast packet buffer”, IEEE HPSR 2001, Dallas. IEE, October 2001 71

IP Lookups 1. 2. 3. 4. 5. 6. Nick Mc. Keown References A. Brodnik, S. Carlsson, M. Degermark, S. Pink. “Small Forwarding Tables for Fast Routing Lookups”, Sigcomm 1997, pp 3 -14. B. Lampson, V. Srinivasan, G. Varghese. “ IP lookups using multiway and multicolumn search”, Infocom 1998, pp 124856, vol. 3. M. Waldvogel, G. Varghese, J. Turner, B. Plattner. “Scalable high speed IP routing lookups”, Sigcomm 1997, pp 25 -36. P. Gupta, S. Lin, N. Mc. Keown. “Routing lookups in hardware at memory access speeds”, Infocom 1998, pp 1241 -1248, vol. 3. S. Nilsson, G. Karlsson. “Fast address lookup for Internet routers”, IFIP Intl Conf on Broadband Communications, Stuttgart, Germany, April 1 -3, 1998. V. Srinivasan, G. Varghese. “Fast IP lookups using controlled prefix expansion”, Sigmetrics, June 1998. IEE, October 2001 72

References Switching • • • N. Mc. Keown, A. Mekkittikul, V. Anantharam, and J. Walrand. Achieving 100% Throughput in an Input-Queued Switch. IEEE Transactions on Communications, 47(8), Aug 1999. A. Mekkittikul and N. W. Mc. Keown, "A practical algorithm to achieve 100% throughput in input-queued switches, " in Proceedings of IEEE INFOCOM '98, March 1998. L. Tassiulas, “Linear complexity algorithms for maximum throughput in radio networks and input queued switchs, ” in Proc. IEEE INFOCOM ‘ 98, San Francisco CA, April 1998. D. Shah, P. Giaccone and B. Prabhakar, “An efficient randomized algorithm for input-queued switch scheduling, ” in Proc. Hot Interconnects 2001. J. Dai and B. Prabhakar, "The throughput of data switches with and without speedup, " in Proceedings of IEEE INFOCOM '00, Tel Aviv, Israel, March 2000, pp. 556 -- 564. C. -S. Chang, D. -S. Lee, Y. -S. Jou, “Load balanced Birkhoff-von Neumann switches, ” Proceedings of IEEE HPSR ‘ 01, May 2001, Dallas, Texas. Nick Mc. Keown IEE, October 2001 73

References Future • • • Nick Mc. Keown C. -S. Chang, D. -S. Lee, Y. -S. Jou, “Load balanced Birkhoffvon Neumann switches, ” Proceedings of IEEE HPSR ‘ 01, May 2001, Dallas, Texas. Pablo Molinero-Fernndez, Nick Mc. Keown "TCP Switching: Exposing circuits to IP" Hot Interconnects IX, Stanford University, August 2001 S. Iyer, N. Mc. Keown, "Making parallel packet switches practical, " in Proc. IEEE INFOCOM `01, April 2001, Alaska. IEE, October 2001 74