EECS 262 a Advanced Topics in Computer Systems

EECS 262 a Advanced Topics in Computer Systems Lecture 18 Software Routers/Route. Bricks November 4 th, 2013 John Kubiatowicz and Anthony D. Joseph Electrical Engineering and Computer Sciences University of California, Berkeley Slides Courtesy: Sylvia Ratnasamy http: //www. eecs. berkeley. edu/~kubitron/cs 262

Today’s Paper • Route. Bricks: Exploiting Parallelism To Scale Software Routers Mihai Dobrescu and Norbert Egi, Katerina Argyraki, Byung-Gon Chun, Kevin Fall Gianluca Iannaccone, Allan Knies, Maziar Manesh, Sylvia Ratnasamy. Appears in Proceedings of the 22 nd ACM Symposium on Operating Systems Principles (SOSP), October 2009 • Thoughts? • Paper divided into two pieces: – Single-Server Router – Cluster-Based Routing 11/4/2013 cs 262 a-S 13 Lecture-18 2

Networks and routers UCB AT&T NYU HP 11/4/2013 MIT cs 262 a-S 13 Lecture-18 3

Routers forward packets UCB to MIT Router 4 payload Router 2 header Route Table 111010010 MIT Router 1 Destination Address Next Hop Router UCB 4 HP 5 MIT 2 NYU 3 Router 5 to HP to NYU Router 3 11/4/2013 cs 262 a-S 13 Lecture-18 4

Router definitions 1 N 2 N-1 3 … 5 R bits per second (bps) 4 • N = number of external router `ports’ • R = line rate of a port • Router capacity = N x R 11/4/2013 cs 262 a-S 13 Lecture-18 5

Networks and routers edge (enterprise) UCB home, small business MIT AT&T core edge (ISP) HP 11/4/2013 core cs 262 a-S 13 Lecture-18 NYU 6

Examples of routers (core) Juniper T 640 • R= 2. 5/10 Gbps • NR = 320 Gbps Cisco CRS-1 • R=10/40 Gbps • NR = 46 Tbps 11/4/2013 cs 262 a-S 13 Lecture-18 72 racks, 1 MW 7

Examples of routers (edge) Cisco ASR 1006 • R=1/10 Gbps • NR = 40 Gbps Juniper M 120 • R= 2. 5/10 Gbps • NR = 120 Gbps 11/4/2013 cs 262 a-S 13 Lecture-18 8

Examples of routers (small business) Cisco 3945 E • R = 10/1000 Mbps • NR < 10 Gbps 11/4/2013 cs 262 a-S 13 Lecture-18 9

Building routers • edge, core – ASICs – network processors – commodity servers Route. Bricks • home, small business – ASICs – network, embedded processors – commodity PCs, servers 11/4/2013 cs 262 a-S 13 Lecture-18 10

Why programmable routers • New ISP services – intrusion detection, application acceleration • Simpler network monitoring – measure link latency, track down traffic • New protocols – IP traceback, Trajectory Sampling, … Enable flexible, extensible networks 11/4/2013 cs 262 a-S 13 Lecture-18 11

Challenge: performance • deployed edge/core routers – port speed (R): 1/10/40 Gbps – capacity (Nx. R): 40 Gbps to 40 Tbps • PC-based software routers – capacity (Nx. R), 2007: 1 -2 Gbps [Click] – capacity (Nx. R), 2009: 4 Gbps [Vyatta] • subsequent challenges: power, form-factor, … 11/4/2013 cs 262 a-S 13 Lecture-18 12

A single-server router sockets with cores mem server I/O hub memory controllers (integrated) ports N router links 11/4/2013 cs 262 a-S 13 Lecture-18 point-to-point links (e. g. , QPI) Network Interface Cards (NICs) 13

Packet processing in a server cores mem I/O hub 11/4/2013 Per packet, cores 1. core polls input port 2. NIC writes packet to memory 3. core reads packet 4. core processes packet (address lookup, checksum, etc. ) 5. core writes packet to port cs 262 a-S 13 Lecture-18 14

Packet processing in a server 8 x 2. 8 GHz cores mem I/O hub Assuming 10 Gbps with all 64 B packets à 19. 5 million packets per second à one packet every 0. 05 µsecs à~1000 cycles to process a packet Today, 200 Gbps memory Today, 144 Gbps I/O Suggests efficient use 10 Gbps? of CPU cycles is key! Teaser: 11/4/2013 cs 262 a-S 13 Lecture-18 15

Lesson#1: multi-core alone isn’t enough `Older’ (2008) cores bot t Current (2009) len cores eck Shared frontside bus `chipset’ cores mem I/O hub mem Memory controller in `chipset’ Hardware need: avoid shared-bus servers 11/4/2013 cs 262 a-S 13 Lecture-18 16

Lesson#2: on cores and ports poll input transmit cores ports output ports How do we assign cores to input and output ports? 11/4/2013 cs 262 a-S 13 Lecture-18 17

Lesson#2: on cores and ports Problem: locking Hence, rule: one core per port 11/4/2013 cs 262 a-S 13 Lecture-18 18

Lesson#2: on cores and ports L 3 cache ok lo ll oll o p p up +t x u lo p+ ok t x lo up+ ok tx up +t x Problem: cache misses, inter-core communication L 3 cache tx +tx +tx +tx + p up up up u k ok ok ok o lo +lo +lo +lo + ll oll oll oll o p p p p L 3 cache packettransferred (may be) transferred packet between cores across caches L 3 cache packet stays at one core packet always in one cache parallel pipelined Hence, rule: one core per packet 11/4/2013 cs 262 a-S 13 Lecture-18 19

Lesson#2: on cores and ports • two rules: – one core per port – one core per packet • problem: often, can’t simultaneously satisfy both Example: when #cores > #ports one core per packet one core per port • solution: use multi-Q NICs 11/4/2013 cs 262 a-S 13 Lecture-18 20

Multi-Q NICs • feature on modern NICs (for virtualization) – port associated with multiple queues on NIC – NIC demuxes (muxes) incoming (outgoing) traffic – demux based on hashing packet fields (e. g. , source+destination address) Multi-Q NIC: incoming traffic 11/4/2013 Multi-Q NIC: outgoing traffic cs 262 a-S 13 Lecture-18 21

Multi-Q NICs • feature on modern NICs (for virtualization) • repurposed for routing – rule: one core per port queue – rule: one core per packet • if #queues per port == #cores, can always enforce both rules 11/4/2013 cs 262 a-S 13 Lecture-18 22

Lesson#2: on cores and ports recap: • use multi-Q NICs – with modified NIC driver for lock-free polling of queues • with – one core per queue (avoid locking) – one core per packet (avoid cache misses, intercore communication) 11/4/2013 cs 262 a-S 13 Lecture-18 23

Lesson#3: book-keeping cores mem I/O hub ports 1. 2. 3. 4. 5. core polls input port NIC writes packet to memory core reads packet core processes packet core writes packet to out port and packet descriptors problem: excessive per packet book-keeping overhead • solution: batch packet operations – NIC transfers packets in batches of `k’ 11/4/2013 cs 262 a-S 13 Lecture-18 24

Recap: routing on a server Design lessons: 1. parallel hardware » at cores and memory and NICs 2. careful queue-to-core allocation » one core per queue, per packet 3. reduced book-keeping per packet » modified NIC driver w/ batching (see paper for “non needs” – careful memory placement, etc. ) 11/4/2013 cs 262 a-S 13 Lecture-18 25

Single-Server Measurements: Experimental setup • test server: Intel Nehalem (X 5560) – dual socket, 8 x 2. 80 GHz cores – 2 x NICs; 2 x 10 Gbps ports/NIC cores mem I/O hub max 40 Gbps 10 Gbps additional servers generate/sink test traffic 11/4/2013 cs 262 a-S 13 Lecture-18 26

Project Feedback from Meetings • Should have Updated your project descriptions and plan – Turn your description/plan into a living document in Google Docs – Share Google Docs link with us – Update plan/progress throughout the semester • Questions to address: – – – 11/4/2013 What is your evaluation methodology? What will you compare/evaluate against? Strawman? What are your evaluation metrics? What is your typical workload? Trace-based, analytical, … Create a concrete staged project execution plan: » Set reasonable initial goals with incremental milestones – always have something to show/results for project cs 262 a-S 13 Lecture-18 27

Midterm • Out this afternoon • Due 11: 59 PM PST a week from Tuesday (11/12) • Rules: – Open book – No collaboration with other students 11/4/2013 cs 262 a-S 13 Lecture-18 28

Experimental setup • test server: Intel Nehalem (X 5560) cores mem packet • software: kernel-mode Click [TOCS’ 00] – with modified NIC driver (batching, multi-Q) cores processing me m Click runtime I/O hub modified NIC driver 10 Gbps additional servers generate/sink test traffic 11/4/2013 cs 262 a-S 13 Lecture-18 29

Experimental setup • test server: Intel Nehalem (X 5560) cores mem packet • software: kernel-mode Click [TOCS’ 00] – with modified NIC driver cores processing me m Click runtime I/O hub modified NIC driver • packet processing – static forwarding (no header processing) – IP routing » trie-based longest-prefix address lookup » ~300, 000 table entries [Route. Views] » checksum calculation, header updates, etc. 11/4/2013 cs 262 a-S 13 Lecture-18 10 Gbps additional servers generate/sink test traffic 30

Experimental setup • test server: Intel Nehalem (X 5560) cores mem packet • software: kernel-mode Click [TOCS’ 00] – with modified NIC driver cores processing me m Click runtime I/O hub modified NIC driver • packet processing – static forwarding (no header processing) – IP routing • input traffic – all min-size (64 B) packets (maximizes packet rate given port speed R) – realistic mix of packet sizes [Abilene] 11/4/2013 cs 262 a-S 13 Lecture-18 10 Gbps additional servers generate/sink test traffic 31

pkts/sec (M) Factor analysis: design lessons 19 5. 9 1. 2 older shared-bus server 2. 8 current Nehalem server Nehalem + `batching’ NIC driver Nehalem w/ multi-Q + `batching’ driver Test scenario: static forwarding of min-sized packets 11/4/2013 cs 262 a-S 13 Lecture-18 32

Single-server performance Gbps 40 Gbps 36. 5 9. 7 min-size packets realistic pkt sizes 6. 35 static forwarding IP routing Bottleneck: traffic generation Bottleneck? 11/4/2013 cs 262 a-S 13 Lecture-18 33

Bottleneck analysis (64 B pkts) Recall: max IP routing = 6. 35 Gbps 12. 4 M pkts/sec memory I/O Per-packet load due to routing Maximum component capacity – nominal (empirical) Max. packet rate as per component capacity -nominal (empirical) 725 bytes/pkt 51 (33) Gbytes/sec 70 (46) Mpkts/sec CPUs are the 191 bytes/pkt bottleneck 16 (11) Gbytes/sec 84 (58) Mpkts/sec Intersocket link 231 bytes/pkt 25 (18) Gbytes/sec 108 ( 78) Mpkts/sec CPUs 1693 cycles/pkt 22. 4 Gcycles/sec 13 Mpkts/sec Test scenario: IP routing of min-sized packets 11/4/2013 cs 262 a-S 13 Lecture-18 34

Recap: single-server performance current servers (realistic packet sizes) current servers (min-sized packets) 11/4/2013 R NR 1/10 Gbps 36. 5 Gbps 1 6. 35 (CPUs bottleneck) cs 262 a-S 13 Lecture-18 35

Recap: single-server performance With upcoming servers? (2010) 4 x cores, 2 x memory, 2 x I/O 11/4/2013 cs 262 a-S 13 Lecture-18 36

Recap: single-server performance R NR 1/10 Gbps 36. 5 Gbps 1 6. 35 (CPUs bottleneck) upcoming servers – estimated (realistic packet sizes) 1/10/40 146 upcoming servers – estimated (min-sized packets) 1/10 25. 4 current servers (realistic packet sizes) current servers (min-sized packets) 11/4/2013 cs 262 a-S 13 Lecture-18 37

Practical Architecture: Goal • scale software routers to multiple 10 Gbps ports • example: 320 Gbps (32 x 10 Gbps ports) – higher-end of edge routers; lower-end core routers 11/4/2013 cs 262 a-S 13 Lecture-18 38

A cluster-based router today interconnect ? 10 Gbps 11/4/2013 cs 262 a-S 13 Lecture-18 39

Interconnecting servers Challenges – any input can send up to R bps to any output 11/4/2013 cs 262 a-S 13 Lecture-18 40

A naïve solution N 2 internal links of capacity R R R 10 Gbps R problem: commodity servers cannot accommodate Nx. R traffic 11/4/2013 cs 262 a-S 13 Lecture-18 41

Interconnecting servers Challenges – any input can send up to R bps to any output » » – 11/4/2013 but need a low-capacity interconnect (~NR) i. e. , fewer (<N), lower-capacity (<R) links per server must cope with overload cs 262 a-S 13 Lecture-18 42

Overload drop at input servers? problem: requires global state need to drop 20 Gbps; (fairly across input ports) 10 Gbps drop at output server? problem: output might receive up to Nx. R traffic 11/4/2013 cs 262 a-S 13 Lecture-18 43

Interconnecting servers Challenges – any input can send up to R bps to any output » » – must cope with overload » » 11/4/2013 but need a lower-capacity interconnect i. e. , fewer (<N), lower-capacity (<R) links per server need distributed dropping without global scheduling processing at servers should scale as R, not Nx. R cs 262 a-S 13 Lecture-18 44

Interconnecting servers Challenges – any input can send up to R bps to any output – must cope with overload With constraints (due to commodity servers and NICs) – internal link rates ≤ R – per-node processing: cx. R (small c) – limited per-node fanout Solution: Use Valiant Load Balancing (VLB) 11/4/2013 cs 262 a-S 13 Lecture-18 45

Valiant Load Balancing (VLB) • Valiant et al. [STOC’ 81], communication in multiprocessors • applied to data centers [Greenberg’ 09], all-optical routers [Kesslassy’ 03], traffic engineering [Zhang. Shen’ 04], etc. • idea: random load-balancing across a lowcapacity interconnect 11/4/2013 cs 262 a-S 13 Lecture-18 46

VLB: operation phase 1 phase 2 R/N R/N R R R/N Packets arriving at external port 2 internal server sends upof tocapacity R/N (of traffic • N 2 internal links of capacity R/NEach • N links R/N are uniformly load balanced Output server transmits received phase-1) to output • each server receives up to R bps • eachinserver receives up toserver; R bps traffic on external port drops excess fairly Packets forwarded in two phases 11/4/2013 cs 262 a-S 13 Lecture-18 47

VLB: operation phase 1+2 R R • N 2 internal links of capacity 2 R/N • each server receives up to 2 R bps • plus R bps from external port • hence, each server processes up to 3 R • or up to 2 R, when traffic is uniform [direct. VLB, Liu’ 05] 11/4/2013 cs 262 a-S 13 Lecture-18 48

VLB: fanout? (1) fewer but faster links fewer but faster servers Multiple external ports per server (if server constraints permit) 11/4/2013 cs 262 a-S 13 Lecture-18 49

VLB: fanout? (2) Use extra servers to form a constant-degree multi-stage interconnect (e. g. , butterfly) 11/4/2013 cs 262 a-S 13 Lecture-18 50

Authors solution: • • assign maximum external ports per servers interconnected with commodity NIC links servers interconnected in a full mesh if possible else, introduce extra servers in a k-degree butterfly • servers run flowlet-based VLB 11/4/2013 cs 262 a-S 13 Lecture-18 51

Scalability • question: how well does clustering scale for realistic server fanout and processing capacity? • metric: number of servers required to achieve a target router speed 11/4/2013 cs 262 a-S 13 Lecture-18 52

Scalability Assumptions • 7 NICs per server • each NIC has 6 x 10 Gbps ports or 8 x 1 Gbps ports • current servers – one external 10 Gbps port per server (i. e. , requires that a server process 20 -30 Gbps) • upcoming servers – two external 10 Gbps port per server (i. e. , requires that a server process 40 -60 Gbps) 11/4/2013 cs 262 a-S 13 Lecture-18 53

Scalability (computed) 160 Gbps 320 Gbps 640 Gbps 1. 28 Tbps 2. 56 Tbps current servers 16 32 128 256 512 upcoming servers 8 16 32 128 256 Transition from mesh to butterfly Example: can build 320 Gbps router with 32 ‘current’ servers 11/4/2013 cs 262 a-S 13 Lecture-18 56

Implementation: the RB 8/4 Specs. 4 x Nehalem servers 2 x 10 Gbps external ports (Intel Niantic NIC) 11/4/2013 • 8 x 10 Gbps external ports • form-factor: 4 U • power: 1. 2 KW • cost: ~$10 k Key results (realistic traffic) • 72 Gbps routing • reordering: 0 -0. 15% • validated VLB bounds cs 262 a-S 13 Lecture-18 57

Is this a good paper? • What were the authors’ goals? • What about the evaluation/metrics? • Did they convince you that this was a good system/approach? • Were there any red-flags? • What mistakes did they make? • Does the system/approach meet the “Test of Time” challenge? • How would you review this paper today? 11/4/2013 cs 262 a-S 13 Lecture-18 58