Jellyfish Networking Data Centers Randomly Ankit Singla ChiYao

Jellyfish: Networking Data Centers Randomly Ankit Singla† , Chi-Yao Hong†, Lucian Popa*, P. Brighten Godfrey† † University of Illinois at Urbana–Champaign *HP Labs 1

Tow Goals of Data Center Design • High throughput • Incremental expansion • Easily add/replace servers & switches 2

Incremental expansion • Facebook doubled their data center scale in the last year (2010) • You can add the servers, how about the network? • Distributed systems: Hadoop 3

Today’s structured networks • CLOS • Three-stage hierarchical datacenter. • Single failure • Fat tree CLOS Fat tree 4

Structure constrains expansion • 3 -level fat trees, commodity switches: • 24 -port switches – 3, 456 servers • 32 -port switches – 8, 192 servers • 48 -port switches – 27, 648 servers • How to expand a datacenter with 32 -port switches, 8, 192 servers to 10, 000 servers? 5

Solution • No structure • Jellyfish: random graph 6

Quantifying expandability • Bisection bandwidth • if the network is bisected into two partitions, the bisection bandwidth of a network topology is the bandwidth available between the two partitions. 7

Throughput: Jellyfish vs Fat tree 8

Why it works? • It fully utilizes all available capacity 9

Example 10

Example: Fat tree 11

Example: Jellyfish 12

Jellyfish has short paths 13

Degree-diameter bounded graph 14

Degree-diameter vs. Jellyfish 15

Routing and congestion control • Routing • K-shortest path • MPLS TE, Open. Flow(SDN) • Congestion control • TCP, Multipath TCP 16

Cabling • One of the main drawbacks of this work • Discussion: possible solutions • wireless datacenter • Xia Zhou, et. al. . "Mirror mirror on the ceiling: Flexible wireless links for data centers. " ACM SIGCOMM Computer Communication Review 42, no. 4 (2012): 443 -454. • Daniel Halperin, et. al. . "Augmenting data center networks with multi-gigabit wireless links. " In ACM SIGCOMM Computer Communication Review, vol. 41, no. 4, pp. 38 -49. ACM, 2011. • Not work • Large latency for small flows • Short transmission range 17

Questions on Piazza • Is the fat-tree topology the most widely used data center network topology? Why weren’t other network topologies as heavily emphasized in the paper? • Are there any theoretical tradeoffs to Jellyfish since the physical and routing complications seemed to be mostly resolved with the author’s proposals? 18