UNIVERSITY OF ELECTRONIC SCIENCE TECHNOLOGY OF CHINA IEEE

UNIVERSITY OF ELECTRONIC SCIENCE & TECHNOLOGY OF CHINA IEEE INFOCOM 2015, Hong Kong RAPIER: Integrating Routing and Scheduling for Coflow-aware Data Center Networks Yangming Zhao (UESTC), Kai Chen (HKUST), Wei Bai (HKUST), Minlan Yu (USC), Chen Tian (HUST), Yanhui Geng (Huawei), Yiming Zhang (NUDT), Dan Li (Tsinghua), Sheng Wang (UESTC) zhaoyangming@uestc. edu. cn

UNIVERSITY OF ELECTRONIC SCIENCE & TECHNOLOGY OF CHINA IEEE INFOCOM 2015, Hong Kong Coflow-aware Traffic Optimization • Why traffic optimization in data center networks? – Improve traffic scalability – Improve Qo. S • Why coflow-aware? – Minimize average flow completion time – Minimize average coflow completion time In cluster computing frameworks, a stage cannot complete, or sometimes even start, before it receives all the flows in a coflow from the previous stage An individual flow can be treated as a special coflow • How to optimize network traffic? – Routing (Hedera, Micro-TE) – Scheduling (Varys, Baraat, p. Fabric) Why not joint optimization? p. 2

UNIVERSITY OF ELECTRONIC SCIENCE & TECHNOLOGY OF CHINA IEEE INFOCOM 2015, Hong Kong Motivation Example Two coflows: Coflow a: fa 1=40 Mb, fa 2=100 Mb; Coflow b: fb 1=60 Mb, fb 2=100 Mb Link bandwidths are all 100 Mbps Case 1: ECMP + Scheduling Average CCT=1. 5 ms Traffic unbalance may occur due to the route collision incurred by ECMP p. 3

UNIVERSITY OF ELECTRONIC SCIENCE & TECHNOLOGY OF CHINA IEEE INFOCOM 2015, Hong Kong Motivation Example Two coflows: Coflow a: fa 1=40 Mb, fa 2=100 Mb; Coflow b: fb 1=60 Mb, fb 2=100 Mb Link bandwidths are all 100 Mbps Case 2: Coflow-agnostic Load balancing + Scheduling Average CCT=1. 5 ms Consider routing and scheduling separately cannot optimize average CCT Routing should also take flow dependence in a coflow into account p. 4

UNIVERSITY OF ELECTRONIC SCIENCE & TECHNOLOGY OF CHINA IEEE INFOCOM 2015, Hong Kong Motivation Example Two coflows: Coflow a: fa 1=40 Mb, fa 2=100 Mb; Coflow b: fb 1=60 Mb, fb 2=100 Mb Link bandwidths are all 100 Mbps Case 3: Coflow-aware routing + scheduling Average CCT=1. 3 ms Jointly optimize routing and scheduling can minimize average CCT p. 5

UNIVERSITY OF ELECTRONIC SCIENCE & TECHNOLOGY OF CHINA IEEE INFOCOM 2015, Hong Kong Desirable Properties of RAPIER p. 6

UNIVERSITY OF ELECTRONIC SCIENCE & TECHNOLOGY OF CHINA IEEE INFOCOM 2015, Hong Kong Main idea • Coflow-level Routing – Distribute all the flows in a coflow evenly in the network • Coflow-level Scheduling – Minimal remaining time first principle • Starvation-free – Scheduling a coflow first if it is waiting for a long time • Work-conserving – Distribute all the bandwidth if there is a demand to serve • Coexistence – Route mice flows with ECMP and highest priority p. 7

UNIVERSITY OF ELECTRONIC SCIENCE & TECHNOLOGY OF CHINA IEEE INFOCOM 2015, Hong Kong RAPIER in a Nutshell For starvation-free For minimal remaining time first For work-conserving p. 8

UNIVERSITY OF ELECTRONIC SCIENCE & TECHNOLOGY OF CHINA IEEE INFOCOM 2015, Hong Kong Minimize single coflow completion time Non -lin ear e Let ai=1/ti l b aria v ger e t h in Relax integer constraint t i w r a ine l n No Non Let Lin e ar pro gra mm ing mk k ij=aix ij -lin ear with inte ger vari able wit hou t in teg er var iab le Route demand i to j on the path with largest x and resolve (2) p. 9

UNIVERSITY OF ELECTRONIC SCIENCE & TECHNOLOGY OF CHINA IEEE INFOCOM 2015, Hong Kong Relaxation and Rounding Problem (2) Theorem 1: Assume the minimum CCT is tmin and talg is the CCT obtained by Algorithm 2, then where K is the number of candidate paths for each flow Problem (4) p. 10

UNIVERSITY OF ELECTRONIC SCIENCE & TECHNOLOGY OF CHINA IEEE INFOCOM 2015, Hong Kong Bandwidth Allocation Large coflow first for starvation-free Large flow first to reduce CCT p. 11

UNIVERSITY OF ELECTRONIC SCIENCE & TECHNOLOGY OF CHINA IEEE INFOCOM 2015, Hong Kong Implementation • Central controller – Algorithm 1 • End host enforcement modules – Open. Flow based explicit routing – Bandwidth enforcement No device modification is required!! p. 12

UNIVERSITY OF ELECTRONIC SCIENCE & TECHNOLOGY OF CHINA IEEE INFOCOM 2015, Hong Kong Experiment on Testbed • Pronto 3295 48 -port Gigabit Ethernet switch with Pic. OS 2. 04 system • Each server has a 4 -core Intel E 5 -1410 2. 8 GHz CPU, 8 G memory, 500 GB hard disk and 1 G Ethernet NICs • The OS of servers is Debian 6. 0 64 bit version with Linux 2. 6. 38. 3 kernel p. 13

UNIVERSITY OF ELECTRONIC SCIENCE & TECHNOLOGY OF CHINA IEEE INFOCOM 2015, Hong Kong Experiment Results Coflow ID 1 2 3 Flow ID source 1 M 4 3. 17 2 M 5 5. 29 3 4 M 3 M 8 M 9 M 6 5. 29 10. 6 5 6 M 7 M 5 M 4 5. 29 17. 9 7 Destination Volume(GB) M 9 M 6 Average completion time 10. 6 Coflow Completion Time(s) RAPIER Routing Baseline 50. 6 84. 1 107. 1 100. 9 203. 0 289. 5 201. 1 204. 1 289. 2 117. 5 163. 7 228. 6 RAPIER can save 48. 6% of the average CCT compared to the baseline scheme, and it can reduce the average CCT by 28. 22% compared to the routing-only scheme p. 14

UNIVERSITY OF ELECTRONIC SCIENCE & TECHNOLOGY OF CHINA IEEE INFOCOM 2015, Hong Kong Simulation Settings • • • C/C++ based flow level simulator CPLEX 10. 0 for solving LP Fattree、VL 2 with 512 servers Flows in a coflow arrive simultaneously Inter-coflow arrival rate follows a Poisson distribution p. 15

UNIVERSITY OF ELECTRONIC SCIENCE & TECHNOLOGY OF CHINA IEEE INFOCOM 2015, Hong Kong Impact of coflow width • Reduce average CCT by up to 79. 44% in Fattree, and 55. 55% in VL 2 • Routing-only scheme performs better when coflow width is small. • Scheduling-only scheme performs better when coflow width is large. p. 16

UNIVERSITY OF ELECTRONIC SCIENCE & TECHNOLOGY OF CHINA IEEE INFOCOM 2015, Hong Kong Impact of coflow number • RAPIER keeps relatively stable performance with different coflow number. • Scheduling-only scheme is more effective in VL 2 than in Fattree p. 17

UNIVERSITY OF ELECTRONIC SCIENCE & TECHNOLOGY OF CHINA IEEE INFOCOM 2015, Hong Kong Impact of inter-coflow arrival interval • The average CCT is decreased with the increase of average intercoflow arrival interval • The same trend as scheduling-only scheme when the inter-coflow arrival interval is small • The same trend as routing-only scheme when the inter-coflow arrival interval is large p. 18

UNIVERSITY OF ELECTRONIC SCIENCE & TECHNOLOGY OF CHINA IEEE INFOCOM 2015, Hong Kong Simulation Results Summary • In light-load scenario, routing contributes more by solving the flow path collision problem in ECMP. • In heavy-load scenario, scheduling contributes more by determining the sending order of flows/coflows. • RAPIER integrates both schemes and gets all the benefits from them. p. 19

UNIVERSITY OF ELECTRONIC SCIENCE & TECHNOLOGY OF CHINA IEEE INFOCOM 2015, Hong Kong Conclusion • RAPIER is a system which optimizes average coflow completion time in DCNs by integrating routing and scheduling. • RAPIER follows the minimal remaining time first to reduce the average coflow completion time. • We implement the prototype of RAPIER • Simulation results show that RAPIER can greatly reduce the average coflow completion time in DCNs. p. 20

UNIVERSITY OF ELECTRONIC SCIENCE & TECHNOLOGY OF CHINA IEEE INFOCOM 2015, Hong Kong • The end! • Thanks for your attention! p. 21