CQRD A Switchbased Approach to Flow Interference in

CQRD: A Switch-based Approach to Flow Interference in Data Center Networks Guo Chen Dan Pei, Youjian Zhao Tsinghua University, Beijing, China

The Problem Flow interference dramatically increases the flow completion time (FCT) of short delay-sensitive flows in data center networks (DCN) 2

Flow Interference • Short delay-sensitive flows (majority in DCN) have to wait a long time at switches for buffer and bandwidth resources occupied by a few of long bandwidth-greedy flows (e. g. , backup, replication) 3

Flow Interference • Short delay-sensitive flows (majority in DCN) have to wait a long time at switches for buffer and bandwidth resources occupied by a few of long bandwidth-greedy flows (e. g. , backup, replication) • Caused by coarse Output Queue (OQ) switch queue management scheme 4

Prior solutions • Transport Layer Rate Control: • • DCTCP [SIGCOMM’ 10] HULL [NSDI’ 12] D 2 TCP [SIGCOMM’ 12] D 3 [SIGCOMM’ 11] Modification to end host and/or switch hardware • Preemptive Flow Scheduling: • PDQ [SIGCOMM’ 12] • p. Fabric [SIGCOMM’ 13] New protocol stack and switch hardware 5

Intuition of CQRD Tackling the root cause of flow interference: Need a more fine-grained queue management scheme 6

The Goal • Goal: • • • Alleviate flow interference Reduce FCT of short delay-sensitive flows Maintain high goodput of long bandwidth-greedy flows • Objectives: • • • Transparent to end host No modification to protocol stack Based on underlying techniques available in commodity productions 7

Our Solution CQRD: A fine-grained switch queue management scheme to flow interference 8

AGENDA n Background & Motivation n Flow Interference n CQRD Design n Evaluation n Conclusion 9

Toy Example: Flow Interference in OQ Switch • NS 2 simulation parameters: • Link capacity=10 Gbps, Link delay=4 us, Total buffer size=288 KB, TCP initial window size=4, TCP initial RTO=200 us. • 8 x 8 switch connected to host 1 -8, Host 1 -5 sending 10 KB TCP flow to host 8, Host 6 -7 sending 100 MB TCP flow to host 8 10

Toy Example: Flow Interference in OQ Switch FCT Goodput Short flows completed in ~100 ms Goodput of short flows collapse 11

Toy Example: Flow Interference in OQ Switch FCT Goodput Short flows completed in ~100 ms Interfered by these 2 long flows Goodput of short flows collapse 12

Toy Example: Flow Interference in OQ Switch FCT Goodput Short flows completed in ~100 ms Goodput of short flows collapse Interfered by these 2 long flows Unfairly served 13

AGENDA n Background & Motivation n Flow Interference n CQRD Design n Evaluation n Conclusion 14

CQRD Design • Crosspoint-Queue 15

CQRD Design • Crosspoint-Queue • Eliminating interference between flows on different switch paths (Output-Contending but not Path-Contending, OC-PC) 16

CQRD Design • Crosspoint-Queue • Eliminating interference between flows on different switch paths (Output-Contending but not Path-Contending, OC-PC) Separate buffer & Fair scheduling 17

CQRD Design • Crosspoint-Queue • Eliminating interference between flows on different switch paths (Output-Contending but not Path-Contending, OC-PC) • Random-Drop • Alleviate the flow interference within the same switch path (Path-Contending, PC) 18

CQRD Design • Crosspoint-Queue • Eliminating interference between flows on different switch paths (Output-Contending but not Path-Contending, OC-PC) • Random-Drop • Alleviate the flow interference within the same switch path (Path-Contending, PC) Occupy more buffer, more likely to be dropped 19

CQRD Design • Crosspoint-Queue • Eliminating interference between flows on different switch paths (Output-Contending but not Path-Contending, OC-PC) • Random-Drop • Alleviate the flow interference within the same switch path (Path-Contending, PC) Occupy more buffer, more likely to be dropped 20

Toy Example: Flow Interference FCT Goodput 3 orders shorter FCT 3 orders higher goodput 21

Toy Example: Flow Interference FCT Goodput 3 orders shorter FCT 3 orders higher goodput Fairly served Almost no cost of goodput 22

Toy Example: Flow Interference FCT Goodput 3 orders shorter FCT 3 orders higher goodput Fairly served Almost no cost of goodput 23

AGENDA n Background & Motivation n Flow Interference n CQRD Design n Evaluation n Conclusion 24

Evaluation • 1. How much FCT of short delay-sensitive flows is reduced in CQRD? • 2. How much goodput of long bandwidth-greedy flows is sacrificed in CQRD? 25

Experiment 1 • Single aggregation/core switch (ns 2 simulations) • Simulation parameters: • Link capacity=10 Gbps, Link delay=4 us, Total buffer size=5 MB, TCP initial window size=4, TCP initial RTO=200 us. • Traffic: • 1200 TCP flows, Flow size & inter-arrival time from realistic distributions, Random source & destination port 26

Single aggregation/core switch FCT of all short flows ( < 100 KB) and goodput of all large flows (> 100 KB) interfered by the giant flows (> 1 MB, included by large flows) at moderate load (0. 1). 27

Single aggregation/core switch ~28% lower ~7% lower ~4% lower ~36% lower FCT of all short flows ( < 100 KB) and goodput of all large flows (> 100 KB) interfered by the giant flows (> 1 MB, included by large flows) at moderate load (0. 1). 28

Experiment 2 • Multi-stage DCN switching fabric (ns 2 simulations) • Simulation parameters: • Link delay=2 us, Agg switch buffer size=5 MB, To. R switch buffer size=4 MB, TCP initial window size=4, TCP initial RTO=200 us. • Traffic: • 2000 TCP flows, realistic distributions; ECMP load-balancing schemes 29

Single aggregation/core switch ~same ~30% lower ~2. 5% lower ~14% lower FCT of all short flows ( < 100 KB) and goodput of all large flows (> 100 KB) interfered by the giant flows (> 1 MB, included by large flows) at moderate load (0. 1). 30

AGENDA n Background & Motivation n Flow Interference n CQRD Design n Evaluation n Conclusion 31

Conclusion • Tackling the root cause of flow interference: • Need a more fine-grained queue management scheme • Simple solution: CQRD—switch queue management scheme • • • Transparent to end host No modification to protocol stack Based on underlying techniques available in commodity productions • Reduces the FCT of short flows by 20 -44% in a single switch and 8 -30% in a multi-stage data center switch network • At the cost of a minor goodput decrease for large flows 32

THANK YOU