TCP Congestion Control Beyond BandwidthDelay Product for Mobile

TCP Congestion Control Beyond Bandwidth-Delay Product for Mobile Cellular Networks Wai Kay Leong, Zixiao Wang, Ben Leong The 13 th International Conference on emerging Networking EXperiments and Technologies

Mobile Internet usage exceeds Desktop Co. NEXT ’ 17 Seoul, Incheon 2

What's different about cellular? Negligible random packet losses ◦ Hybrid ARQ scheme ◦ As compared to 802. 11 Wi-Fi Large buffers ◦ In the Megabytes Asymmetric uplink/downlink ◦ ACK delay Fair scheduling at “base station” ◦ No contention with other users Co. NEXT ’ 17 Seoul, Incheon 3

Why Traditional TCP does not work IN MOBILE/CELLULAR NETWORKS 4

1. Large/Deep buffers Buffer overflow Lack of congestion signal Reno CUBIC Cwnd Packets in flight (buffer) Deep buffer causes high latency (hundreds of ms) Actual RTT Time Co. NEXT ’ 17 Seoul, Incheon 5

2. Uplink Congestion More predominant in slower 3 G/HSPA networks ACK gets delayed in return uplink ◦ Stuck in deep buffer/ high volume of users ◦ Server is prevented from sending new packet even though downlink is clear Idle link ACK Co. NEXT ’ 17 Seoul, Incheon Data …… 6

Rethink congestion control for mobile networks Traditional TCP congestion control ◦ Lack of congestion signal (ECN not popular) ◦ Long delay/high latency (CUBIC) ◦ ACK clocked Rise in new mobile TCP algorithms ◦ Sprout ◦ Verus ◦ PCC ◦ BBR Co. NEXT ’ 17 Seoul, Incheon 7

Key Idea Purpose of congestion control is to ◦ avoid congestion ◦ finding the correct rate to send packets ◦ ideally keep 1×BDP packets in transit Why not just send at the correct rate? ◦ Vary conditions of mobile networks Our Insight: ◦ Try to forecast the condition (Sprout, PROTEUS, Verus, etc. ) Timely estimation of the bandwidth ◦ Try to build a model (PCC, Remy) + quick reaction to new network condition is sufficient Co. NEXT ’ 17 Seoul, Incheon 8

Our Approach Abandon ACK clocking Pure rate-based sending of packets 1. 2. 3. 4. Estimate current bandwidth/receive rate Send packets at estimated rate Observe buffer delay Update send rate Takes advantage of large buffer Congestion with others mitigated by fair scheduling in base station Co. NEXT ’ 17 Seoul, Incheon 9

We need a means to 1. Estimate the bandwidth/receive rate 2. Detect congestion by measuring one-way delay Make use of TCP timestamp option ◦ Enabled by default on most servers and phones Co. NEXT ’ 17 Seoul, Incheon 10

Estimating Receive Rate Receiver will send ACK when packet is received Sender Receiver ACK will be timestamped Compute rate by ◦ comparing timestamps: tr 1 – tr 0 = Δt ◦ and bytes ACK: ΔACK/Δt = ρ TSval = tr 0 Δt TSval = tr 1 Co. NEXT ’ 17 Seoul, Incheon 11

Estimating Buffer/Queuing Delay Sender Receiver tsnd Relative delay RD = tack – tsnd tack Queuing delay tbuff = RD – RDmin Actual delay (RDmin) TSval = tack Only relative increase/decrease of tbuff matters Co. NEXT ’ 17 Seoul, Incheon 12

Putting it together Estimate Receive Rate/Bandwidth Pure Rate-based Mechanism Detect Congestion from Queuing delay Co. NEXT ’ 17 Seoul, Incheon 13

Self-oscillating feedback loop 1 Increase in queuing delay Slow Start Send burst of 10 packets tbuff > T Link Congested 2 Estimate bandwidth (ρ) Buffer Fill State Send faster than bandwidth (σf>ρ) 3 ρ and tbuff constantly updated Buffer Drain State Send slower than bandwidth (σd<ρ) Decrease in queueing delay tbuff < T No Congestion Co. NEXT ’ 17 Seoul, Incheon 14

Packet Trace, aka Sawtooth Packets in buffer/ Queuing Delay Buffer Fill State delay Buffer Drain State Takes at least 1×RTT to get feedback on queuing delay Latency oscillates between the peaks and throughs T delay Time Co. NEXT ’ 17 Seoul, Incheon 15

Prop. Rate Sending rate is a proportion of bandwidth/receive rate ◦ σf = kf ρ ◦ σd = kd ρ Three parameters controls the sawtooth ◦ kf – proportion to fill buffer ◦ kd – proportion to drain buffer ◦ T – threshold for switching state Co. NEXT ’ 17 Seoul, Incheon 16

Parameters Packets in buffer/ Queuing Delay By adjusting the parameters, kf , kd and T, we can change the shape of the sawtooth. T Average latency Time Co. NEXT ’ 17 Seoul, Incheon 17

Parameters Packets in buffer/ Queuing Delay By adjusting the parameters, kf , kd and T, we can change the shape of the sawtooth. T Average latency Time Co. NEXT ’ 17 Seoul, Incheon 18

Parameters Packets in buffer/ Queuing Delay By adjusting the parameters, kf , kd and T, we can change the shape of the sawtooth. T Average latency Time Co. NEXT ’ 17 Seoul, Incheon 19

Parameters Packets in buffer/ Queuing Delay By adjusting the parameters, kf , kd and T, we can change the shape of the sawtooth. T Average latency Time Co. NEXT ’ 17 Seoul, Incheon 20

Parameters By adjusting the parameters, kf , kd and T, we can change the shape of the sawtooth. Packets in buffer/ Queuing Delay Throughput is maximum because buffer is always filled T Average latency Time Co. NEXT ’ 17 Seoul, Incheon 21

Parameters Throughput is maximum because buffer is always filled Packets in buffer/ Queuing Delay Average latency can be adjusted T Average latency Time Co. NEXT ’ 17 Seoul, Incheon 22

Parameters Throughput is maximum because buffer is always filled Packets in buffer/ Queuing Delay Average latency can be adjusted T Average latency Time Co. NEXT ’ 17 Seoul, Incheon 23

Two optimization modes Optimizing for Throughput Optimizing for Latency Queuing Delay ◦ Buffer to be kept filled ◦ Implies maximum throughput ◦ Latency suffers due to queuing delay T ◦ Buffer needs to be emptied ◦ Reduced utilization reduced throughput ◦ More responsive latencies Average latency T Time Co. NEXT ’ 17 Seoul, Incheon Average latency Time 24

Please read our paper Parameter tuning ◦ Specify target latency to set the parameter Updating Threshold ◦ Due to network volatility Some math Co. NEXT ’ 17 Seoul, Incheon 25

Evaluation 26

Performance Evaluation 1. Compare with other TCP protocols ◦ ◦ Traditional TCP: CUBIC, Vegas, Westwood, LEDBAT State-of-art Mobile: Sprout, PCC, Verus, BBR 2. Delayed ACK/Saturated Uplink 3. Throughput vs Delay tradeoff 4. Fairness/Contention Two Scenarios 1. Emulated networks 2. Real cellular networks Three flavours of Prop. Rate Low, Medium, High + Frontier Enumerate parameter space 5. Computation overhead Co. NEXT ’ 17 Seoul, Incheon 27

Implementation Prop. Rate was implemented in the Linux kernel (version 3. 13) ◦ Replaced TCP congestion control code ◦ With our rate-based framework (~200 lines) ◦ Prop. Rate as a kernel module (~1, 500 lines) Run actual TCP in our evaluations ◦ Not simulate using UDP or NS 2/3 ◦ Measure actual computation overhead ◦ Proof-of-“deployability” Co. NEXT ’ 17 Seoul, Incheon 28

Trace-based Emulation Keep network constant – for fair comparison Cellsim Emulator (from MIT) Mobile Actual Network Traces wired Cellsim 1010 0101 1100 1011 wired Server uplink downlink ◦ Three local cellular ISPs ◦ Two scenarios: stationary (in our lab) and mobile (on a bus) ◦ MIT traces [Winstein et al. ] Co. NEXT ’ 17 Seoul, Incheon 29

Results – Local ISP, Stationary Good throughput/Bad latency r tte be Good latency/ Bad throughput Co. NEXT ’ 17 Seoul, Incheon 30

Results – Local ISP, Mobile Good throughput/Bad latency Good latency/ Bad throughput Co. NEXT ’ 17 Seoul, Incheon 31

Results – Real LTE Network Co. NEXT ’ 17 Seoul, Incheon 32

Results Prop. Rate more optimal than other TCP variants ◦ Achieves higher throughput ◦ or, lower latency Co. NEXT ’ 17 Seoul, Incheon 33

Congested/Saturated Uplink o pl u t n rre u c on C) I B CU ad ( c Mobile Server LTE USB Tether Smartphone congested uplink downlink Co. NEXT ’ 17 Seoul, Incheon 34

Congested Uplink – Real LTE Network Co. NEXT ’ 17 Seoul, Incheon 35

Results Prop. Rate more optimal than other TCP variants ◦ Achieves higher throughput ◦ or, lower latency Decoupling ACK clocking improves resilience ◦ Towards asymmetric links Co. NEXT ’ 17 Seoul, Incheon 36

Performance Frontiers Co. NEXT ’ 17 Seoul, Incheon 37

Results Prop. Rate more optimal than other TCP variants ◦ Achieves higher throughput ◦ or, lower latency Decoupling ACK clocking improves resilience ◦ Towards asymmetric links Frontier hull shows Prop. Rate is always most optimal Co. NEXT ’ 17 Seoul, Incheon 38

Fairness – Self Contention Co. NEXT ’ 17 Seoul, Incheon 39

Fairness – Contention from others Co. NEXT ’ 17 Seoul, Incheon 40

Results Prop. Rate more optimal than other TCP variants ◦ Achieves higher throughput ◦ or, lower latency Decoupling ACK clocking improves resilience ◦ Towards asymmetric links Frontier hull shows Prop. Rate is always most optimal Prop. Rate can secure some foothold with CUBIC flows Co. NEXT ’ 17 Seoul, Incheon 41

CPU Utilization 25% Intel i 7 -2600 @ 3. 4 GHz Intel Xeon E 5640 @ 2. 67 GHz 20% Intel Q 9550 @ 2. 83 GHz 15% 10% No CPU overhead. Identical to existing traditional TCP algorithms 5% 0% Sprout LEDBAT Co. NEXT ’ 17 Seoul, Incheon PCC Verus BBR CUBIC RRE Prop. Rate 42

Whither the future? Resurgence in interest in TCP ◦ Different emergent networks: Datacenter, Wi-Fi, Cellular, etc. Traditional TCP: CUBIC/Compound ◦ Floods buffer Increased latency Delay-based algorithms: Vegas, Westwood, etc. ◦ Good latency ◦ Starved by CUBIC Co. NEXT ’ 17 Seoul, Incheon 43

Is TCP ready for rate-based algorithms? Pure rate-based algorithms: Prop. Rate & BBR ◦ Handles bufferbloat ◦ Compete well against CUBIC Can co-exist with CUBIC ◦ Facilitate transition to better rate-based TCP algorithms Prop. Rate builds on a framework ◦ More optimal algorithms in the future? ◦ Better integration with TCP stack in the future? Co. NEXT ’ 17 Seoul, Incheon 44

Thank You QUESTIONS? 45