FAST TCP Cheng Jin David Wei Steven Low

FAST TCP Cheng Jin David Wei Steven Low netlab. CALTECH. edu GNEW, CERN, March 2004

Acknowledgments o Caltech n o UCLA n o Aiken, Doraiswami, Yip Level(3) n o Cottrell, Mount Cisco n o Almes, Shalunov SLAC n o Martin, Martin-Flatin Internet 2 n o Paganini, Z. Wang CERN/Data. TAG n o Bunn, Choe, Doyle, Jin, Newman, Ravot, Singh, J. Wang, Wei Fernes LANL n Wu

FAST project NSF RI (2003) NSF STI (2002) HEP networks Tera. Grid Abilene Dummy. Net Abilene HEP networks Planet. L WAN in Lab Linux TCP kernel NSF ITR (2001) Performance Stability Other platforms Fairness TCP/IP IETF GGF Deployment Ultra. Light testbed Experiment Monitoring Debugging Implement Noise Random -ness Theory

Outline o. Experiments n n o Status n n o Results Future plan Open issues Code release mid 04 Unified framework n n Reno, FAST, HSTCP, XCP, … Implementation issues

Aggregate throughput 88% FAST o Standard MTU o Utilization averaged over > 1 hr 90% Average utilization 92% 95% 1 hr 1 flow 1 hr 2 flows 6 hr 7 flows 1. 1 hr 9 flows 6 hr 10 flows Data. TAG: CERN – Star. Light – Level 3/SLAC (Jin, Wei, Ravot, etc SC 2002)

FAST Linux Dynamic sharing: 3 flows Dynamic sharing on Dummynet o o o capacity = 800 Mbps delay=120 ms 3 flows iperf throughput Linux 2. 4. x (HSTCP: UCL)

FAST Linux Dynamic sharing: 3 flows Steady throughput HSTCP

queue FAST loss Linux throughput 30 min Dynamic sharing on Dummynet capacity = 800 Mbps HSTCP o delay=120 ms o 14 flows o iperf throughput o Linux 2. 4. x (HSTCP: UCL) o STCP

queue Room for mice ! FAST loss Linux throughput HSTCP 30 min STCP

Aggregate throughput ideal performance Dummynet: cap = 800 Mbps; delay = 50 -200 ms; #flows = 1 -14; 29 expts

Aggregate throughput large window 8000 small window 800 pkts Dummynet: cap = 800 Mbps; delay = 50 -200 ms; #flows = 1 -14; 29 expts

H ST CP ~ R en o Fairness Jain’s index Dummynet: cap = 800 Mbps; delay = 50 -200 ms; #flows = 1 -14; 29 expts

Stability stable in diverse scenarios Dummynet: cap = 800 Mbps; delay = 50 -200 ms; #flows = 1 -14; 29 expts

Outline o Experiments n n o Status n n o Results Future plan Open issues Code release Unified framework n n Reno, FAST, HSTCP, XCP, … Implementation issues

Benchmarking TCP o Not just static throughput n o Tests to zoom in on specific properties n n n o Throughput, delay, loss, fairness, stability, … Critical for basic design Test scenarios may not be realistic Tests with realistic scenarios n n n o Dynamic sharing, what protocol does to network, … Same performance metrics Critical for refinement for deployment Just started Input solicited n What’s realistic for your applications?

Open issues: well understood o base. RTT estimation n n o Small network buffer n n o at least like TCP adapt a on slow timescale, but how? TCP-friendliness n n o route changes, dynamic sharing does not upset stability friendly at least at small window tunable, but how to tune? Reverse path congestion n should react? rare for large transfer?

Status: code release o Source release mid 2004 n n n o Pre-release trials n o For any non-profit purposes Re-implementation of FAST TCP completed Extensive testing to complete by April 04 CFP for high-performance sites! Incorporate into Web 100 n with Matt Mathis

Status: IPR Caltech will license royalty-free if FAST TCP becomes IETF standard n n IPR covers more broadly than TCP Leave all options open

Outline o Experiments n n o Status n n o Results Future plan Open issues Code release mid 04 Unified framework n n Reno, FAST, HSTCP, XCP, … Implementation issues

Packet & flow level Reno TCP o o Packet level W W + 1/W Loss: W W – 0. 5 W ACK: Flow level n Equilibrium n Dynamics pkts (Mathis formula)

Reno TCP o Packet level n o Flow level n o Understood afterwards Flow level dynamics determines n o Designed and implemented first Equilibrium: performance, fairness Stability Design flow level equilibrium & stability Implement flow level goals at packet level

Reno TCP o Packet level n o Flow level n o Designed and implemented first Understood afterwards Flow level dynamics determines n n Equilibrium: performance, fairness Stability Packet level design of FAST, HSTCP, H-TCP, … guided by flow level properties

Packet level o o Reno ACK: W W + 1/W AIMD(1, 0. 5) Loss: W W – 0. 5 W HSTCP ACK: W W + a(w)/W AIMD(a(w), b(w)) Loss: W W – b(w)W STCP ACK: W W + 0. 01 MIMD(a, b) Loss: W W – 0. 125 W FAST

Flow level: o Reno, HSTCP, FAST Similar flow level equilibrium MSS/sec a = 1. 225 (Reno), 0. 120 (HSTCP), 0. 075 (STCP)

Flow level: o Common flow level dynamics window adjustment o = control gain flow level goal Different gain k and utility Ui n o Reno, HSTCP, FAST They determine equilibrium and stability Different congestion measure pi n n Loss probability (Reno, HSTCP, STCP) Queueing delay (Vegas, FAST)

FAST TCP o Reno, HSTCP, and FAST have common flow level dynamics window adjustment o flow level goal Need to estimate “price” pi(t) = queueing delay n o control gain Equation-based n o = Easier to estimate at large window k(t) and U’i(t) explicitly designed for n n n Performance Fairness Stability

Window control algorithm o Full utilization n o Globally stable n o regardless of bandwidth-delay product exponential convergence Intra-protocol fairness n n weighted proportional fairness parameter a

FAST tunes to knee Goal: • Less delay • Less jitter FAST TCP stabilized oscillation

Window adjustment FAST TCP

netlab. caltech. edu/FAST TCP: motivation, architecture, algorithms, performance o IEEE Infocom March 2004 o FAST TCP: from theory to experiments Submitted for publication April 2003

Panel 1: Lessons in Grid Networking

Metrics o Performance n o o Availability, reliability Simplicity n n o Throughput, loss, delay, jitter, stability, responsiveness Application Management Evolvability, robustness

Constraints o Scientific community n n n o Small & fixed set of major sites Few & large transfers Relatively simple traffic characteristics and quality requirements General public n n n Large, dynamic sets of users Diverse set of traffic characteristics & quality requirements Evolving/unpredictable applications

Mechanisms o o o o Fiber infrastructure Lightpath configuration Resource provisioning Traffic engineering, adm control Congestion/flow control Months - years Mintes - days Service: sec - hrs Flow: sec - mins RTT: ms - sec Timescale: desired, instead of feasible Balance: cost/benefit, simplicity, evolvability