Sizing Router Buffers How much packet buffers does

Sizing Router Buffers How much packet buffers does a router need? Smaller buffers have a major effect on router design What determines router buffer size? An Example: • 10 Gb/s linecard with 200, 000 x 56 kb/s flows – Rule-of-thumb: Buffer = 2. 5 Gbits • TCP can’t fully utilize a link without any buffers – Only with maximum congestion window the link is full – Window scales down just after it reaches maximum • Requires external, slow DRAM The current “Rule of Thumb” • A router needs a buffer size: Our new rule for sizing buffers • A router needs a buffer size: • 2 T is the two-way propagation delay (or just 250 ms) • C is capacity of bottleneck link • Context – Mandated in backbone and edge routers. – Appears in RFPs and IETF architectural guidelines – Usually referenced to Villamizar and Song: “High Performance TCP in ANSNET”, CCR, 1994. – Already known by inventors of TCP [Van Jacobson, 1988] – Becomes: Buffer = 6 Mbits W 8 • Can use on-chip, fast SRAM • 40 Gb/s linecard with 40, 000 x 1 Mb/s flows – Rule-of-thumb: Buffer = 10 Gbits – Becomes: Buffer = 50 Mbits 4 t It is estimated that reduced buffers on a line cards could • 2 T is the two-way propagation delay • C is capacity of bottleneck link • n is the number of long-lived flows through the router – Reduce power consumption by 10%-40% – Save 10%-40% of board space – Reduce cost by 10%-30% 0 Do these results hold for a physical router? – Measurements on a Cisco GSR with an OC 3 line card and 400 flows – Simulations with ns 2 over a broad range of parameters Router C Destination 2 T – Too much buffer causes the buffer to never empty and increases latency – Too little buffering causes the buffer to be empty over extended periods of time and the link to be underutilized Right amount of buffering t • For many flows, TCP windows are multiplexed – Buffer size is reduced B Below is a side by side comparison of Model, Simulation and Experiments on a physical router TCP Flows What is the effect of too much or too little buffer? Right amount of buffering – TCP Window sizes changes over time (the TCP saw tooth) – Number of outstanding packets changes over time – Buffer has to buffer these packets B Yes. We verified our model by simulation and on a physical router Source • The main role of a router buffer is to compensate for the difference in outstanding packets • Why? What has changed? – High-end routers serve many unsynchronized flows – Statistical multiplexing reduces amount of buffering needed Link not fully utilized Congestion Window 100 400 Router Buffer Pkts 0 Link Utilization RAM Model Sim Exp 0. 5 x 1 x 2 x 3 x 64 129 258 387 1 Mb 2 Mb 4 Mb 8 Mb 96. 9% 99. 9% 100% 94. 7% 99. 3% 99. 9% 99. 8% 94. 9% 98. 1% 99. 8% 99. 7% 0. 5 x 1 x 2 x 3 x 32 64 128 192 512 kb 1 Mb 2 Mb 4 Mb 99. 7% 100% 99. 2% 99. 8% 100% 99. 5% 100% 99. 9% Probability Distribution • Sum of congestion windows is Gaussian – Reason: The central limit theorem – Width of Gaussian is dependent on square root of number of flows – Buffer has to be wide enough to “fit” Gaussian Smaller buffers help short flows Buffer Size In mixes of long and short TCP flows, short flows benefit from smaller buffers – RTT is decreased significantly (almost haved) by smaller buffers – Loss increases, but effect of RTT decrease outweighs loss Example: Completion time of short (14 packet) flows in a mix of long and short flows Overbuffered Link Underbuffered Link – We increase the number of long flows and measure average completion time – With smaller buffers completion time is substantially lower • Buffer requirement decreases with 1/sqrt(n) – Verified experimentally Simulation