Checksums and Packet Reordering Sridhar Machiraju March 21

Checksums and Packet Reordering Sridhar Machiraju March 21 st, 2001 Network Reading Group, UCB

Checksums Measurements and Analysis of End-to. End Internet Dynamics, V. Paxson, Ph. D. thesis, 1997, UCB. When the CRC and TCP Checksum Disagree, J. Stone and C. Partridge, Sigcomm 2000.

Paxson’s Thesis Since packet contents were not captured, bit errors were inferred. LBL(? ) had a large share of (bursty!) checksum errors. It used SLIP compression (CSLIP). It is better to have a strong datalink checksum. (More on this later) About 1 in 5000 TCP packets had a corrupt checksum. 16 -bit TCP checksum implies that 1 in 300 million packets have incorrect checksum.

Paxson (contd. ) Data packets had more errors than ACKs. Why? ACKs are smaller in length than data packets (assuming uniform bit-errors). Also, IP checksum would take care of integrity of the IP header (which accounts for a larger portion of the packet with ACKs). Likelihood of desynchronization increases with packet length and if link is point-to-point.

Stone and Partridge 1 in 7500 packets passed the receiver datalink CRC but failed the TCP checksum. Contradict Paxson’s observation – non-data packets had as many errors as data packets! Errors occurred in end-host software, router memory, network-interface hardware etc. Consecutive datagrams sent on links implementing Van Jacobson compression had wrong checksums due to framing errors and dropped packets.

Stone et al. (contd. ) ACK-of-FIN bug in Windows NT was also noticed. About 1 in O(1 billion) packets has an error undetected by checksum!

Conclusions Be wary of header compression. (E 2 E) Application-level checksums are very necessary; remember the NFS story – see the Feb, 2002 e 2 e mailing list archives? Hardware cannot be trusted. Note that application-level checksums are probably needed and IPSec would not be sufficient.

Packet Reordering is not Pathological Behavior, Bennet, C. Partridge, N. Shectman, IEEE/ACM TON, Dec 1999. On Making TCP More Robust to Packet Reordering, E. Blanton and M. Allman, ACM Computer Communication Review, Jan 2002.

In 1995, Paxson observed reordering in 1994/1995 due to different paths. Another explanation – a router gets a route update and forwards the existing queue of packets whenever possible! Bennet et. al. performed a study in 1998. They made extensive observations at the Mae-East peering point.

In 1998, At the Mae-East exchange, l l l Wide variety of equipment, hosts. Well-known topology and traffic statistics. Heavily loaded site. Sent bursts of ICMP messages to hosts topologically close to Mae-East. The result was that 90% of the hosts experienced reordering! Reason: Use of Parallel Links between ISPs at the heavily loaded Mae-East exchange.

A tale of 2 ISPs

Hunt Groups in Switches The manual for the DEC Giga. Switch says that tcprexmtthresh should be 100! Reordering occurs because of a parallel set of links (called hunt groups) between 2 switches and Head-Of-Line Blocking. This could be eliminated in output-buffered router and input-buffered routers too but the problem is cost and complexity. Question: Can such routers be built today?

IP/TCP layer Solutions? Hash all packets of a flow and send them on only one physical link. Same solution applies for multiple ASICs too. Question: How bad will the utilization be if hashing is not symmetric. TCP SACK option could be used to detect reordering and increase tcprxmtthresh (more on this later).

Router Architectures Why not number packets and reorder them? The algorithms are non-work conserving. A Reliable and Scalable Striping Protocol, H. Adiseshu, G. Parulkar, and G. Varghese, ACM Sigcomm ’ 96. Unsolved problem – bit-by-bit striping. The appendix proves that under a set of assumptions, absence of reordering and work -conserving are not both achievable.

Impact of reordering on TCP Blanton et. al. – completely simulation-driven study. Spurious (fast) retransmits. Spurious retransmission also causes suspension of RTT estimation. With fast retransmit, reordering is interpreted as packet loss and sending rate is decreased. Make TCP bursty because an ACK will acknowledge receipt of >1 segments. Reordering inflates RTT estimates. Is this a problem, considering the 100 s of ms granularity of TCP clocks.

Detection of Unnecessary Retransmits Use of the D(uplicate)SACK option. The receiver sends information about previous reordering too. If the RTO expires and the ACK for the lost packet arrives soon, it is an ACK for the original packet. Use of Eifel retransmission timer.

Possible Solution The impact of reordering reduces as the amount of reordering increases. The authors consider the use of DSACK option – a retransmission is noted along with the SACK option. One solution: The cwnd is not decreased if no “actual loss” occurred. But this does not prevent spurious retransmissions. The rxmtthresh needs to be modified based on the amount of reordering. Tradeoffs: Experiencing timeout versus converging to optimal rxmtthresh quickly.