Transport Layer for Mobile Ad Hoc Networks Based

Transport Layer for Mobile Ad Hoc Networks Based on slides by: Eric Law, UCR March 2005 1

Introduction The Quest for the Transport Layer n Assess the state of the art of transport protocols ¨ Target n n n environment: mobile ad hoc nets (MANET) Which is the best TCP variant? Do we need a new transport protocol? The question is very timely… 2

Introduction Network Architecture at a Crossroads n n n The community recognizes the need for change Wireline-centric network design is “obsolete” New network environments have emerged ¨ Ad n hoc, sensors, consumer-owned, delay-tolerant New networking technologies have emerged ¨ UWB, cooperative approaches, MIMO, directed antennas 3

Introduction New Class of Networks n Tens of nodes, resource constrained, wireless links, line powered (computers) n Tens of nodes, resource constrained, wireless links, line powered (embedded devices) n Tens of nodes, resource constrained, wireless links, charged every day (PDAs) n Hundreds of nodes, resource constrained, unreliable wireless links (Sensors) n Thousands of nodes, highly resource constrained, highly unreliable wireless links, low duty cycle (smartdust) Tens - thousands of nodes, Nano-sensors 4 n

Introduction A New Era Has Begun New Machines New Environments Applications New Networks 5

Introduction The Role of Networking is Central Embedded Sensor Applications Wireless Networking Embedded Systems Sensors 6

Introduction Revisiting the Architecture n The vision: ¨ Wireless as an integral part of the network ¨ Multiple wireless hops: not just the last mile ¨ Pockets of wireless ad hoc connectivity n A new protocol stack is required ¨ Is TCP/IP capable of delivering? 7

Introduction Revisiting The Hourglass Email WWW Voice. . . SMTP HTTP RTP. . . TCP UDP IP Ethernet PPP… Multi. Access async. . . copper fiber radio. . . User Application Protocol Transport Protocol Internet Protocol Media Access Protocol Media Sharing Principles Physical System 8

Problem: Evaluate TCP n Why does TCP perform poorly in MANETs? ¨ Developed for wire-line networks. ¨ Assume all losses are due to congestion. n Many TCP variants have been proposed. ¨ How n good are they? Are they sufficient? Are there any other alternatives? ¨ Are non-TCP protocols the solution? 9

Our goal Identify the problems of TCP in MANETs. n Evaluate various major TCP variants. n ¨ 12 n TCP variants, 7 improvement techniques Observations: ¨ Most TCP variants are NOT sufficient. ¨ A new transport layer protocol is needed. 10

Overview of Results n The best TCP variants: TCP-Westwood and TCP-Jersey seem the best. ¨ Both protocols estimate bandwidth more accurately. ¨ n TCP mechanisms: ¨ n Feedback from intermediate nodes leads to big gains. The best non-TCP approaches: ¨ Ad-hoc Transport Protocol (ATP) seems to address most issues n ¨ Split-TCP: promising new way of looking at transport layer n ¨ Non-window based: estimates achievable rate periodically Dynamically buffer packets mid-path Key: Separation of congestion control from reliability. 11

Roadmap n n Overview of TCP The problems of TCP over MANETs Overview of best transport protocols In depth ¨ Specific problems of TCP over MANETs ¨ Details of major TCP variants ¨ Discussion - other efforts n Conclusion 12

Overview of TCP concepts n n Conventional TCP: Tahoe, Reno, New-Reno Sending rate is controlled by Congestion window (cwnd): limits the # of packets in flight ¨ Slow-start threshold (ssthresh): when CA start ¨ n Loss detection 3 duplicate ACKs (faster, more efficient) ¨ Retransmission timer expires (slower, less efficient) ¨ n Overview of congestion control mechanisms Slow-start phase: cwnd start from 1 and increase exponentially ¨ Congestion avoidance (CA): increase linearly ¨ Fast retransmit and fast recovery: Trigger by 3 duplicate ACKs ¨ Slow-start Congestion avoidance 13

Overview What is different in MANETs? 1. Mobility ¨ 2. High bit error rate ¨ 3. Difficult to estimate time-out, RTT, bandwidth Contention: packets compete for airtime ¨ 5. Packets can be lost due to “noise” Unpredictability/Variability ¨ 4. Route stability and availability Intra-flow and inter-flow contentions Long connections have poor performance q More than 4 hops thruput drops dramatically 14

Overview of the Best Protocols n TCP-Westwood ¨ n [Casetti et. al. ] Estimate bandwidth to alleviate the effect of wireless errors. TCP-Jersey [Xu et. al. ] Estimate bandwidth to alleviate the effect of wireless errors. ¨ Congestion warning assists the determination of packet loss due to wireless error from congestion. ¨ n ATP ¨ n [Sundaresan et. al. ] Rate based transmission, periodic rate feedback, no timeout concept, reliability provided by SACK. Split-TCP [Kopparty et. al. ] Separating congestion control from reliability. ¨ Dropped packets are recovered from the most recent proxy instead of the source. ¨ 15

Overview Why does TCP fail in MANETs? Specific problems are identified: 1. 2. 3. 4. TCP misinterprets route failures as congestion TCP misinterprets wireless errors as congestion Intra-flow and inter-flow contention reduce throughput and fairness Delay spike causes TCP to invoke unnecessary retransmissions n 5. RTO too small unnecessary retransmissions. Inefficiency due to the loss of retransmitted packet n When retransmitted packet is lost timer expires performance drops 16

Roadmap n n Overview of TCP The problems of TCP over MANETs Overview of best transport protocols In depth ¨ Specific problems of TCP over MANETs ¨ Details of major TCP variants ¨ Discussion - other efforts n Conclusion 17

Specific problems of TCP over MANETs n TCP misinterprets route failures as congestion ¨ Effects: Reduce sending rate ¨ Buffered packets (Data and ACKs) at intermediate nodes are dropped. ¨ Sender encounters timeout. n Under prolonged disconnection, a series of timeouts may be encountered. 18

Specific problems of TCP over MANETs n TCP misinterprets wireless errors as congestion ¨ Effects: Incorrect execution of congestion control Performance drops. ¨ Wireless channel is error-prone compared to wireline n Fading, interference, noise 19

Specific problems of TCP over MANETs n Intra-flow and inter-flow contention ¨ Effects: Increased delay, unpredictability, and unfairness. ¨ Inter-flow contention: contention of nearby flows. ¨ Intra-flow contention: between packets of the same flow (e. g. forward data and reverse ACKs). ¨ Wireline: only packet on same link “compete” Two nearby flows Data stream ACKs stream 20

Specific problems of TCP over MANETs n Delay spike causes TCP to invoke unnecessary retransmissions ¨ Effects: Performance drops and many unnecessary retransmissions. [Ludwig & Katz] ¨ Variability: Spikes are not uncommon here ¨ Spikes throw off parameter estimation and tuning n RTO, window size, slow-start threshold 21

Specific problems of TCP over MANETs n Inefficiency due to the loss of retransmitted packet ¨ Effects: Performance drops significantly under high loss environment (e. g. MANETs). ¨ Losing a retransmitted packet hurts n TCP can recover from one loss (fast retransmission) ¨ Wired networks: packet loss rate is low. ¨ Here, high packet loss makes the problem significant 22

Classification of Transport protocols n TCP variants try to improve the performance by the following ways: ¨ Estimating the available bandwidth ¨ Determining route failure and wireless error ¨ Reducing contention ¨ Detecting spurious retransmission ¨ Exploiting buffering capability n New approaches: Non TCP variants ¨ Use rate based instead of window based approach ¨ Enable dynamic buffering (split TCP) 23

Details of some major TCP variants Estimating the available bandwidth n TCP-Vegas ¨ ¨ ¨ n Use difference = expected rate - actual rate to regulate the sending rate. Pro: Avoid congestion and typical loss at the end of the slow-start. Con: Route changes invalidate rate calculation. Unfairness problems. TCP-Westwood [Casetti et. al. ] ¨ ¨ n [Brakmo et. al. ] Use the rate of return of ACKs to estimate bandwidth. Sender re-computes the cwnd and ssthresh upon any loss. Pro: Robust to wireless errors. Con: Route changes invalidate bandwidth estimation. Depends on the behavior of returned ACKs. TCP-Jersey ¨ ¨ ¨ [Xu et. al. ] Use the rate of return of ACKs to estimate bandwidth. Intermediate nodes warn sender of congestion Pro: Robust to wireless error Pro: Avoids congestion proactively Con: Require assistance from intermediate nodes. Depends on the behavior of returned ACKs. 24

Details of some major TCP variants Determining Route Failure and Wireless Error n Explicit Link Failure Notification (ELFN) [Holland & Vaidya] Upstream node of the failed link notifies the sender of the failure. ¨ Disable timers, stop all data transmission and send probing to detect restored route. ¨ Pro: Avoid execution of congestion control upon route failures. ¨ Con: Old TCP states (cwnd and timer) are used after route restoration. ¨ n Ad-hoc TCP (ATCP) [Liu & Singh] Hide errors from TCP sender (timeout and the 3 rd DUPACKs). ¨ Employ ECN and route failure notification to assist TCP’s decision. ¨ Pro: Does not modify TCP itself. ¨ Con: Require assistance from intermediate nodes. ¨ n ADTCP [Fu et. al. ] ¨ Use multiple metrics to determine the network behavior: n Congestion, Channel error, Route change, Disconnection states. Pro: Does not rely on intermediate node feedback. ¨ Con: Thresholds used to determine network states must be carefully defined. ¨ 25

Details of some major TCP variants Reducing Contention n 2 Adaptive Congestion Window Limit (CWL) ¨ ¨ ¨ n 1 3 4 5 6 7 [Chen et. al. ] Upper bound of bandwidth-delay product of a chain is k. N. 1/8 < k < 1/4, N is the “round-trip hop-count x packet size” From experiment, k approximately 1/5 of the round-trip hop-count. Pro: Adapt the maximum cwnd to avoid excessive contention. Con: Can only apply in chain of nodes. Does not account for the nearby flows. Link RED (LRED) and Adaptive Pacing (AP) [Fu et. al. ] Intermediate nodes mark packets when: n the average # of retires of current transmission > threshold. ¨ Sender reduces sending rate. ¨ Nodes back-off an additional time if they start to markpackets ¨ Pro: Avoid congestion and reduce contention. ¨ Con: Require modification at link-layer on each node. ¨ n TCP Adaptive Delayed Acknowledgement (TCP-ADA) [Singh & Kankipati] ¨ Send one ACK for a window of data. Pro: Reduce the number of ACKs and thus reduce contention. ¨ Con: Increase burstiness of the forward transmissions. ¨ 26

Details of some major TCP variants Detecting Spurious Retransmission n TCP-Eifel [Ludwig & Katz] ¨ Use timestamp option of TCP to solve retransmission ambiguity. ¨ Sender can determine whether the received ACK is from original transmission or retransmission. n After retransmission, if next ACK is from original transmission spurious! ¨ Restore the cwnd and ssthresh if spurious retransmission is detected. ¨ Pro: Robust to sudden delay spike. ¨ Con: Require the use of timestamp option or modification of TCP header. 27

Details of some major TCP variants Exploiting Buffering Capability n Buffering capability and Sequence information (TCP-Bu. S)[Kim et. al. ] ¨ ¨ ¨ Use explicit route failure notification to detect route failure. When route failure occurs, intermediate nodes buffer the pending packets and TCP sender doubles the retransmission timeout (RTO) value. Avoid timeouts and unnecessary retransmissions. Pro: Reduce the number of timeout events reduce the number of retransmissions. Con: Require assistance from intermediate nodes. Special routing protocol is used. 28

Non-TCP based approaches 29

A Rate-based Transport Layer Protocol n Ad-hoc Transport Protocol (ATP) [Sundaresan et. al. ] ¨ Feedback from intermediate nodes n ¨ Rate based transmission n n ¨ path failure, queueing delay, periodic feedback on rate Entirely rate-controlled. (no window concept) Evenly distribute transmissions over time. (reduce burstniess) Decoupling of congestion control and reliability Does not require the arrival of ACKs to clock out segment. n Does not employ cumulative ACKs but solely relies on periodic SACK (with 20 SACK blocks) to identify losses. ¨ Pro: 1) Estimate rate accurately. 2) Reduce traffic on the reverse path. 3) Recover more than one lost segment at a time. ¨ Con: 1) Incompatibility problem. 2) Require the assistance from the intermediate nodes. 3) Fastest possible time to detect and recover packet lost is 1 second. n 30

Split-TCP: a new approach n Split-TCP: work done at UCR ¨ ¨ ¨ [Kopparty et. al. ] Setup proxies along the connection many short TCP connections. Congestion control and reliability are separated. Proxies buffer packets from the previous proxy or the source. Any dropped packets are recovered from the most recent proxy but not from the source. Pro: Enhance parallelism. Reduce bandwidth consumption on retransmission. Con: Optimal frequency of proxy placement is not clear. 31

Split TCP in more detail 32

What is our idea: Split-TCP • Split long TCP connection into shorter segments: • Small segments can be more adaptive to conditions • Proxies “glue” segments together. • Proxies buffer packets and “becomes responsible” for the delivery of the packet. • INTUITION: separate reliability from congestion control • the former is end-to-end but the latter is not! 33

TCP Proxies : Functionality • Proxies send an Local ACK (LACK) to the previous proxy or the source upon the receipt of a TCP packet. • Thus, the flow of the packet now occurs in stages. • End to end ack to ensure end to end reliability. 34

Improving Throughput • When link failures occur in one segment, TCP data may still be transmitted on other segments. • Thus the throughput improves. S A B C P X Y Z D Link Failure Data transfer continues in spite of failure 35

Improving Fairness • Proxies alleviate the unfair advantage that shorter connections have over longer connections. • Now longer connections are split into shorter segments. • The throughput of longer connections however cannot equal that of shorter connections due to interactions between segments. • Packets cannot be sent and received at the proxy at the same time – adjacent TCP segments have to transport data in stages. 36

Alleviating The 802. 11 Capture Effect S 1 S 2 D 1 P 2 P 3 D 2 • Intuition: Localize capture effect • Instead of the capturing the whole path TCP connection, only the region spanned by a segment is captured • Thus, it is possible to have two TCP sessions in the vicinity of each other: S 1 -P 1 segment does not bother P 3 - D 2 segment 37

1. Throughput Improvement • Mobile Ad Hoc Network with 50 nodes in 1 Km X 1 Km area – 3 TCP connections • Speed uniform between 0 and 10 m/s – random waypoint model. • At a time between 50 and 60 seconds a link failure occurs. • Notice that after this, if proxies are used performance is better. • Fair share for all three connections in spite of failure. • If no proxies are used the share of channel of the connection with failed link taken by other two connections. • Elimination of intra-segment contention on connection with failed link improves performance 38

2. Fairness Improvement • Multiple TCP connections with varying lengths in terms of hop count. • Longer connections achieve lower throughput than shorter ones. • Introduction of proxies improves throughput • For a connection of length 16 hops, the throughput improves from around 22 Kbps to 27 Kbps. • Improvement in fairness 39

3. Alleviating Capture Effect • Two connections are active in the vicinity of each other. • Both are heavily loaded. • Connection 2 begins slightly later than Connection 1. • If TCP is used without proxies, Connection 2 has a very low share of the throughput until Connection 1 is done sending its data. • TCP with proxies alleviates this effect – fairer share of bandwidth is available to Connection 2. 40

Conclusion of Overview We identify the factors that degrade the performance of TCP over MANETs. n We identify the most promising TCP variants: n ¨ TCP-Westwood, TCP-Jersey ¨ Current TCP variants do not seem sufficient n Promising transport protocols emerge ¨ Split-TCP, ATP 41

TCP Variants Additional Information 42

Bandwidth Estimation TCP-Vegas n Rate-based congestion control diff = expected rate – actual rate ¨ If diff < a, Vegas increases cwnd linearly ¨ If diff > b, Vegas decreases cwnd linearly ¨ If a< diff < b, Vegas keeps cwnd unchanged ¨ n Modified slow-start Allows cwnd to grow exponentially only in every other RTT ¨ If diff > c, Vegas switches from slow-start to congestion avoidance ¨ n New retransmission Reads and records transmission time. ¨ When DUPACK arrives, checks if it is expired. ¨ Retransmits without waiting for third DUPACK. ¨ 43

Bandwidth Estimation TCP-Westwood - I n Bandwidth estimation ¨ Measure the rate of return of ACKs. ¨ Adaptive to variation of inter-arrival of ACKs. ¨ Exponentially goes to zero upon prolonged absence of ACKs. n Faster recovery ¨ Compute cwnd and ssthresh using the estimated bandwidth upon any loss event. 44

TCP-Westwood - II n Main idea: Bandwidth estimation Sender monitors ACKs to estimate the bandwidth available to the connection. ¨ Two pieces of information are used: ¨ n n n ACK reception rate. Information the ACKs conveys (amount of data delivered). Bandwidth calculation ¨ The sampled bandwidth (bk) at time tk is: where dk is amount of data delivered and Δk is the time difference between the recent and the last reception of ACK. ¨ Discrete-time filter (Tustin approximation). The filtered estimate of the available bandwidth at time tk is: where filter. ¨ and 1/τ is the cutoff frequency of the The weight ak are made to depend on Dk to counteract the effect of nondeterministic inter-arrival times. 45

Bandwidth Estimation TCP-Jersey n Available Bandwidth Estimation ¨ n Time-sliding window estimator Congestion Warning (CW) Mark all packets if the average queue length exceeds a threshold. ¨ DUPACK with CW mark congestion ¨ DUPACK without CW mark wireless error ¨ Adjust cwnd and ssthresh if receives (DUP)ACK with CW mark ¨ n Explicit retransmit ¨ If DUPACK without CW mark, retransmits with cwnd unchanged 46

Route Failure and Wireless Error Determination Explicit Link Failure Notification (ELFN) n n Upstream node of the failed link sends a “host unreachable” ICMP message to the sender. Sender disables retransmission timers and enters a “standby” mode. Periodic probing to detect restored route. Restores retransmission timers and continues transmissions when ACK is received. 47

Contention Reduction Link RED (LRED) and Adaptive Pacing (AP) n Link RED (LRED) Maintain an average # of retries of recent packet transmission. ¨ If exceeds the minimum threshold minth, LRED marks packets with probability depending on the average # of retries value. ¨ TCP will then reduces sending rate. ¨ n Adaptive Pacing (AP) Distribute traffic in a more balanced way. ¨ Let some nodes wait an extra amount of back-off period. ¨ Use in conjunction with LRED. ¨ When LRED starts to mark packets, AP increases the back-off time of the pending transmission. ¨ 48

Contention Reduction Congestion Window Limit (CWL) n The maximum spatial reuse of wireless channel is 1/4 of the chain. 1 2 3 4 5 6 7 8 9 When 1 transmits, 2 and 3 cannot transmit, but 4 can. ¨ Assume perfect scheduling and no contention. ¨ n n TCP achieves best throughput when cwnd is approximately 1/5 of the round-trip hop-count (RTHC). Adaptively adjust the maximum cwnd to ensure the spatial reuse is not exceeded. 49

50

Other Transport Layer Efforts Can FAST TCP and XCP work well over MANETs? n Do not seem suitable for MANETs. ¨ Basic idea: React faster to change. ¨ Fast TCP [Jin et. al. ] n ¨ XCP [Katabi et. al. ] n n n Determine equilibrium by queuing delay and loss information. ¨ cwnd far away from equilibrium? Rapid (Large) change. ¨ cwnd approach equilibrium? Small change. Explicit congestion signaling. ¨ Intermediate nodes estimate spare bandwidth and generate feedback to the sender. Neither protocol can deal with mobility. Mobility and route changes will throw off calculations. 51

Route Failure and Wireless Error Determination Ad-hoc TCP (ATCP) n Normal state ¨ n Connection initialization. TCP works normally. Loss state RTO almost expired or ATCP receives the 3 rd DUPACK. ¨ TCP in persist mode (no congestion control). ¨ ATCP retransmits lost segment. ¨ n Congested state ¨ n When ECN is received. TCP works normally. Disconnected state When “Destination Unreachable” is received. ¨ TCP in persist mode. ¨ Send probe packets to detect re-connection. ¨ 52

Route Failure and Wireless Error Determination ADTCP n Classification of network states ¨ n Congestion(CONG), Channel error(CHERR), Route change(RTCHG), Disconnection(DISC) Multiple metrics Inter-packet delay difference (IDD) ¨ Short-term throughout (STT) ¨ Packet Out-of-order delivery Ratio (POR) ¨ Packet Loss Ratio (PLR) ¨ n Identifying network states 53

Details of some major TCP variants Detecting Spurious Retransmission n TCP-Eifel [Ludwig & Katz] ¨ ¨ ¨ n Use timestamp option of TCP to solve retransmission ambiguity. Sender can determine whether the received ACK is from original transmission or retransmission. n After retransmission, if next ACK is from original transmission spurious! Restore the cwnd and ssthresh if spurious retransmission is detected. Pro: Robust to sudden delay spike. Con: Require the use of timestamp option or modification of TCP header. Forward RTO-Recovery (F-RTO) [Sarolahti et. al. ] If the first ACK after retransmission advances the window, send two new segments. ¨ If the next ACK still advances the window retransmission is likely to be spurious. (It should be a DUPACK generated by the new transmitted segments) ¨ Pro: Does not require the use of timestamp option. ¨ Con: Only detect spurious retransmission triggered by timeout. ¨ 54

Contention Reduction TCP Adaptive Delayed Acknowledgement (TCP-ADA) n n n Reduce # of ACKs reduce contention on forward data transmissions. Maximizing the number of packet received before an ACK is sent, K, increases TCP throughout. K is equal to a full window of packets. Receiver: estimate the average inter-arrival time of data packets. Wait “wait factor x average inter-arrival time” before sending the corresponding cumulative ACK. If the waiting time is completed, sends ACK. 55

Asymmetry Alleviation ACK Congestion Control (ACC) & ACK Filtering (AF) n ACK Congestion Control (ACC) Reduce frequency of ACKs. ¨ Use RED to mark ECN bit of packets. ¨ Packet with ECN bit set ¨ n n ¨ For each subsequent RTT with no packet with ECN bit set n n Sender: reduces the sending rate Receiver: increases multiplicatively the delayed-ACK factor, d Receiver: decrease linearly the delayed-ACK factor, d ACK Filtering (AF) ACKs are cumulative. ¨ Traverse the queue to remove some (or all) of the ACKs of the same flow. ¨ 56

Specific problems of TCP over MANETs n Delayed retransmissions due to the use of coarse-grained timer [Brakmo et. al. ] ¨ Timer ¨ Effects: Inaccuracy in the calculation of the RTO performance drops. n n n RTT and mean variance estimates are computed using coarse-grained timer (around 500 ms). This granularity influence also how often TCP checks for an timeout event. Calculated RTO is >= actual RTO. ¨ Why is it bad for MANETs? n TCP heavily relies on this inefficient timeout mechanism to detect losses. n Why? ¨ cwnd is usually small. ¨ Not enough DUPACKs to trigger the fast retransmit. 57

So Far, So Good: Next Steps n Split-TCP: a new way to look at transport layer ¨ Separate congestion control from reliability ¨ Use control theory to analyze the problem n Use directed antennas to improve TCP thruput ¨ “Focusing” the beam, minimizes interference ¨ Challenge: co-develop an appropriate MAC protocol n Exploit other physical layer capabilities ¨ CDMA, cooperative diversity 58

Exciting Projects at UCR n Exploit Spatial Diversity in MANETs ¨ MIMO systems: multiple element antennas ¨ Cooperative diversity: virtual MIMO n Develop secure wireless networks ¨ Detecting misbehavior and intrusions ¨ Develop a complete comprehensive architecture n Develop UWB-based networks ¨ Design a CDMA-based MAC protocol ¨ Develop a routing protocol 59

Details of some major TCP variants Alleviating Asymmetry n ACK Congestion Control (ACC) ¨ ¨ ¨ n Use Random Early Detection (RED) algorithm to mark packets. When sender receives packets with mark reduce the sending rate. When receiver receives packets with mark multiplicatively increase the delay-ACK factor, d. If receiver receives no mark for a RTT linearly decrease the factor d. Pro: Reduce congestion on the constrained reverse path. Con: Require RED to be implemented in each node. Increase burstiness of the forward data flow. ACK Filtering (AF) Take the advantage of the fact that ACKs are cumulative. ¨ Remove some (or all) the ACKs buffered in the queue that belong to the same flow. ¨ Pro: Reduce congestion on the constrained reverse path. ¨ Con: Special queue management is needed. Increase burstiness of the forward data flow. ¨ 60