Multipath TCP Jennifer Rexford Fall 2016 TTh 3

Multipath TCP Jennifer Rexford Fall 2016 (TTh 3: 00 -4: 20 in CS 105) COS 561: Advanced Computer Networks http: //www. cs. princeton. edu/courses/archive/fall 16/cos 561/

Multipath • Mobile user – Wi. Fi and cellular at the same time • High-end servers – Multiple Ethernet cards • Data centers – Rich topologies with many paths • Benefits of multipath – Higher throughput – Failover from one path to another – Seamless mobility 2

Multipath TCP Protocol 3

Working With Unmodified Apps • Present the same socket API and expectations – Identified by the “five tuple” (IP address, port #, protocol) From http: //queue. acm. org/detail. cfm? id=2591369 4

Working With Unmodified Hosts • Establish the TCP connection in the normal way – Create a socket to a single remote IP address/port A SYN CK SYN A ACK B Each host tells its Initial Sequence Number (ISN) to the other host. Data • And then add more subflows, if possible 5

Negotiating MPTCP Capability • How do hosts know they both speak MPTCP? – During the 3 -way SYN/SYN-ACK/ACK handshake • If SYN-ACK doesn’t contain MP_CAPABLE – Don’t try to add any subflows! 6

Adding Subflows, Idealized • How to associate a new subflow with the connection? – Use a token generated from original subflow set-up • How to start using the new subflow? – Simply start sending packets with new IP/port pairs – … and associate them with the existing connection • How could two end-points learn about extra IP addresses for establishing new subflows? – Implicitly: one end-point establishes a new subflow, to already-known address(es) at the other end-point 7

Sequence Numbers • Challenges across subflows – Out-of-order packets due to RTT differences – Access networks that rewrite sequence numbers – Middleboxes upset by discontinuous TCP byte stream – Need to retransmit lost packets on a different subflow • Two levels of sequence numbers – Sequence numbers per subflow – Sequence numbers for the entire connection • Enables – Efficient detection of loss on each subflow – Retransmission of lost packet on a different subflow 8

Receive Buffer Space • Each TCP connection has a receive buffer – Buffer space to store incoming data – … until it is read by the application • TCP flow control – Receiver advertises the available buffer space – … using the “receive window” • Should each subflow have its own receive window? – Starvation of some subflows in a connection? – Fairness relative to other TCP connections? – Fragmentation of the available buffer space? • Instead, use a common receive window 9

Fairness and Efficiency in Multipath Congestion Control Slides from Damon Wischik 10

Goal #1: Fairness at Shared Bottlenecks A multipath TCP flow with two subflows Regular TCP To be fair, Multipath TCP should take as much capacity as TCP at a bottleneck link, no matter how many paths it is using. 11

Goal #2: Use Efficient Paths 12 Mb/s Each flow has a choice of a 1 -hop and a 2 -hop path. How should split its traffic? 12

Use Efficient Paths 12 Mb/s 8 Mb/s 12 Mb/ s 8 Mb/s If each flow split its traffic 1: 1. . . 13

Use Efficient Paths 12 Mb/s 9 Mb/s 12 Mb/s If each flow split its traffic 2: 1. . . 12 Mb/s 9 Mb/s 14

Use Efficient Paths 12 Mb/s 12 Mb/s Better: Each connection on a one-hop path Each connection should send all traffic on the leastcongested paths 15

Use Efficient Paths 12 Mb/s 12 Mb/s Better: Each connection on a one-hop path Each connection should send all traffic on the leastcongested paths But keep some traffic on the alternate paths as a probe 16

Goal #3: Be Fair Compared to TCP • Least-congested paths may not be best! – Due to differences in round-trip time • Two paths – Wi. Fi: high loss, low RTT – Cellular: low loss, high RTT • Using the least-congested path – Choose the cellular path, due to low loss – But, the RTT is high – So throughput is low! 17

Be Fair Compared to TCP • To be fair, Multipath TCP should give a connection at least as much throughput as it would get with a single-path TCP on the best of its paths. – Ensure incentive for deploying MPTCP • A Multipath TCP should take no more capacity on any path (or collection of paths) than if it was a single-path TCP flow using the best of those paths. – Do no harm! 18

Achieving These Goals • Regular TCP – Maintain a congestion window w – On an ACK, increase by 1/w (increase 1 per window) – On a loss, decrease by w/2 • MPTCP – Maintain a congestion window per path wr – On an ACK on path r, increase wr – On a loss on path r, decrease by wr/2 • How much to increase wr on an ACK? ? – If r is the only path at that bottleneck, increase by 1/wr 19

If Multiple Paths Share Bottleneck? • Don’t take any more bandwidth on a link than the best of the TCP paths would – But, where might the bottlenecks be? – Multiple paths might share the same bottleneck • So, consider all possible subsets of the paths – Set R of paths – Subset S of R that includes path r • E. g. , consider path 3 – Suppose paths 1, 3, and 4 share a bottleneck – … but, path 2 does not – Then, we care about S = {1, 3, 4} 20

Achieving These Goals • What is the best of these subflows achieving? – Path s is achieving throughput of ws/RTTs – So best path is getting maxs(ws/RTTs) • What total bandwidth are these subflows getting? – Across all subflows sharing that bottleneck – Sum over s in S of ws/RTTs • Consider the ratio of the two – Increase by less if many subflows are sharing • And pick the results for the set S with min ratio – To account for the most paths sharing a bottleneck 21

Incremental Deployment Challenges of Middleboxes 22

Middleboxes • In-network services, e. g. , – Firewall – Network address translator – Transparent proxy – Intrusion detection system • Interaction with TCP – Change IP addresses and port numbers – Change TCP initial sequence number – Remove TCP options – Dividing large block of data into smaller packets – Expect to see all packets of the connection – Etc. 23

Negotiating MPTCP Capability • What if middleboxes strip the TCP option? – On the SYN? On the SYN-ACK? • Include capability on the ACK of the SYN-ACK? – What if the ACK is lost? – Carry on all subsequent packets • What if the middlebox drops SYN packets with unfamiliar options? – Sender can retransmit lost SYN without the option – … and fall back to regular TCP behavior 24

Challenges: NAT • Network Address Translators (NAT) – Problem: NAT changes the IP address and port number Wi. Fi LTE NAT 1 NAT 2 • How to identify a connection? – Using a token established during connection set-up • How to establish new subflows? – Allow one end-point to tell another about its addresses 25

Challenges: Security • Security – Malicious parties creating subflows – To highjack (part of) the connection • How to bootstrap security? – Include a random key during connection set-up – … and use it to verify authenticity of new subflows • How to identify the connection on new subflows? – A token generated from the key • How to authenticate the addition of subflows? – Exchanging nonces and computing message authentication codes using the keys 26

Use of Multipath TCP in i. OS 7 • Multipath TCP in i. OS 7 (fall 2013) – Primary TCP connection over Wi. Fi – Backup TCP connection over cellular data • Failover – If Wi. Fi becomes unavailable… – … i. OS 7 will use the cellular data connection • For destinations controlled by Apple – E. g. , Siri • See https: //support. apple. com/en-us/HT 201373 27

Discussion 28

Backup Slides: Review of TCP Protocol 29

Establishing a TCP Connection A B SYN C SYN A K ACK Each host tells its Initial Sequence Number (ISN) to the other host. Data • Three-way handshake to establish connection – Host A sends a SYN (open) to the host B – Host B returns a SYN acknowledgment (SYN ACK) – Host A sends an ACK to acknowledge the SYN ACK 30

Initial Sequence Number (ISN) • Sequence number for the very first byte – E. g. , Why not a de facto ISN of 0? • Practical issue: reuse of port numbers – Port numbers must (eventually) get used again – … and an old packet may still be in flight – … and associated with the new connection • Security issue: adversary injecting packets – Adversary may try to inject packets in a connection – … by guessing the Initial Sequence Number – … to send counterfeit packets to the receiving host – … e. g. , counterfeit packets that reset the connection – Some firewalls change the ISN to further randomize 31

Step 1: A’s Initial SYN Packet A’s port B’s port A’s Initial Sequence Number Flags: SYN FIN RST PSH URG ACK Acknowledgment 20 Flags 0 Checksum Advertised window Urgent pointer Options (variable) A tells B it wants to open a connection… 32

Step 2: B’s SYN-ACK Packet B’s port A’s port B’s Initial Sequence Number Flags: SYN FIN RST PSH URG ACK A’s ISN plus 1 20 Flags 0 Checksum Advertised window Urgent pointer Options (variable) B tells A it accepts, and is ready to hear the next byte… … upon receiving this packet, A can start sending data 33

Step 3: A’s ACK of the SYN-ACK A’s port B’s port Sequence number Flags: SYN FIN RST PSH URG ACK B’s ISN plus 1 20 Flags 0 Checksum Advertised window Urgent pointer Options (variable) A tells B it is okay to start sending … upon receiving this packet, B can start sending data 34

Sequence Number Host A ISN (initial sequence number) Byte 81 Sequence number = 1 st byte Host B TCP Data 35

TCP Header Source port Destination port Sequence number Flags: SYN FIN RST PSH URG ACK Acknowledgment Hdr. Len 0 Flags Advertised window Checksum Urgent pointer Options (variable) Data 36

Receive Buffering: Flow Control • Receive window size – Amount that can be sent without acknowledgment – Receiver must be able to store this amount of data • Receiver tells the sender the window – Tells the sender the amount of free space left Window Size Data ACK’d Outstanding Un-ack’d data Data OK to send Data not OK to send yet 37

TCP Header: Receive Window Source port Destination port Sequence number Flags: SYN FIN RST PSH URG ACK Acknowledgment Hdr. Len 0 Flags Advertised window Checksum Urgent pointer Options (variable) Data 38

Tearing Down the Connection ACK FIN Data ACK FIN ACK CK A SYN B time • Closing (each end of) the connection – Finish (FIN) to close and receive remaining bytes – And other host sends a FIN ACK to acknowledge – Reset (RST) to close and not receive remaining bytes 39

Extending TCP: TCP Options • TCP header – Ten mandatory fields – Optional extension field (usually during handshake) • Examples – Maximum segment size (MSS) – Window scaling – Support for Selected ACKs But, some middleboxes: (i) • Unknown options – Ignored by receiving host strip TCP options from some packets or (ii) drop packets with TCP options • Routers and TCP options – Should ignore them, passing them through unchanged 40