Vytautas Valancius Nick Feamster Akihiro Nakao and Jennifer

Vytautas Valancius, Nick Feamster, Akihiro Nakao, and Jennifer Rexford Wide-Area Route Control for Distributed Services

Cloud Computing Cloud computing is on the rise Provides computing resources and storage in cloud data centers Hosting on the steroids for Internet services 2

Accessing Services in the Cloud ISP 1 Interactive Service Data Center Router Bulk transfer Cloud Data Center Internet Hosted services have different requirements Too slow for ISP 2 Routing updates Packets interactive service, or Too costly for bulk transfer! 3

Cloud Routing Today Multiple upstream ISPs Amazon EC 2 has at least 58 routing peers in Virginia data center Data center router picks one route to a destination for all hosted services Packets from all hosted applications use the same path 4

Route Control: “Cloudless” Solution Obtain connectivity to upstream ISPs Physical connectivity Contracts and routing sessions Obtain the Internet numbered resources from authorities Expensive and time-consuming! 5

Routing with Transit Portal (TP) Interactive Service ISP 1 Virtual Router A Transit Portal Virtual Router B Bulk transfer Cloud Data Center Internet ISP 2 Routes Packets Full Internet route control to hosted cloud services! 6

Outline Motivation and Overview Connecting to the Transit Portal Advanced Transit Portal Applications Scaling the Transit Portal Future Work & Summary 7

Connecting to the TP Separate Internet router for each service Virtual or physical routers Links between service router and TP Each link emulates connection to upstream ISP Routing sessions to upstream ISPs TP exposes standard BGP route control interface 8

Basic Internet Routing with TP ISP 1 Cloud client with two ISP 2 upstream ISPs Traffic Transit Portal BGP Sessions Virtual BGP Router Interactive Cloud Service ISP 1 is preferred ISP 1 exhibits excessive jitter Cloud client reroutes through ISP 2 9

Current TP Deployment Server with custom routing software 4 GB RAM, 2 x 2. 66 GHz Xeon cores Three active sites with upstream ISPs Atlanta, Madison, and Princeton A number of active experiments BGP poisoning (University of Washington) IP Anycast (Princeton University) Advanced Networking class (Georgia Tech) 10

TP Applications: Fast DNS Internet services require fast name resolution IP anycast for name resolution DNS servers with the same IP address announced to ISPs in multiple locations Internet routing converges to the closest server Available only to large organizations 11

TP Applications: Fast DNS TP allows hosted applications use IP anycast Asia ISP 1 North America ISP 2 Transit Portal Name Service ISP 3 Anycast Routes ISP 4 Transit Portal Name Service 12

TP Applications: Service Migration Internet services in geographically diverse data centers Operators migrate Internet user’s connections Two conventional methods: DNS name re-mapping ▪ Slow Virtual machine migration with local re-routing ▪ Requires globally routed network 13

TP Applications: Service Migration Asia ISP 1 Internet ISP 2 Transit Portal Active Game Service North America ISP 3 Tunneled Sessions ISP 4 Transit Portal 14

Scaling the Transit Portal Scale to dozens of sessions to ISPs and hundreds of sessions to hosted services At the same time: Present each client with sessions that have an appearance of direct connectivity to an ISP Prevented clients from abusing Internet routing protocols 15

Conventional BGP Routing Conventional BGP router: ISP 2 ISP 1 Receives routing updates from peers Propagates routing update about one path only Selects one path to forward packets Scalable but not transparent or flexible BGP Router Client BGP Router Updates Packets 16

Scaling BGP Memory Use Store and propagate BGP routes from ISPs all Separate routing tables Reduce memory consumption ISP 1 ISP 2 Routing Process Routing Table 1 Routing Table 2 Single routing process - shared data structures Reduce memory use from Virtual Router 90 MB/ISP to 60 MB/ISP Interactive Service Bulk Transfer 17

Scaling BGP CPU Use Hundreds of routing ISP 1 ISP 2 sessions to clients High CPU load Schedule and send routing Routing Process Routing Table 1 Routing Table 2 updates in bundles Reduces CPU from 18% to 6% for 500 client sessions Virtual Router Interactive Service Virtual Router Bulk Transfer 18

Scaling Forwarding Memory for TP Connecting clients ISP 1 ISP 2 Tunneling and VLANs Curbing memory usage Separate virtual routing tables with default to upstream 50 MB/ISP -> ~0. 1 MB/ISP memory use in forwarding table Forwarding Table 1 Table 2 Virtual BGP Router Interactive Service Virtual BGP Router Bulk Transfer 19

Future Work Future work: More deployment sites Making TP accessible for network research test-beds (e. g. , GENI, Core. Lab) Faster forwarding (Net. FPGA, Open. Flow) Lightweight interface to route control 20

Conclusion Limited routing control for hosted services Transit Portal gives wide-area route control Advanced applications with many TPs Open source implementation Scales to hundreds of client sessions The deployment is real Can be used today for research and education More information http: //valas. gtnoise. net/tp Questions? 21