VROOM Virtual ROuters On the Move Jennifer Rexford

VROOM: Virtual ROuters On the Move Jennifer Rexford Joint work with Yi Wang, Eric Keller, Brian Biskeborn, and Kobus van der Merwe (AT&T) http: //www. cs. princeton. edu/~jrex/papers/vroom 08. pdf

Virtual ROuters On the Move • Key idea – Routers should be free to roam around • Useful for many different applications – Simplify network maintenance – Simplify service deployment and evolution – Reduce power consumption –… • Feasible in practice – No performance impact on data traffic – No visible impact on routing protocols 2

The Two Notions of “Router” • IP-layer logical functionality, and physical equipment Logical (IP layer) Physical 3

Tight Coupling of Physical & Logical • Root of many network-management challenges (and “point solutions”) Logical (IP layer) Physical 4

VROOM: Breaking the Coupling • Re-mapping logical node to another physical node VROOM enables this re-mapping of logical to Logical physical through virtual router migration. (IP layer) Physical 5

Case 1: Planned Maintenance • NO reconfiguration of VRs, NO reconvergence VR-1 A B 6

Case 1: Planned Maintenance • NO reconfiguration of VRs, NO reconvergence A VR-1 B 7

Case 1: Planned Maintenance • NO reconfiguration of VRs, NO reconvergence A VR-1 B 8

Case 2: Service Deployment/Evolution • Move (logical) router to more powerful hardware 9

Case 2: Service Deployment/Evolution • VROOM guarantees seamless service to existing customers during the migration 10

Case 3: Power Savings • $ Hundreds of millions/year of electricity bills 11

Case 3: Power Savings • Contract and expand the physical network according to the traffic volume 12

Case 3: Power Savings • Contract and expand the physical network according to the traffic volume 13

Case 3: Power Savings • Contract and expand the physical network according to the traffic volume 14

Virtual Router Migration: Challenges 1. Migrate an entire virtual router instance • All control plane & data plane processes / states 15

Virtual Router Migration: Challenges 1. Migrate an entire virtual router instance 2. Minimize disruption • • Data plane: millions of packets/sec on a 10 Gbps link Control plane: less strict (with routing message retrans. ) 16

Virtual Router Migration: Challenges 1. Migrating an entire virtual router instance 2. Minimize disruption 3. Link migration 17

Virtual Router Migration: Challenges 1. Migrating an entire virtual router instance 2. Minimize disruption 3. Link migration 18

VROOM Architecture Data-Plane Hypervisor Dynamic Interface Binding 19

VROOM’s Migration Process • Key idea: separate the migration of control and data planes 1. Migrate the control plane 2. Clone the data plane 3. Migrate the links 20

Control-Plane Migration • Leverage virtual server migration techniques • Router image – Binaries, configuration files, etc. 21

Control-Plane Migration • Leverage virtual server migration techniques • Router image • Memory – 1 st stage: iterative pre-copy – 2 nd stage: stall-and-copy (when the control plane is “frozen”) 22

Control-Plane Migration • Leverage virtual server migration techniques • Router image • Memory CP Physical router A DP Physical router B 23

Data-Plane Cloning • Clone the data plane by repopulation – Enable migration across different data planes – Avoid copying duplicate information Physical router A DP-old CP Physical router B DP-new 24

Remote Control Plane • Data-plane cloning takes time – Installing 250 k routes may take several seconds • Control & old data planes need to be kept “online” • Solution: redirect routing messages through tunnels Physical router A DP-old CP Physical router B DP-new 25

Remote Control Plane • Data-plane cloning takes time – Installing 250 k routes takes over 20 seconds • Control & old data planes need to be kept “online” • Solution: redirect routing messages through tunnels Physical router A DP-old CP Physical router B DP-new 26

Remote Control Plane • Data-plane cloning takes time – Installing 250 k routes takes over 20 seconds • Control & old data planes need to be kept “online” • Solution: redirect routing messages through tunnels Physical router A DP-old CP Physical router B DP-new 27

Double Data Planes • At the end of data-plane cloning, both data planes are ready to forward traffic DP-old CP DP-new 28

Asynchronous Link Migration • With the double data planes, links can be migrated independently A DP-old B CP DP-new 29

Prototype Implementation • Virtualized operating system – Open. VZ, supports VM migration • Routing protocols – Quagga software suite • Packet forwarding – Linux kernel (software), Net. FPGA (hardware) • Router hypervisor – Our extensions for repopulating data plane, remote control plane, double data planes, … 30

Experimental Evaluation • Experiments in Emulab – On realistic Abilene Internet 2 topology 31

Experimental Results • Data traffic – Linux: modest packet delay due to CPU load – Net. FPGA: no packet loss or extra delay • Routing-protocol messages – Core router migration (intradomain only) • Inject an unplanned link failure at another router • At most one retransmission of an OSPF message – Edge router migration (intra and interdomain) • Control-plane downtime: 3. 56 seconds • Within reasonable keep-alive timer intervals – All routing-protocol adjacencies stay up 32

Where To Migrate • Physical constraints – Latency • E. g, NYC to Washington D. C. : 2 msec – Link capacity • Enough remaining capacity for extra traffic – Platform compatibility • Routers from different vendors – Router capability • E. g. , number of access control lists (ACLs) supported • Constraints simplify the placement problem – By limiting the size of the search space 33

Conclusions on VROOM • VROOM: useful network-management primitive – Separate tight coupling between physical and logical – Simplify network management, enable new applications • Evaluation of prototype – No disruption in packet forwarding – No noticeable disruption in routing protocols • Future work – Migration scheduling as an optimization problem – Extensions to router hypervisor for other applications 34

Other Projects Related to Router Virtualization

Bug-Tolerant Routers • Seriousness of routing software bugs – Cause serious outages, misbehavior, vulnerability – Violate protocol semantics, so not handled by traditional failure detection and recovery • Handling bugs at run time – Run multiple routing instances in parallel – Use different execution environments, message timings/orderings, or code bases – Vote on “answers” forwarding-table entries and messages to neighboring routers Collaboration with Matt Caesar and Yuanyuan Zhou at UIUC

Virtual Network Infrastructure (VINI) • Experimental platform for network research – Evaluating prototypes of network architectures – Supporting multiple experiments in parallel – Carrying real user traffic & connecting to Internet • VINI platform (www. vini-veritas. net) – Virtual nodes, links, and network stack in Linux – Instantiation of virtual topology for experimenters – VINI nodes deployed in NLR and Internet 2 Collaboration with Nick Feamster (GA Tech) and Andy Bavier and Larry Peterson (Princeton)

Concurrent Architectures are Better than One (CABO) • Overcome limitations of today’s Internet – Applications with diverse requirements – Too many (sometimes conflicting) goals – Difficulty of coordinating across domains • New architecture based on virtualization – Infrastructure providers: own and manage equipment, and host virtual nodes and links – Service providers: run virtual networks customized to their end-to-end services Collaboration with Nick Feamster (GA Tech) and Lixin Gao (UMass)

Dynamically Adaptive Virtual Networks for a Customized Internet (Da. Vinci) • Multiple applications with different goals – E. g. , throughput-sensitive and delay-sensitive – Want to operate customized network protocols • How to allocate bandwidth between classes? – Static is inefficient, but dynamic may be risky • Theoretical foundation based on optimization – Customized protocol for each traffic class – Dynamic bandwidth allocation rule for each link – Provably maximizes aggregate performance Collaboration with Mung Chiang (Princeton)

Conclusions • Router virtualization is exciting – Enables wide variety of new networking techniques – … for network management & service deployment – … and even rethinking the Internet architecture • Fascinating space of open questions – What is the right interface to router hardware? – What is the right programming environment for customized protocols on virtual networks? • Looking forward to discussing more!