In VINI Veritas Realistic and Controlled Network Experimentation

In VINI Veritas Realistic and Controlled Network Experimentation Andy Bavier Nick Feamster* Mark Huang Larry Peterson Jennifer Rexford Princeton University *Georgia Tech Modified and presented by Arjumand Younus and Moonyoung Chung

Outline n n Motivation Scientific Value and Importance of VINI Fundamental Philosophy of VINI Design Requirements n n Strategy for building VINI PL-VINI: prototype on Planet. Lab Experimental results Conclusion 2

Motivation - How to Validate an Idea? Emulation Simulation VINI Small-scale experiment Live deployment Fixed, shared among many experiments n Runs real routing software n Exposes realistic network conditions n Gives control over network events n Carries traffic on behalf of real users n 3

Realism has Scientific Value The most exciting phrase to hear in science, the one that heralds new discoveries, is not ‘Eureka!’ (I found it!) but ‘That’s funny …’ -- Isaac Asimov n Discover unanticipated interactions ¨ Can only simulate / emulate known properties ¨ Build richer models n Test robustness and scalability of ideas 4

Fundamental Philosophy “Controlled Realism” Arbitrary, emulated Actual network n Real clients, servers n ¨ Reproduce results ¨ Methodically change or relax constraints Topology Synthetic or traces Traffic Inject faults, anomalies Observed in operational network Network Events Control: Realism: ¨ Long-running services attract real “customers” ¨ Forward high traffic volumes (Gb/s) ¨ Robustly handle unexpected events 5

Overview n VINI requirements ¨ Fixed, shared infrastructure ¨ Flexible network topology ¨ Expose/inject network events ¨ External connectivity and routing adjacencies PL-VINI: prototype on Planet. Lab n Experimental results n Conclusion n 6

Fixed Infrastructure VINI nodes embedded in Abilene 7

Shared Infrastructure Experiments given illusion of dedicated hardware 8

Shared Infrastructure Experiments given illusion of dedicated hardware 9

Flexible Topology VINI supports arbitrary virtual topologies 10

Flexible Topology VINI supports arbitrary virtual topologies 11

Network Events VINI can inject network failures 12

Network Events real link fails virtual link fails VINI can expose network failures (coming soon) 13

External Connectivity c s Experiments can carry traffic for real end-users 14

External Routing Adjacencies BGP c BGP s BGP Experiments can participate in Internet routing 15

Planet. Lab VINI n Build VINI from Planet. Lab, a global testbed for distributed services ¨ Begun in 2002 ¨ 700 nodes at 336 sites in 35 countries ¨ 600 projects and 2500 researchers ¨ Serves 3 -4 TB/day to ~1 M clients n My. PLC: Planet. Lab software distribution ¨ Anyone can run their own private Planet. Lab 16

PL-VINI: Prototype on Planet. Lab n n Planet. Lab: testbed for planetary-scale services Simultaneous experiments in separate VMs ¨ Each n has “root” in its own VM, can customize Can reserve CPU, network capacity per VM Node Mgr Local Admin VM 1 VM 2 … VMn Planet. Lab node Virtual Machine Monitor (VMM) (Linux++) 17

Resource Isolation n Issue: Forwarding packets in user space ¨ Planet. Lab sees heavy use ¨ CPU load affects virtual network performance Property Depends On Solution Throughput CPU% received Planet. Lab provides CPU reservations Latency CPU scheduling delay PL-VINI: boost priority of packet forward process 18

PL-VINI: Prototype on Planet. Lab n Enable experiment: Internet In A Slice ¨ XORP open-source routing protocol suite (NSDI ’ 05) ¨ Click modular router (TOCS ’ 00, SOSP ’ 99) n Clarify issues that a VINI must address ¨ Unmodified routing software on a virtual topology ¨ Forwarding packets at line speed ¨ Illusion of dedicated hardware ¨ Injection of faults and other events 19

Synthesis of Software Components to enable PL-VINI n n XORP – Open Source Routing Protocol Suite especially developed for network research community. [9, NSDI ’ 05] Click – Software Architecture for Building Flexible and Configurable routers [10] UML – Linux-Kernel running as regular Linux-User. Process without Root-Privileges. Open. VPN – Open source SSL VPN solution. 20

XORP: Control Plane XORP (routing protocols) n n BGP, OSPF, RIP, PIM -SM, IGMP/MLD Goal: run real routing protocols on virtual network topologies 21

User-Mode Linux: Environment UML XORP n (routing protocols) eth 0 eth 1 eth 2 eth 3 n Interface ≈ network Planet. Lab limitation: ¨ Slice cannot create new interfaces n n Run routing software in UML environment Create virtual network interfaces in UML 22

Click: Data Plane UML XORP n (routing protocols) eth 0 eth 1 eth 2 Performance ¨ Avoid UML overhead ¨ Move to kernel, FPGA eth 3 Control Data Packet Forward Engine Click 23

Click: Data Plane UML XORP n (routing protocols) eth 0 eth 1 eth 2 Performance ¨ Avoid UML overhead ¨ Move to kernel, FPGA eth 3 Control Data Packet Forward Engine Uml. Switch element Click 24

Click: Data Plane UML XORP n (routing protocols) eth 0 eth 1 eth 2 ¨ Avoid UML overhead ¨ Move to kernel, FPGA eth 3 Control Data Packet Forward Engine Performance Uml. Switch element Tunnel table n Interfaces tunnels ¨ Click UDP tunnels correspond to UML network interfaces Click 25

Injecting Link Failures UML XORP n (routing protocols) eth 0 eth 1 eth 2 ¨ Avoid UML overhead ¨ Move to kernel, FPGA eth 3 Control n Data Packet Forward Engine Click Performance ¨ Click UDP tunnels correspond to UML network interfaces Uml. Switch element Tunnel table Filters Interfaces tunnels n Filters ¨ “Fail a link” by blocking packets at tunnel 26

Life of a packet in IIAS 27

Experiments n Microbenchmarks ¨ Capacity ¨ Behavior n Intra domain Route changes ¨ Failure and recovery 28

Overlay Efficiency n IIAS’s user-space Click forwader vs. inkernel forwarding 29

Performance is bad User-space Click: ~200 Mb/s forwarding n Can do a lot with 200 Mb/s n ¨ 20 experiments can have dedicated 10 Mb/s nationwide networks n Improving performance is ongoing work ¨ Allow experiments to load custom Click modules into the VINI kernel 30

Overlay on Planet. Lab n on dedicated hardware (DETER) vs. shared platform (Planet. Lab) and PL-VINI 31

Overlay on Planet. Lab n PL-VINI and IIAS together provide a close approximation of the underlying network’s behaviors. 32

Intra-domain Route Changes s 856 2095 700 260 1295 c 639 366 548 902 1893 233 587 846 1176 Watch OSPF route convergence on Abilene 33

Intra-domain Route Changes s 856 2095 700 260 1295 c 639 366 548 902 1893 233 587 846 1176 Watch OSPF route convergence on Abilene 34

Intra-domain Route Changes s 856 2095 700 260 1295 c 639 366 548 902 1893 233 587 846 1176 Watch OSPF route convergence on Abilene 35

Intra-domain Route Changes s 856 2095 700 260 1295 233 c 548 366 902 1893 587 846 1176 Watch OSPF route convergence on Abilene 36

Intra-domain Route Changes s 856 2095 700 260 1295 c 639 366 548 902 1893 233 587 846 1176 Watch OSPF route convergence on Abilene 37

Ping During Link Failure Link down 38

Ping During Link Failure Link down Link up 39

Ping During Link Failure Routes converging Link down Link up 40

TCP Throughput Link down 41

TCP Throughput Link down Link up 42

TCP Throughput Link down Link up Zoom in 43

Arriving TCP Packets Slow start 44

Arriving TCP Packets Megabytes in stream 2. 45 2. 4 Packet received 2. 35 2. 3 Slow start 2. 25 2. 2 Retransmit lost packet 2. 15 2. 1 17. 5 18 18. 5 19 Seconds 19. 5 20 45

Arriving TCP Packets PL-VINI enables user-space virtual network Slowa start to behave like a real network on Planet. Lab Retransmit lost packet 46

Conclusion n VINI: Virtual Network Infrastructure ¨ Controlled and realistic experimentation ¨ New routing, forwarding, and addressing n Initial experiment ¨ Intradomain routing convergence in XORP ¨ Later, perhaps run commercial router code? n Long term goal ¨ Shared platform for networking experiments ¨ Real user traffic and upstream ISP connections 47