Virtualization the Generalized Virtualization Model and Innovation within

Virtualization, the Generalized Virtualization Model, and Innovation within GEANT 4 Jerry Sobieski Activity Leader, JRA 2 Network Service Development Chief Research Officer, NORDUnet TERENA 2016, Prague June 13, 2016 Networks ∙ Services ∙ People www. geant. org

Outline • I wanted to give a GTS presentation. . . But I think it better to focus on two topics today: 1. Generalized Virtualization Model 2. Virtualized SDN Switching hardware • Virtualization – what it is, and is not • GTS Virtualization • A Generalized Virtualization Model (GVM) • Innovations: A virtual SDN switching instance Networks ∙ Services ∙ People www. geant. org 2

What is “Virtualization” ? • Interestingly, finding a definition for “Virtual” as it is used today in einfrastructure context is not easy: vir·tu·al (vûr′cho o-əl) (the Free Dictionary) adj. 1. Existing or resulting in essence or effect though not in actual fact, form, or name: the virtual extinction of the buffalo. 2. Existing in the mind, especially as a product of the imagination. Used in literary criticism of a text. 3. Computers Created, simulated, or carried on by means of a computer or computer network: virtual conversations in a chatroom. Virtualization (wikipedia) In computing, virtualization refers to the act of creating a virtual (rather than actual) version of something, including virtual computer hardware platforms, operating systems, storage devices, and computer network resources. (Circular definition? Really? !) Networks ∙ Services ∙ People www. geant. org 3

What is “Virtualization” ? My perspective: • Virtualization is the process of defining and dis-associating the functional service aspects (behaviour) of a thing from the physical infrastructure on which that thing is modeled and/or realized. • Virtual machines(VMs) – look to the user like standalone servers • Virtual circuits (offer a transparent transport conduits for user data regardless if it is realized as sub-rate link sharing, inverse muxing (lag), or multi-layer transport) • Virtual addressing (paging and chip level memory mapping provide a simple memory access model to the user application) • Virtual storage (networked filesystems vs drive/cyl/head/sector) • Virtual objects are abstractions - a sophisticated con game • The behaviour of a virtual object is said to be something, and it may be parameterized • But there is no innate requirement in how that behaviour is achieved – if the user perceives it to be so, then it is virtually that thing. Networks ∙ Services ∙ People www. geant. org 4

What Virtualization is NOT. . . • Virtualization is not “emulation” or “simulation” • Today, the virtual objects are mostly realized in/over hardware, • E. g VMs run natively on real CPU hardware, VCs use real hardware to transport the data • “Virtual” does not mean “slow” !!! • Virtualization is not SDN, nor is it an alternative to SDN • These are complementary coexisting concepts – in a virtual world they can each leverage the other • Virtualization is not “cloud computing” • . . but cloud computing does rely on virtualization to provide many of its elasticity, agility, and scaling features • Virtualization is not “partitioning” / ”slicing” / “delegation”. . . • While partitioning/slicing/etc enable hardware sharing, they still require the application to be aware of the physical context in which they are executing – this exposure breaks the virtualization • E. g. VLAN slicing vs fully encapsulated transport. Networks ∙ Services ∙ People www. geant. org 5

Why is Virtualization important? • Dramatically Improves hardware utilization efficiency – • Multiple users/applications/tenants are able to share hardware infrastructure • Examples: virtual machines, virtual circuits, virtual storage/file systems • Rigorous virtualization models allow migration and grooming of vos to efficiently distribute workload across available infrastructure. • Increased Reliability and Availability • Because the virtual service object can be relocated to different hardware, hardware outages and maintenance windows do not interrupt applications • Security • A well defined virtualization model isolates and insulates the virtual objects from one another • Virtualization provides a well bounded service object in multi-domain service environments – A provider can allocate a virtual circuit or virtual switch for a user, but would not trust a user to interact directly with the local infrastructure. Networks ∙ Services ∙ People www. geant. org 6

Why is Virtualization important? • Flexibility • Virtual objects can be created when needed, and released when idle - freeing resources for other applications, or reducing costs (both Cap. Ex and Op. Ex) • Virtual objects are dynamically created to have the attributes required - no more, no less. • VMs with needed memory or cores or network interfaces, virtual circuits with needed capacity, etc. • Virtual objects can be re-sized as needs evolve • Geographic span and multi-domain • Virtual objects are not tied to specific physical infrastructure and so they can be realized wherever there are capable available facilities • Virtual objects can be explicitly placed anywhere in the world that supports the virtualization service model and that has available capacity • Green • Because virtualization is more efficient with physical infrastructure, less infrastructure is needed (reduced Cap. Ex and Op. Ex) and unused physical infrastructure can be powered down Networks ∙ Services ∙ People www. geant. org 7

Are virtual e-Infrastructure environments useful? . . . I mean really? !. . . For more than just novelties? ? • • • Virtual Machines are already enterprise quality production resources Virtual Circuits are standard production services for 20+ years Bare Metal Servers are managed in huge clusters with standard IPMI tools. . Virtual Storage in various forms is ubiquitous and enterprise quality Emerging Virtual Routers and Switching • Quaga, VMX, OVS, Virtual Open. Flow Switch, . . Better late than never Virtual ≠ Imaginary !! • Not {emulated, simulated, fake, toy, pretend, ephemeral, . . . } • Virtual Environments can support mature production network services and advanced distributed applications as well as experimental research Networks ∙ Services ∙ People www. geant. org 8

GEANT Testbeds Service Networks ∙ Services ∙ People www. geant. org 9

GEANT Testbeds Service – What it is? • The GEANT Testbeds Service provides virtual environments containing transport circuits, switching elements, computational elements, and storage that can be configured to create a user defined and user controlled e-infrastructure facility. . . some users intend to run large scale experimental applications. For these users, GTS provides “Testbeds” • But these virtual environments, i. e. the “Testbeds”, that GTS creates are themselves very stable production capable environments. • What if these flexible dynamic environments were used by production services? Or put another way – What if these types of dynamic virtual environments were the “infrastructure” for more than just the lunatic fringe – what if these virtualization concepts underlay all global network services? Networks ∙ Services ∙ People www. geant. org 10

The GÉANT Testbeds Service –. . . how it works: t works Virtual Circuit “L 1” VC “L 2” Virtual VM “C” Machine VC “A” “L 3” 2. Network conceived to test brilliant idea 3. Researcher logs in, and describes a testbed using a web GUI Resource A port p 0, p 1; Resource B port out 1, out 2; Adj B/out 1==A/p 0; UA 1. Researcher has a brilliant idea Networks ∙ Services ∙ People Switch “B” 4. The User Agent sends the testbed description to GTS using the GTS API Deactivate() Activate() Release() Reserve() GTS API PA 5. The GTS Provider Agent finds and reserves resources for the testbed Release. Resp() Deactivate. Resp() Activate. Resp() Reserve. Resp() 6. Resource ID information is returned to the user and user controls the testbed via the User GUI and other GTS API primitives www. geant. org B L 2 L 1 A C L 3 11

Global ONOS Testbed and Demo - 2016 • ONOS: Demo included US, Brazil, and Europe at ONS (San Fran) and Sig. COMM (London) 2015 • ONOS-EU: Used 32 [GTS] resources • Ran continuously June to September (with the exception of one maintenance window) • ONS 2016 - International ONOS Demo March 2016 • 45 GTS resources across 7 locations in Europe. Networks ∙ Services ∙ People www. geant. org 12

Current GTS Deployment – 2016 -Q 1 • Current GTS Pod locations: • In-service: Amsterdam, Bratislava, Ljubljana, Prague, London, Milan, Hamburg • Coming 2016 -Q 2: Paris, Madrid US-GENI • GTS Infrastructure • ~288 Ubuntu VMs / 36 servers (4 servers/pod) • ~36 Open. Flow fabrics/ 9 Switches @1. 3, OVS @1. 0 • 1 Gbps VCs, 10 G GTS backbone (Hypercube) • Multi-Domain pilots: • • • HEAnet: Dublin CESnet: Prague, Bruno NORDUnet: CPH, WDC, MIA, GVA DFN: Nuremburg (Erlangen), US research: Star. Light (Chicago), Clemson University Networks ∙ Services ∙ People www. geant. org BR-RNP i. Minds 1 AMS LON HAM PRG i. Minds 2 PAR MIL MAD Creat. Net DLab BRA LJU Univ of Roma

An engineering slide: How do we deploy such a service? Internet Access CSF (Central Server Facility) GTS Resource Mgr, Web GUI, RDB, Open. Stack agents, User VPN GWs, NAS Eth/MPLS & 10 G Wave in hypercube mesh AMS LJU PRA LON Pods PODs: Compute Servers, Open. Flow switch, MX MPLS router, To. R mgmt switch Networks ∙ Services ∙ People Red= Control & Mgmt Net Blue= Data plane infra. GN 4 Core Waves Ctrl VRF BRA External Testbeds and/or other networks www. geant. org HAM MIL . . .

The GTS Infrastructure Pod TOR Ctrl Plane Access Switch TOR Access Switch 4 x Servers (Dell 520) VMs 14 x 1 G HP 5900 Open. Flow Switch 32 x 1 G Juniper MX 80 1 x 24 +4 x 10 G GN 4 -1 Networks ∙ Services ∙ People GN 4 -2 Planned Expansion www. geant. org 4 x Servers (Dell 520) 2 x 10 G + 12 x 1 G 8 VMs/server (1 VM/core) 4 x Servers (tbd) 2 x 10 G + 4 x 1 G Bare Metal Servers Corsa Open. Flow Switch 10 x 10 G IB switch 100 TB RAID Array GEANT Core MX 960 40 x 10 Gbps, 100 G backbone

Innovations in GTS • Formal Generalized Virtualization Model – Abstract resource model, standard lifecycle. • Derived resource graph - Separates “resource” specifications from “topology” description. • Programmable, object oriented, dataflow grammar - “DSL” – • Technology agnostic – The model does not rely on any specific underlying hardware or technology • Multi-domain One Stop Shopping – Users can request a global virtual environment from a single GTS service provider • Guaranteed bookahead reservations – Assured resource availability and future service scheduling. • Fully encapsulated Virtual Circuits – RFC 4448 Eo. MPLS (and similar encapsulating protocols) provide transparent performance engineered data transport end to end. (Not “slicing”) • NSI based inter-domain circuit provisioning – Global dynamic virtual circuits • Virtualized Open. Flow Switching – Simplifies multi-domain SDN applications, allows efficient infrastructure sharing, and enables operational grooming and management of physical hardware. Networks ∙ Services ∙ People Objectives www. geant. org Challenges Conclusions Q&A 1

The Generalized Virtualization Model Networks ∙ Services ∙ People www. geant. org 17

Virtualization, the GEANT Testbeds Service, and a Generalized Virtualization Model • Given these advantages of virtualized e-infrastructure. . . Why do we not have these features as part of our basic underlying infrastructure? • Can we develop a common virtualization model that covers existing virtualized e-infrastructure and that is general enough to be applicable to a much broader set of e-infrastructure components and capabilities? • Yes. Existence proof: GEANT Testbeds Service (GTS) • GTS – the service – is based on a number of design decisions and innovations that together we refer to as the GTS Architecture • This architecture is relatively new. It is well thought out (IMHO) and draws from a number of predecessor projects. And it is working well in the GTS implementation. . • . . . but it is still high level and there are lots of refinements and additions that must be done – which should be done through an open community based process. • Therefore, the architecture that GTS implements has been renamed as the Generalized Virtualization Model (GVM), and will be the topic of community input and extension. • This also allows service providers to deploy “GVM compliant” services that will be interoperable. Networks ∙ Services ∙ People www. geant. org 18

A Generalized Virtualization Model - “GVM” Virtualized resources, in user defined/controlled topologies • GVM treats all networks/workflows/distributed applications as data flow. Openflow graphs. Switch B “B” Link “L 1” Virtual Machine “A” A Link “L 2” Link “L 3” C • All network components (the nodes and links in the graph) are treated as generalized Resources • Data transits resources thru explicitly defined interfaces, or Ports • Data flow topology is defined by port Adjacencies Resources dst class: Link src class: Host “Derived Resource Graph” data plane Networks ∙ Services ∙ People www. geant. org Adjacencies class: OFX X 86 Server “C” Testbed “Alpha” as conceived Ports p 0 B p 1 src L 2 L 1 if 0 A if 2 if 1 src L 3 dst Class: Link class: Link dst if 2 if 3 if 1 C class: BMS

What are “Resources” • “Resources” are the objects that make up the user’s testbed. – – • Examples: Virtual circuits, servers, VMs, switches, routers, functional elements, etc. The virtualization services take the users’ resource descriptions and map those requirements to underlying infrastructure. A Resource “Class” describes resource with a particular set of attributes – – One attribute is a functional human-readable description of what the resource does. . . The other attributes provide more parametric information: – • Ex: VM : = #cores=1, 2, 3, 4; Memory= 4, 8, 16 GB; . . . Resource. Class descriptors define the key attributes of that class of resource: – – – Ports – define specific ingress and egress data interfaces for this type of element Attributes – characteristics of the resource, e. g. Bandwidth=100 Mbps, Cores=4 Control primitives /methods – e, g Reserve(), Activate(), Set Datapath. ID(), etc. Children resources – other resources incorporated in a hierarchical composite Adjacencies – Adjacencies define how ports are connected and thus the topology Networks ∙ Services ∙ People www. geant. org

Common Resource Control Primitives: The standard resource lifecycle API • Define() – define a new class of composite resource for a project • Reserve() – A request from RA to PA to find resources and to reserve them for this user/project. • Activate() – Given a reserved resource, this primitive provisions the resource and places the resource into service. • Query() – Obtain the resource specific state information for a particular resource instance • Deactivate() – Take a resource instance out of service, but retain the reservation. • Release() – deactivate a resource and release the reservation • Undefine() – remove a resource class definition from a project Networks ∙ Services ∙ People www. geant. org

Resource taxonomy Computational node • Resources are organized by their shared functional characteristics: Transport link Bare Metal Server “BMS” P 2 P VC (EFTS) Composite Virtual Machine “Host” Containers (soft NFV/NSC) L 2 MP (EVPN) External Domains Switching/forwarding element Open. Flow Fabric “OFX” Virtual Open. Flow Switch “VOX” Soft Open. Flow Switch “OVX” Soft Virtual Router (VMX) Networks ∙ Services ∙ People www. geant. org L 3 MP (VRF) LTE (mobile wireless) Storage resources TBD Simple scheduled resources Instruments NFV/NSC constructs: Encryption Encapsulation Filters Rate limiters MP switching. .

The “Resource” – The GVM building block From atomic resource classes to running networks “Host” atomic class: “Link” atomic class: Ubuntu VM, 4 GB, P 1 P 2 1 core. Src Dst P 1 P 2 Host Src L 1 Dst P 2 H 2 Link Host Src Dst P 2 work Src Dst Paris. Ofc prot Link Regional. Ofc Networks ∙ Services ∙ People www. geant. org Link work. Dst Src prot Dst “DRnetwork” composite class Link Src A composite resource contains other Resources, external ports, and port adjacencies P 1 Acme. Widgets “Regional. Ofc” composite class P 1 P 2 H 1 Instance of class “DRnetwork” named “Acme. Widgets” Eo. MPLS VC, BW P 1 Host Paris. Ofc P 2 Src. L 1 Dst Link P 1 P 2 London. Ofc Regional. Ofc Link P 1 P 2 P 1 H 2 P 2 Host P 1 P 2 P 1 Host London. Ofc P 2 Src. L 1 Dst Link P 1 H 2 P 2 Host Class=Regional. Ofc instance #1 instance #2 Class=DRnetwork

GVM Functional Layers: Virtualization, Management, and User Control User Agent User Network Environment GVM API GVM Resource Manager Provider Agent GVM API Virtual Circuits External Resources Other GVM Domains Virtual Storage Virtual Switches Virtual Machines RCA-VC Open. NSA RCA-OFX HPOS, OFL Composite Resources (User Testbed layer) RCA-VM RCA-ST Open. Stack <tbd> GVM Virtual Resources Generalized Virtualization Services Layer Physical Infrastructure Layer Networks ∙ Services ∙ People www. geant. org Challenges Achievements Conclusions Q&A 2

GVM Layered Resource Virtualization User C Requesting Bare. Metal User B User A Ubuntu Virtual Machine Resources Open Vswitch Resources “OVS” Resource Virtualization Infrastructure Resources “Host” Resource Virtualization Layer (Ubuntu VM/Open. Stack) Infrastructure Bare Metal Server Resources Networks ∙ Services ∙ People User X User Y User X www. geant. org Resources Bare. Metal Resource Virtualization Layer Infrastructure Data Center blade server Resources Rack servers

Virtualization Layering – Why a formal specification is important Resources are formally described using the DSL specification Virtualization layer Infrastructure objects Resources Virtualization layer The virtualization model allows stacking of virtualization layers so that each layer can act as provider or client as needed. The resources are instantiated using class specific configuration processes acting upon infrastructure object(s) to generate an output resource of the specified type or class. Infrastructure objects Resources Virtualization layer Infrastructure objects Networks ∙ Services ∙ People www. geant. org Since the “resources” and the “infrastructure” are at times the same object(s), they can be described using the same DSL specifications ! This allows the same tools that monitor and manage Resources to also handle Infrastructure, and they can function at all levels for any client/provider.

GVM Multi-Domain Peering Basic User-Provider relationship 1. Request Basic User-Provider agent interaction GTS API UA PA 2. Response In the Mult-Domain model, the Multi-Domain Provider Agent plays both roles: “Provider” to upstream User Agents, and “User” to downstream Provider Agents 1. Request GTS API UA 4. Response Networks ∙ Services ∙ People www. geant. org 2. Request MD PA MDPA GTS API PA 3. Response

Multi-Domain One-Stop Shopping UA User Local PA 0 RCA-VM RCA-VC GVM API Provider MD PA User Provider GVM API Local PA 1 The MDPA can implement any heuristic search algorithm from exhaustive breadth-first search to a directory based discovery. . . This is an implementation issue. GVM Domain A “downstream” domains User GVM API Local PA 1 RCA-OFX MD PA Networks ∙ Services ∙ People Provider www. geant. org GVM Domain D MD PA GVM Domain C GVM Domain B . . to other downstream domains

Why is GVM important? • GVM recognizes that a global “network” consists of many components – not just transport links • GVM is not just about networking, but about how we deliver application specific e-infra services more generally • Network transport and switching capabilities being just a part of the whole environmnent • GVM asserts a service model that allows us to define global einfrastructure services according to the community’s requirements • These services are not dependent upon specific physical technologies – • Nor are they dependent upon specific organizations. • GVM allows the user to define their e-infrastructure services Networks ∙ Services ∙ People www. geant. org 29

Virtualized Environments – A future vision • The Generalized Virtualization Model could support a wide range of global services: • “Testbeds” for early TRL e-infrastrucutre protocol research • Insulated opt-in environments to develop and mature new services • Custom, high performance, global, production virtual networks for science communities (e. g. High Energy Physics, Radio Astronomy, Bio-informatics) • Common lifecycle model and API across data centers into WAN • Simplified NFV and NSC capabilities • Sophisticated complex functional services/applications – e. g. global realtime video services network with end points, dynamic MCUs, capture/streaming services, etc. • Both providing flexible virtual switching facilities for SDN services *AND* simultaneously leveraging those same SDN services as flexible programmable underlayment (!) Networks ∙ Services ∙ People www. geant. org 30

Virtual SDN Fabrics Networks ∙ Services ∙ People www. geant. org 31

Innovation: Virtual Open. Flow Fabrics • Problem: SDN controllers don’t share switch fabrics • This poses problems where a single SDN service does not require an entire switch, or where multiple SDN applications want to control their own fabric/ports. • Poses problems where different researchers require different controllers • Solution 1: “Slicing” • Applications assigned a sub-space or “slice” (typically a subset of VLANs) from of a global network flow space • The proxy controller must filter and authorize all flow specs (e. g. FLow. Space. Fire. Wall) • Solution 2: “Partitions” (sometimes called “port delegation”) • Split the switch into multiple fabric “instances” each with its own context, ports are assigned to an instance • Still slices a single network flow space into port based subspaces, so flowspecs must still be inspected before installation • No port sharing means backbone links can only be assigned to one particular instance • Flowspecs still reference physical port and label information - Instances cannot be relocated/remapped without breaking the flowspecs and thereby breaking the application. Networks ∙ Services ∙ People www. geant. org 32

Controller based Partitioning 0 Flow space A Controller User A Application User B Application 1 2 3 4 5. . . 31 Master Controller Filters user flowspecs Physical flowspecs Flow space B Controller Flow space C Control. Ler User A Controler User b Controler Master Controller must inspect all flow specs to insure each controller only manipulates its own flowspace. Controllers are allocated only a subset flow space of the full physical network flow space, typically (though not always), the same VLAN set in all platforms Networks ∙ Services ∙ People www. geant. org

Openflow Switch Sharing • Flowspace Delegation (Internet 2 FSFW) • A resource manager allocates flow subspaces to users (the whole system shares a single network flowspace) • A single master controller intercepts and inspects all flow specs to insure they fall within the user’s [sub]space • The flowspace is a physical subspace, all control traffic is proxied, inverted proxy trees for become complex • Early innovative thinking to implement slicing in Open. Flow environment • Partitioning / Port Delegation (HPOS on HP 5900) • Separate switching instances in the switch device, delegates specific phy ports to each instance • The Openflow protocol handler on the switch inspects flowspecs to insure subspace conformance by instance • Each instance can have its own controller (good), but. . • Uses physical flowspecs (no migration), does not share ports across multiple Networks ∙ Services ∙ People www. geant. org instances (no grooming) 34

Switch Partitioning OFX Instances with port partioning OFX 0 5 4 1 10 9 OFX 1 2 12 8 OFX 5 15 13 7 6 31 3 Breaks application [physical] flow specs Port Map Port -> OFX 0 n/ a 1 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15. . . 31 2 1 3 5 4 0 Ctlr 1 Ctlr n Ctlr 0 5. . 0 6 5 7 5 Each instance has its own controller, the switch proc only installs flowspecs from that controller 8 1 that match ports assigned to that instance. 9 0 Except for port dimension, the user has full network flow space (no VLAN slicing is needed) User Flow specs are physical port based flowspecs – moving the instance will break the flowspecs, 10. . . 0 n/a Ports cannot be split – the entire port is assigned to an instance – no flow grooming / aggregation 11 12 1 Useful in small but not scalable. Networks ∙ Services ∙ People ways, www. geant. org

Innovation in GTS: Virtual Open. Flow Fabrics • Solution: Virtualized Openflow Switch Fabrics • This is Partitioning with a twist: • Each fabric instance has “virtual Ports” defined by the user: vport(1). . . v. Port(n) • For in-packets, physical Port/outer. VLAN (Label ) 2 -tuple of the frame are remapped to an instance. ID/virtual. Port • The outer tag can be optionally popped on input, or pushed on output • The result: • The user writes flowspecs against the virtual ports, • When combined with fully encapsulating virtual circuits (ala GTS), the user is able to command their own full network flowspace i. e. no more flowspace proxies, or slicing • Because the Virtual Switch is no longer tied to specific physical ports, it can be moved and remapped-> Enables operational migration and grooming – and “save” of the switch state. • The instance can be relocated (vports remapped to different physical ports) without the user application needing to rediscover the topology and rewrite all their flowspecs. Networks ∙ Services ∙ People www. geant. org 36

Virtual Switch Instances Virtual Switch with Virtual Circuit port mapping VOX 0 0 1 2 3 4 100 127 2386 0 1 VOX 1 0 1 2 2 3 Port, label -> VOX, v. Port 0, 100 0, 2 in: pop 100, out: push 100 0, 127 n, 0 in: pop 127, out: push 127 0, 2386 0, 3 in: pop 2386, out: push 2386 1, 0, 1 (no xport header processing) 2, 100 n, 4 in: pop 100, out: push 100 2, 3140 1, 0 in: pop 100, out: push 100 3, 25 0, 0 in: pop 100, out: push 100 3, 1870 n, 2 in: pop 100, out: push 100. . . Networks ∙ Services ∙ People www. geant. org 0 4 5 6 VOX n 1 2 3 4 7 8 9 10 11 12 13. . . 31 Port+Tag remapping allows users to use virtual flowspecs Allows instances to share a physical port Allows transport tagging to be used for VCs, and to be popped before user sees it. Enables migration and grooming, Checkpoint/restart. External RM (GTS)

Physical to Virtual Port Mapping Physical Server Platform VM 3 VM 1 0 1 2 3 VM 2 VM 0 vif 0. . 3 bridging/encapsulation Phy ports 0 1 2 3 nic 0 0 1 2 3 nic 1 0 1 2 3 nic 2 0 1 2 3 nic 3 VM Port mapping: phy. Port/VLAN > VM/vif, Pop tagging (inbound) or push tagging (outbound) Networks ∙ Services ∙ People www. geant. org VOX 1 VOX 3 0 1 2 3 VOX 2 VOX 0 0 1 2 3 SW Port mapping Physical Switch Platform Firmware Port mapping ofport 0. . 3 0 1 2 3 bridging/encapsulation Phy ports 0 1 2 3 VOX Port mapping: phy. Port/VLAN > VOX/vport, Pop tagging(inbound) or push tagging (outbound)

How did we do this? • What does it require? • Modify switch/router software to support virtual Open. Flow instances • Add configuration commands to device mgmt API to define instances, specify port map • Define an “instance” – Each virtual instance has its own context: memory for counters, FIB, fabric ID, routing processor, protocol state for each instance, . . . • Add the port map prolog/epilog microcode into the fast path microcode: • Ingress: Lookup Phy. Port : qtag/label in table; set meta. Data to corresponding Vswitch : Vport, pop outer qtag/label. • Egress: Lookup Vswitch, Vport in table, push outer qtag, queue to Phy. Port • Virtualization sw (GTS) coordinates VC tags/labels, STPs, and Vports to build the port mapping table. • Vendor collaboration – We need vendor buy-in that such virtual routers can be a useful service model for global virtual environments • Corsa Technologies is the first vendor to work with us to deliver this feature • Demo’d in the lab, integration into GTS this summer, available in production Sep 2016 • Line rate- at 100 Gbps (!) Networks ∙ Services ∙ People www. geant. org

Why does this work? • The pipeline design of fastpath packet handling is required to handle very high packet rates. . . 100 G • The pipeline completes the processing of one packet on each cycle Matching rule-> Time -> Add vlookup+pop prolog Each Packet has a fixed latency of n=3 clocks Add vlookup+push epilog 4 3 4 2 3 4 1 2 3 1 2 1 Result is slight increase in latency (prolog+epilog), but the aggregate packet processing rate of 1 pkt/clock is the same !! Networks ∙ Services ∙ People www. geant. org 40

Summary • Virtualization is Production ready and capable • The GEANT Testbeds Service is pushing these capabilities aggressively • The Generalized Virtualization Model is emerging as a community opportunity to build critical mass around virtualization • Virtual Open. Flow Switching poses new capabilities and prospects for provider based SDN services, especially multidomain SDN applications • Planned roll out for September 2016 with GTS v 4. 0 Networks ∙ Services ∙ People www. geant. org 41

The End Networks ∙ Services ∙ People www. geant. org 42