The Universal Dataplane FD io The Universal Dataplane

The Universal Dataplane

FD. io: The Universal Dataplane • Project at Linux Foundation • Multi-party • Multi-project • Software Dataplane • • • High throughput Low Latency Feature Rich Resource Efficient Bare Metal/VM/Container Multiplatform • Fd. io Scope: • Network IO - NIC/v. NIC <-> cores/threads • Packet Processing – Classify/Transform/Prioritize/Forward/Terminate • Dataplane Management Agents - Control. Plane Bare Metal/VM/Container Dataplane Management Agent Packet Processing Network IO fd. io Foundation 2

Fd. io in the overall stack Application Layer/App Server Orchestration Network Controller Data Plane Services Dataplane Management Agent Packet Processing Network IO Operation System Hardware fd. io Foundation 3

Multiparty: Broad Membership Service Providers Network Vendors Chip Vendors Integrators fd. io Foundation 4

Multiparty: Broad Contribution Qiniu Yandex Universitat Politècnica de Catalunya (UPC) fd. io Foundation 5

Code Activity • In the period since its inception, fd. io has more commits than OVS and DPDK combined, and more contributors than OVS 2016 -02 -11 to 2017 -04 -03 Fd. io OVS DPDK Commits 6283 2395 3289 Contributors 163 146 245 Organizations 42 52 78 Commits Contributors 8000 300 250 200 150 100 50 0 6000 4000 2000 0 80 60 40 20 Commits fd. io OVS Contributors DPDK fd. io Foundation Organizations fd. io OVS DPDK 0 Organizations fd. io OVS DPDK 6

Multiproject: Fd. io Projects Dataplane Management Agent Honeycomb Testing/Support hc 2 vpp CSIT puppet-fdio Packet Processing NSH_SFC CICN ONE odp 4 vpp TLDK VPP Sandbox trex VPP Network IO deb_dpdk rpm_dpdk fd. io Foundation 7

Fd. io Integrations Openstack Neutron Control Plane Integration work done at ODL Plugin Fd. io Plugin GBP app Lispflowmapping app VBD app SFC Netconf/Yang Data Plane LISP Mapping Protocol Netconf/yang Honeycomb REST Fd. io ML 2 Agent VPP fd. io Foundation 8

Vector Packet Processor - VPP Bare Metal/VM/Container Dataplane Management Agent Packet Processing Network IO • Packet Processing Platform: • High performance • Linux User space • Run’s on commodity CPUs: / / • Shipping at volume in server & embedded products since 2004. fd. io Foundation 9

Packet VPP Architecture: Packet Processing 0 1 2 3 … n Vector of n packets dpdk-input vhost-user-input … af-packet-input Input Graph Node ethernet-input ip 6 -rewrite ip 6 -input ip 4 -input ip 6 -lookup ip 4 -lookup ip 6 -local ip 4 -local mpls-input ip 4 -rewrite Graph Node … arp-input Packet Processing Graph

Packet VPP Architecture: Splitting the Vector 0 1 2 3 … n Vector of n packets dpdk-input vhost-user-input … af-packet-input Input Graph Node ethernet-input ip 6 -rewrite ip 6 -input ip 4 -input ip 6 -lookup ip 4 -lookup ip 6 -local ip 4 -local mpls-input ip 4 -rewrite Graph Node … arp-input Packet Processing Graph

Packet VPP Architecture: Plugins 0 1 2 3 Hardware Plugin hw-accel-input dpdk-input vhost-user-input … af-packet-input ethernet-input ip 6 -input ip 4 -input mpls-input ip 6 -lookup ip 4 -lookup ip 6 -local ip 4 -local … arp-input Plugin custom-1 ip 4 -rewrite Packet Processing Graph Node /usr/lib/vpp_plugins/foo. so ip 6 -rewrite n Vector of n packets Input Graph Node Skip sftw nodes where work is done by hardware already … custom-2 custom-3 Plugins are: First class citizens That can: Add graph nodes Add API Rearrange the graph Can be built independently of VPP source tree

VPP Architecture: Programmability Example: Honeycomb Architecture Control Plane Protocol Request Message 900 k request/s Netconf/Restconf/Yang Request Message Linux Hosts Shared Memory … … Agent Request Queue … VPP Response Queue Honeycomb Agent Request Queue … VPP Response Queue Can use C/Java/Python/or Lua Language bindings Async Response Message fd. io Foundation Async Response Message 13

Universal Dataplane: Performance at Scale IPv 6, 24 of 72 cores [Gbps]] 500. 0 450. 0 400. 0 350. 0 300. 0 250. 0 200. 0 150. 0 100. 0 50. 0 IPv 4+ 2 k Whitelist, 36 of 72 cores Phy-VS-Phy Zero-packet-loss Throughput for 12 port 40 GE [Gbps]] Hardware: Cisco UCS C 460 M 4 500 400 300 Intel® C 610 series chipset 200 4 x Intel® Xeon® Processor E 7 -8890 v 3 (18 cores, 2. 5 GHz, 45 MB Cache) 100 0 480 Gbps zero frame loss 12 1 k 100 k 500 k 1 M 2 M routes routes 1518 B IMIX 64 B 1 k 500 k 1 M 2 M 4 M 8 M routes routes 1518 B IMIX 64 B IMIX => 342 Gbps, 1518 B => 462 Gbps 2133 MHz, 512 GB Total 9 x 2 p 40 GE Intel XL 710 18 x 40 GE = 720 GE !! Latency 18 x 7. 7 trillion packets soak test [Mpps] Average latency: <23 usec 300 250. 0 Min Latency: 7… 10 usec 250 200 150. 0 150 Max Latency: 3. 5 ms Headroom Average vector size ~24 -27 100. 0 Max vector size 255 50 50. 0 12 1 k 100 k 500 k 1 M 2 M routes routes 1518 B IMIX 64 B 200 Mpps zero frame loss 0 1 k 500 k 1 M 2 M 4 M 8 M routes routes 64 B => 238 Mpps 1518 B IMIX 64 B Headroom for much more throughput/features NIC/PCI bus is the limit not vpp

Universal Dataplane: Features Hardware Platforms Pure Userspace - X 86, ARM 32/64, Power Raspberry Pi Interfaces DPDK/Netmap/AF_Packet/Tun. Tap Vhost-user - multi-queue, reconnect, Jumbo Frame Support Language Bindings C/Java/Python/Lua Tunnels/Encaps GRE/VXLAN-GPE/LISP-GPE/NSH IPSEC Including HW offload when available MPLS over Ethernet/GRE Deep label stacks supported Routing IPv 4/IPv 6 14+ MPPS, single core Hierarchical FIBs Multimillion FIB entries Source RPF Thousands of VRFs Controlled cross-VRF lookups Multipath – ECMP and Unequal Cost Segment Routing SR MPLS/IPv 6 Including Multicast Switching VLAN Support Single/ Double tag L 2 forwd w/EFP/Bridge. Domain concepts VTR – push/pop/Translate (1: 1, 1: 2, 2: 1, 2: 2) Mac Learning – default limit of 50 k addr Bridging Split-horizon group support/EFP Filtering Proxy Arp termination IRB - BVI Support with Router. Mac assigmt Flooding Input ACLs Interface cross-connect L 2 GRE over IPSec tunnels Security LISP x. TR/RTR L 2 Overlays over LISP and GRE encaps Multitenancy Multihome Map/Resolver Failover Source/Dest control plane support Map-Register/Map-Notify/RLOC-probing Mandatory Input Checks: TTL expiration header checksum L 2 length < IP length ARP resolution/snooping ARP proxy SNAT Ingress Port Range Filtering Per interface whitelists Policy/Security Groups/GBP (Classifier) fd. io Foundation Network Services DHCPv 4 client/proxy DHCPv 6 Proxy MAP/LW 46 – IPv 4 aas Mag. Lev-like Load Identifier Locator Addressing NSH SFC SFF’s & NSH Proxy LLDP BFD Policer Multiple million Classifiers – Arbitrary N-tuple Inband i. OAM Telemetry export infra (raw IPFIX) i. OAM for VXLAN-GPE (NGENA) SRv 6 and i. OAM co-existence i. OAM proxy mode / caching i. OAM probe and responder Monitoring Simple Port Analyzer (SPAN) IP Flow Export (IPFIX) Counters for everything Lawful Intercept 15

Rapid Release Cadence – ~3 months 16 -02 Fd. io launch 16 -06 Release- VPP 16 -06 New Features Enhanced Switching & Routing IPv 6 SR multicast support LISP x. TR support VXLAN over IPv 6 underlay per interface whitelists shared adjacencies in FIB Improves interface support vhost-user – jumbo frames Netmap interface support AF_Packet interface support Improved programmability Python API bindings Enhanced JVPP Java API bindings Enhanced debugging cli Hardware and Software Support for ARM 32 targets Support for Raspberry Pi Support for DPDK 16. 04 16 -09 Release: VPP, Honeycomb, NSH_SFC, ONE 16 -09 New Features 17 -01 Release: VPP, Honeycomb, NSH_SFC, ONE 17 -01 New Features Enhanced LISP support for Hierarchical FIB L 2 overlays Performance Improvements Multitenancy DPDK input and output nodes Multihoming L 2 Path Re-encapsulating Tunnel Routers (RTR) support IPv 4 lookup node Map-Resolver failover algorithm IPSEC New plugins for Softwand HWCrypto Support SNAT HQo. S support Mag. Lev-like Load Simple Port Analyzer (SPAN) Identifier Locator Addressing BFD NSH SFC SFF’s & NSH Proxy IPFIX Improvements Port range ingress filtering L 2 GRE over IPSec tunnels Dynamically ordered subgraphs LLDP LISP Enhancements Source/Dest control plane L 2 over LISP and GRE Map-Register/Map-Notify RLOC-probing ACL Flow Per Packet SNAT – Multithread, Flow Export fd. io Foundation LUA API Bindings 16

New in 17. 04 – Due Apr 19 VPP Userspace Host Stack TCP stack DHCPv 4 relay multi-destination DHCPv 4 option 82 DHCPv 6 relay multi-destination DHPCv 6 relay remote-id ND Proxy Segment Routing v 6 Security Groups Routed interface support L 4 filters with IPv 6 Extension Headers SR policies with weighted SID lists Binding SID SR steering policies SR Local. SIDs Framework to expand local SIDs w/plugins API SNAT CGN: Configurable port allocation CGN: Configurable Address pooling CPE: External interface DHCP support NAT 64, LW 46 Move to CFFI for Python binding Python Packaging improvements CLI over API Improved C/C++ language binding i. OAM UDP Pinger w/path fault isolation IOAM as type 2 metadata in NSH IOAM raw IPFIX collector and analyzer Anycast active server selection IPFIX Collect IPv 6 information Per flow state fd. io Foundation 17

Continuous Quality, Performance, Usability Built into the development process – patch by patch Submit Automated Verify Code Review Build/Unit Testing 120 Tests/Patch System Functional Testing 252 Tests/Patch Build binary packaging for Ubuntu 14. 04 Ubuntu 16. 04 Centos 7 Automated Style Checking Unit test : IPv 6 IPFIX IP Multicast BFD L 2 FIB Classifier L 2 Bridge Domain DHCP MPLS FIB SNAT GRE SPAN IPv 4 VXLAN IPv 4 IRB IPv 4 multi-VRF DHCP – Client and Proxy GRE Overlay Tunnels L 2 BD Ethernet Switching L 2 Cross Connect Ethernet Switching LISP Overlay Tunnels IPv 4 -in-IPv 6 Softwire Tunnels Cop Address Security IPSec IPv 6 Routing – NS/ND, RA, ICMPv 6 u. RPF Security Tap Interface Telemetry – IPFIX and Span VRF Routed Forwarding i. ACL Security – Ingress – IPv 6/Mac IPv 4 Routing Qo. S Policer Metering VLAN Tag Translation VXLAN Overlay Tunnels fd. io Foundation Performance Testing 144 Tests/Patch, 841 Tests L 2 Cross Connect L 2 Bridging IPv 4 Routing IPv 6 Routing IPv 4 Scale – 20 k, 200 k, 2 M FIB Entries IPv 4 Scale - 20 k, 200 k, 2 M FIB Entries VM with vhost-userr PHYS-VPP-VM-VPP-PHYS L 2 Cross Connect/Bridge VXLAN w/L 2 Bridge Domain IPv 4 Routing COP – IPv 4/IPv 6 whiteless i. ACL – ingress IPv 4/IPv 6 ACLs LISP – IPv 4 -o-IPv 6/IPv 6 -o-IPv 4 VXLAN Qo. S Policer L 2 Cross over L 2 Bridging Merge Publish Artifacts Usability Merge-by-merge: apt installable deb packaging yum installable rpm packaging autogenerated code documentation autogenerated cli documentation Per release: autogenerated testing reports report perf improvements Puppet modules Training/Tutorial videos Hands-on-usecase documentation Merge-by-merge packaging feeds Downstream consumer CI pipelines 18 Run on real hardware in fd. io Performance Lab

Universal Dataplane: Infrastructure Bare Metal Cloud/NFVi Container Infra Server VM VM VM Con FD. io Kernel/Hypervisor Kernel fd. io Foundation Con 19

Universal Dataplane: VNFs FD. io based VNFs Server VM VM FD. io Con FD. io Kernel/Hypervisor fd. io Foundation 20

Universal Dataplane: Embedded Device Smart. Nic Device Server Kernel/Hypervisor FD. io Kernel/Hypervisor Hw Accel fd. io Foundation Smart. Nic FD. io Hw Accel 21

Universal Dataplane: CPE Example Physical CPE v. CPE in a VM v. CPE in a Container Device Server VM VM FD. io Con FD. io Kernel/Hypervisor Hw Accel fd. io Foundation 22

Opportunities to Contribute • • • Firewall IDS Hardware Accelerators Integration with Open. Cache Control plane – support your favorite SDN Protocol Agent Spanning Tree DPI Test tools Cloud Foundry Integration Container Integration Packaging Testing fd. io Foundation We invite you to Participate in fd. io • Get the Code, Build the Code, Run the Code • Try the vpp user demo • Install vpp from binary packages (yum/apt) • Install Honeycomb from binary packages • Read/Watch the Tutorials • Join the Mailing Lists • Join the IRC Channels • Explore the wiki • Join fd. io as a member 23