An open source user space fast path TCPIP

An open source user space fast path TCP/IP stack and more…

Enter Open. Fast. Path! A TCP/IP stack that § lives in user space § is optimized for scalability, throughput and latency § uses Open Data Plane (ODP) to access network hardware § works with Data Plane Development Kit (DPDK) § runs on ARM, x 86, MIPS, PPC hardware § runs natively, in a guest or in the host platform A modular protocols library § for termination and forwarding usecases § providing a framework extensible with new protocols § augments existing or implements missing HW acceleration The Open. Fast. Path project § is a true open source project § uses well known open source components § open for all to participate – no lock-in to HW or SW § Nokia, ARM and Enea key contributors

Features implemented Fast path protocols processing: § Layer 4: UDP, TCP, ICMP protocols § Layer 3 § ARP/NDP § IPv 4 and IPv 6 forwarding and routing § IPv 4 fragmentation and reassembly § VRF for IPv 4 § IGMP and multicast § Layer 2: Ethernet, VLAN § GRE and VXLAN Tunneling Routes and MACs are in sync with Linux Command line interface § Packet dumping and other debugging § Statistics, ARP, routes, and interface printing § Configuration of routes and interfaces with VRF support IP and ICMP implementations PASS Ixia conformance tests IP and UDP implementations have been optimized for performance -> linear scalability § TCP impl. optimization is in progress Integrated with NGINX webserver Integration with Linux IP stack through TAP Binary compatibility with Linux applications interface no recompilation is needed to use OFP.

Open. Fast. Path System View User Termination or Forwarding Host OS (Linux) TAP Slow path Netlink Route tables User Conf Code Init API Socket Egress API Socket Hook API Open. Fast. Path (OFP) pkt_cnt = odp_pktio_recv(pktio, pkt_tbl, OFP_PKT_BURST_SIZE); or buf = odp_schedule(&queue, ODP_SCHED_WAIT); PKTIO Interface Management ODP SW ODP/DPDK FW/HW HW / NICs Ingress API User/Default Dispatcher DPDK Application OFP Linux ODP/DPDK HW Ctrl Packets

Open. Fast. Path multicore System View TAP Slow path Netlink Route tables User Conf Code User Termination or Forwarding B Socket callback /Hook API User Termination or Forwarding X Socket callback /Hook API Open. Fast. Path (OFP) (SMP multicore library) PKTIO Ingress API PKTIO Dispatcher 2 Dispatcher 1 Single thread context PKTIO …. Ingress API Dispatcher N ODP SW DPDK Core 0 …. Init API Host OS (Linux) User Termination or Forwarding A Core 1 Core 2 ODP/DPDK FW/HW NICs …. Core N Application OFP Linux ODP/DPDK HW

Ingress Packet Processing IP, UDP, TCP, … classified by HW IPv 4/v 6 VXLAN Ingress API Ethernet VLAN IPv 4 Reassembly Loopback to VXLAN IPv 4/v 6 local hook API UDP input IPv 4/v 6 forward hook API TCP input Transport(L 4) classifier IPv 4 GRE BSD Socket API Callback API GRE hook API ICMP NDP Fallback to slowpath for unknown traffic ARP IPv 4/v 6 routing IGMP Update MAC table IPv 4/v 6 output Send ARP request User API L 4 L 3 L 2 Pre-classified Packets Information

Egress Packet Processing UDP output TCP output BSD Socket or Egress API User API L 4 L 3 L 2 Pre-classified Packets Information IPv 6 output IPv 4 Fragmentation IPv 4 output ICMP error IPv 4 GRE tunneling VXLAN Ethernet VLAN

Optimized Open. Fast. Path socket APIs New zero-copy APIs optimized for single thread run-to-completion environments § UDP § Send: Optimized send function with a packet container (packet + meta-data) § Receive: A function callback can be registered to read on a socket. Receives a packet container and socket handle § TCP § Accept event: A function callback can be registered for TCP accept event. Receives socket handle. § Receive: A function callback can be registered to read on socket. Receives a packet container and a socket handle Standard BSD Socket interface § For compatibility with legacy Linux applications

OFP Performance on ARM IP Forwarding on Cavium Thunder. X 100 90 90 80 80 70 60 ODP-OFP UDPECHO Native UDPECHO 50 40 30 Percent of 10 Gbit line rate Percent of 10 GBit line rate UDP ECHO on Cavium Thunder. X 70 60 50 40 30 20 20 10 10 0 1 2 4 6 8 10 12 14 16 18 Number of Cores 20 22 24 26 28 30 Native IPFWD ODP-OFP IPFWD 0 1 2 4 Number of Cores 6 8

OFP Performance on x 86 IP forwarding application in user space - 256 routes, 4 x 10 Gbps, 64 Byte packets 60 Kernel 50 OFP Mpps 40 30 x 20 20 10 0 1 2 3 4 5 6 7 Physical Cores 8 9 10 11 12

NGINX – OFP TCP/IP integration Standard TCP/IP NGINX worker process OFP TCP/IP NGINX worker process OFP BSD Socket API Context switch OFP TCP/IP stack library PKT IO API Locks Kernel TCP/IP NIC Hardware Packet flows • • Avoid context switches Avoid locks Streamlined packet path Better scalability, throughput and latency ODP/DPDK RX/TX queues NIC Hardware RX/TX queues

Binary compatibility with Linux applications Standard TCP/IP Application binary Overload Socket API with OFP Socket API libofp_netwrap_crt Application binary Linux TCP/IP OFP TCP/IP Libraries to LD_PRELOAD Open. Fast. Path library ODP/DPDK library libofp . /ofp_netwrap. sh. /binary Open. Fast. Path library ODP/DPDK Library Start and configure OFP TCP/IP stack NIC Hardware libofp_netwrap_proc Packet flows NIC Hardware

What’s next? - Get involved! Download the source code from: https: //github. com/Open. Fast. Path/ofp Check us out at www. openfastpath. org to get more information about the project Subscribe to Mailing-list: http: //www. openfastpath. org/mailman/listinfo Ping us on our freenode chat: #Open. Fast. Path

Thank You For additional information, please visit www. openfastpath. org