Achieve Low Latency NFV with Openstack Yunhong Jiang

Achieve Low Latency NFV with Openstack* Yunhong Jiang Yunhong. Jiang@intel. com *Other names and brands may be claimed as the property of others.

Agenda NFV and network latency Why network latency on NFV How to achieve low latency NFV Setups and configurations *Other names and brands may be claimed as the property of others.

NFV and Network Latency

NFV Drivers ØWhat is NFV Classical Network Appliance Approach Independent Software Vendors Running VNFs (Virtualized Network Functions) on high volume servers ØBenefits Flexibility Lower Cap. Ex and Op. Ex Network functions Virtualisation Approach Message Router DPI CDN Firewall Session Border WAN Controller Acceleration Carrier Grade NAT Tester/Qo. E monitor Orchestrated, automatic & remote install. Time-to-market ØRequirements Performance Reliability Security Manageability Interoperability SGSN/GGSN PE Router BRAS Radio/Fixed Access Network Nodes High volume standard servers • Fragmented non-commodity hardware. • Physical install per appliance per site. • Hardware development large barrier to entry for new vendors, constraining innovation & competition. High volume standard storage High volume Ethernet switches

Low network latency is important Voice quality for VOIP service - 150 ms (tolerable by ears) LTE Cloud RAN (Radio Access Network): 1 ms - With 5 G, the latencies need to be 1/10 th of that A VNF that receives 10 Gbit/s network traffic of 256 byte packets will receive 3900 packets in 1 ms Profit or loss in algorithmic trading requires microsecond latencies BUT NFV MAY INCREASE LATENCY!

Why network latency on NFV

Network latency on NFV – An example v. CPU scheduling VNF VCPU Network Function Open v. Switch Daemon Open Stack* Service v. CPU Apps Kernel v. HW Interrupt Latency v. CPU Preemption Virt. IO NICs Threads/ISR Soft IRQ Latency Linux* KVM OVS Kernel Module v. Host Kernel Network Stack HW Resource Contention CPU core Multiple Bridges Virtualization Overhead HW Memory NIC *Other names and brands may be claimed as the property of others. L 3 Cache

Root causes of Latency on NFV Resource contention CPU/Cache/TLB/Memory/IO v. CPU/apps/services/kernel service Virtualization overhead CPU/IO virtualization Network virtualization 8

How to achieve low latency NFV

How to achieve low latency NFV Resource contention Reduce the contention cost Priority VNF v. CPU for the contention Exclusively resource assignment Virtualization overhead Advanced virtualization features and hypervisor DPDK OVS for network virtualization or SR-IOV

A systematic solution for low latency NFV v. CPU • Advanced hardware platform Apps Open Stack* Service OVS DPDK PMD Thread • Real-time Hypervisor VNF 1 v. CPU VNF 2 v. CPU v. Host • DPDK OVS • Open. Stack* RT Linux KVM Threads/ISR CPU core HW NIC *Other names and brands may be claimed as the property of others. Memory L 3 Cache NIC

Advanced hardware platform VT-d and SR-IOV I/O device reservation PI (Posted Interrupt) Reducing interrupt virtualization overhead CAT (Cache Allocation Technology) Cache reservation ……

Real-time hypervisor/Linux* Real-time scheduler Full preemption support All activities (ISR, soft IRQ etc) on thread context CPU isolation support …… *Other names and brands may be claimed as the property of others.

KVM 4 NFV Project An OPNFV project https: //wiki. opnfv. org/nfv-kvm Based on upstream real-time Linux* Included in Brahmaputra release Will be integrated into OPNFV test framework in C release Active contributors from OPNFV community Intel, Nokia, Wind River …

DPDK OVS for low latency NFV Data Plane Development Kit (DPDK) Improves packet processing on Intel® architecture Bypasses Linux* network stack and maps hardware registers to user space. PMD (Polling Model Driver) thread Queue & buffer management, packet flow classification, memory management *Other names and brands may be claimed as the property of others.

DPDK OVS for low latency NFV Accelerate OVS data plane with DPDK PMD thread avoids the interrupt/soft DPDK PMD Thread VNF Open v. Switch Daemon v. Host v. HW IRQ latency Virt. IO NIC Linux* Queue & buffer management KVM v. Host OVS Kernel Module User space v. Host HW *Other names and brands may be claimed as the property of others. NIC

Open. Stack*- Nova Resource tracking on compute node Avoid VM resource contention VM creation Reserved resources allocation v. CPU thread scheduler priority Guest NUMA support Guest huge page support *Other names and brands may be claimed as the property of others.

2. /: Open. Stack*- Nuetron v. NIC port with SR-IOV NIC v. NIC port with DPDK OVS and v. Host-user *Other names and brands may be claimed as the property of others.

setups and configurations *Other names and brands may be claimed as the property of others.

Host configuration RT Linux* setup Static kernel boot parameter (isolcpus/rcu_nocbs/nohz_full) Runtime configuration (Interrupt affinity/watchdog/RCU CB ……) DPDK OVS Upgrade to DPDK OVS service DPDK PMD thread affinity *Other names and brands may be claimed as the property of others.

Configure Open. Stack* Compute node resource reservation vcpu_pin_set/cpu_allocation_ratio/ram_allocation_ratio pci_passthrough_whitelist VM flavor hw: cpu_realtime_mask/hw: cpu_thread_policy/ hw: cpu_realtime_mask/hw: cpu_realtime/hw: mem_page_size/pci: alias

Configure Open. Stack* Network node for DPDK OVS * datapath_type/ vhostuser_socket_dir Network node for SR-IOV supported_pci_vendor_devs/ physical_device_mappings

Call To Action • Document • Automatic host deployment • Automatic latency test *Other names and brands may be claimed as the property of others.

Configuration Reference https: //wiki. opnfv. org/display/kvm/Nfv-kvm-tuning https: //github. com/openstack/networking-ovs-dpdk https: //github. com/openvswitch/ovs/blob/master/INSTALL. DPDK. md#performanc e-tuning http: //docs. openstack. org/liberty/networking-guide/adv_config_sriov. html *Other names and brands may be claimed as the property of others.

Legal notices and disclaimers Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at intel. com, or from the OEM or retailer. No computer system can be absolutely secure. Tests document performance of components on a particular test, in specific systems. Differences in hardware, software, or configuration will affect actual performance. Consult other sources of information to evaluate performance as you consider your purchase. For more complete information about performance and benchmark results, visit http: //www. intel. com/performance. Intel, the Intel logo and Intel architecture are trademarks of Intel Corporation in the U. S. and/or other countries. *Other names and brands may be claimed as the property of others. © 2016 Intel Corporation.