IX A Protected Dataplane Operating System for High

IX: A Protected Dataplane Operating System for High Throughput and Low Latency By: Adam Belay, George Prekas, Ana Klimovic, Samuel Grossman, and Christos Kozyrakis, Edouard Bugnion, Presented by: Xianghan Pei EECS 582 – W 16 1

Outline • Overview • Motivation • IX Design • IX Implementation • Evaluation EECS 582 – W 16 2

Overview • OSDI 2014 Best Paper • Extension work from Dune (an extension to Linux that provides safe and efficient access to kernel-only CPU features) • Some slides borrow from IX conference slides EECS 582 – W 16 3

Challenges for Datacenter Application • Microsecond Tail Latency • High Packet Rates • Protection • Resource Efficiency EECS 582 – W 16 4

Motivation • HW is fast, but SW is a Bottleneck 64 -byte TCP Echo EECS 582 – W 16 5

Motivation • Why SW Slow? • HW and Workload is changing • Dense Multicore +10 Gb. E • Scale out workloads • Berkeley sockets, designed for CPU time sharing • Complex Interface & Code Paths Convoluted by Interrupts and Scheduling • Packet inter-arrival times being many times higher than the latency of interrupts and system call EECS 582 – W 16 6

Motivation • IX Closes the SW Performance Gap • Protection and direct HW access through virtualization • Execution model for low latency and high throughput 64 -byte TCP Echo EECS 582 – W 16 7

IX Design • Separation and protection of control and data plane • Control plane responses for resource configuration, provisioning, ect. • Data plane runs for networking stack and application logic • Run to completion with adaptive batching • Data and instruction cache locality • Native, zero-copy API with explicit flow control • Flow consistent, synchronization-free processing EECS 582 – W 16 8

IX implementation • Separation of Control and Data Plane EECS 582 – W 16 9

IX implementation • Run to Completion with Adaptive Batching EECS 582 – W 16 10

IX implementation • Dataplane Details • Different Memory Management • Large blocks, allow internal memory fragmentation, turn off swappable memory • Hierarchical timing wheel implementation, such as TCP retransmissions timeout • Redesign API EECS 582 – W 16 11

IX implementation • Dataplane API and Operation EECS 582 – W 16 12

IX implementation • Multi-core scalability • Elastic threads operate in a synchronization and coherence free manner • API commute • Flow-consistent hashing at the NICs • Small number of shared structures • Security model • The malicious application can’t corrupt the networking stack or other applications EECS 582 – W 16 13

Evaluation • Comparison IX to Linux and m. TCP • TCP microbenchmarks and Memcached EECS 582 – W 16 14

Evaluation • Net. PIPE performance EECS 582 – W 16 15

Evaluation • Multicore Scalability for Short Connections EECS 582 – W 16 16

Evaluation • n round-trips per connection EECS 582 – W 16 17

Evaluation • Different message sizes s (n=1) EECS 582 – W 16 18

Evaluation • Connection Scalability EECS 582 – W 16 19

Evaluation • Memcached over TCP EECS 582 – W 16 20

Discussion • Why IX fast? • Subtleties of adaptive batching • Limitation EECS 582 – W 16 21

Conclusion • A protected dataplane OS for datacenter applications with an event-driven model and demanding connection scalability requirements • Efficient access to HW, without sacrificing security, through virtualization • High throughput and low latency enabled by a dataplane execution model EECS 582 – W 16 22

Conventional Wisdom • Bypass the kernel • Move TCP to user-space (Onload, m. TCP, Sandstorm) • Move TCP to hardware (TOE) • Avoid the connection scalability bottleneck • Use datagrams instead of connections (DIY congestion management) • Use proxies at the expense of latency • Replace classic Ethernet • Use a lossless fabric (Infiniband) • Offload memory access (r. DMA) EECS 582 – W 16 23