Network Applications in the DataPlane CHAN Mun Choon

Network Applications in the Data-Plane CHAN Mun Choon (in collaboration with Pravein Govindan Kannan, Raj Joshi & Qu Ting) School of Computing National University of Singapore

Motivation for Software Defined Networking (SDN) • Networks: • Notoriously difficult to manage • Evolves very slowly Abstraction is the key to extracting simplicity: easier to write, maintain and reason about the programs that manage and control the network Ref: Scott Shenker, et. al. The Future of Networking, and the Past of Protocols, Open Network Summit, 2011 2

In the Pre-SDN era… CLI/custom script interface --> Device/vendor specific “Closed” Control Plane ASIC APIs (closed) Parser VLAN ACL L 2/MAC Match-Action Pipeline L 3 Deparser 3

Before Open. Flow … • Open Signaling (1990 s) • Make network control functions more open, extensible, and programmable • Separation between hardware and control software • Access to the network hardware via open programmable network interfaces. • Focuses on connection-oriented network services in the early days • IETF RFC 3294 (2003) General Switch Management Protocol (GSMP) • IEEE P 1520 (1998) standards initiative for programmable network interfaces 4

2008 -09: SDN Era (Open. Flow/SDN 1. 0) Open, vendor-agnostic interface: Easier network management Centralized control Code reuse/interoperatibility “Closed” Control Plane ASIC APIs (closed) Parser VLAN ACL L 2/MAC Match-Action Pipeline L 3 Deparser 5

2008 -09: SDN Era (Open. Flow/SDN 1. 0) Open, vendor-agnostic interface: Easier network management Centralized control Code reuse/interoperatibility “Closed” Control Plane It didn’t change the core network functionality! ASIC APIs (closed) (Control plane became a bit more programmable) Parser VLAN ACL L 2/MAC Match-Action Pipeline L 3 Deparser 6

2013 -14: Programmable Switches (SDN 2. 0) Changing interface: C/Python APIs (auto-generated) P 4 Run. Time (led by Google) “Open” Control Plane ASIC APIs (closed/licensed) Programmable Parser ACL L 3 MPLS Programmable Match-Action Pipeline Programmable Deparser 7

Benefits of Dataplane Programmability • Flexible Parsing and matching on non-standard fields: • Faster and easier network evolution: new protocols/headers • Traditionally a new protocol addition takes 4 -5 years!! • Hardware upgrades software upgrades: protection on investment • H/w goes beyond this • Exposing other datapath processing primitives (existing + new) • Accessible and programmable via P 4 (high-level DSL) • Realize new functions (not fully arbitrary) in the datapath • Researchers: Propose an ASIC-level solution for new/existing problems and readily “realize” it in production hardware 8

Extra Dataplane Primitives • Transactional Memory (SRAM) + Stateful ALUs • Stateful operations across multiple packets • Simple computations: add, subtract, approx. multiply/divide • Queuing Telemetry Information • Enqueue/dequeue depth • Time spent in the queue • High-resolution Timestamping • nanosecond-scale time stamps • Ingress/egress MAC timestamps, ingress/egress pipeline timestamps, etc. • Packet cloning/replication • Flexible mirroring or conditional multicasts (at run time) 9

Limitations • High-level constraint: all processing MUST maintain line rate • No “loop” constructs • No floating point computations • Only approximate computations possible • Single Ported, per-stage SRAM memory • Single memory entry can be read/updated in one pkt pass 10

Burst. Radar Practical Real-time Microburst Monitoring for Datacenter Networks (APSys 2018) Raj Joshi 1, Ting Qu 2, Mun Choon Chan 1, Ben Leong 1, Boon Thau Loo 3 1 2 3

Microbursts (µbursts) • Events of intermittent congestion lasting 10’s or 100’s of µs ◦ Common Causes: TCP Incast, Bursty UDP traffic, TCP segment offloading ◦ Intermittent increase in latency variability ◦ Network jitter and Packet loss 12

Detecting & characterizing µbursts is hard • Measurement study from FB’s datacenter • Last for less than 200 µs • Occur unpredictably • Traditional sampling-based techniques • Cannot even detect microbursts • Commercial Solutions • Can detect the occurrence of microbursts • Provide no information about the cause 13

Solution: • Key Insight: Egress Port Queues µbursts are localized to a switch’s egress port queue Switch’s Queuing Engine Key Idea: ◦ We can detect the microburst directly on the switch where it happens 14

Burst. Radar Overview Queuing Telemetry Markbit (metadata) Snapshot Algorithm Egress Ports Courier Pkt Generator Ring Buffer Egress Processing Pipeline Egress Port Queues Egress Deparser 15

Burst. Radar Overview Egress Ports Courier Packet Snapshot Algorithm Courier Pkt Generator Ring Buffer Egress Processing Pipeline Egress Port Queues Egress Deparser Mirror Port Queue 16

Burst. Radar Overview Egress Ports Courier Packet Snapshot Algorithm Courier Pkt Generator Ring Buffer Egress Processing Pipeline Egress Port Queues Egress Deparser Mirror Port Queue 17

Burst. Radar Overview Telemetry Info: - Pkt 5 -tuple - Queuing telemetry data Courier Packet Snapshot Algorithm Courier Pkt Generator Egress Ports Ring Buffer Egress Processing Pipeline Egress Port Queues Mirror Port Queue Egress Deparser Mirror Port 18

Burst. Radar Overview Egress Ports Snapshot Algorithm Courier Pkt Generator Ring Buffer Egress Processing Pipeline Egress Port Queues Egress Deparser Mirror Port 19

Evaluation Setup • Hardware Testbed Burst. Radar Prototype Send/Receive µburst Traffic • ◦ About 550 lines of p 4 code • Generated µburst Traffic Traces • µbursts data for “web” and “cache” traffic [IMC ‘ 17] • Compare Burst. Radar against • In-band Telemetry (INT) dataplane-based solution • “Oracle” Algorithm ground truth (exact pkts in µbursts) 20

Efficiency 5 5% RTT 10 times less packets compared to INT 21

Precise Time-synchronization using Programmable Switching ASICs ACM SOSR 2019 (Best Paper) Pravein Govindan Kannan, Raj Joshi & Mun Choon Chan

Time Synchronization in Data Center NTP PTP Server CPU P NIC H Y milliseconds 10 s of ns to us CPU Switch CPU P HY Queues Server P HY Network Delays & Jitter affect accuracy!! Clock Drifts upto 30µs/sec [HUYGENS ’ 18] P H NIC Y

Portable Switch Architecture High Precision Hardware Timestamps in the Processing Pipeline 24

25

Line-rate traffic along the direction of the response packet. 26

Conclusion • Two applications that exploit data plane programmability to demonstrate the potential of modern programmable ASICs • Burst. Radar: characterize microbursts at multi-gigabit line rates in high-speed datacenter networks. • DPTP: precise time synchronization protocol running in the network data-plane. • Future Work: enable new monitoring frameworks, control paradigms, virtualization strategies and speedup of large scale distributed computations. 27