UDPI Project Introduction Hongjun Ni Xiang Wang UDPI

  • Slides: 25
Download presentation
UDPI Project Introduction Hongjun Ni, Xiang Wang UDPI Community Sept 1, 2019

UDPI Project Introduction Hongjun Ni, Xiang Wang UDPI Community Sept 1, 2019

Acknowledgement DPDK Community VPP Community John Di. Giglio @ Intel Jerome Tollet @ Cisco

Acknowledgement DPDK Community VPP Community John Di. Giglio @ Intel Jerome Tollet @ Cisco Ray Kinsella @ Intel Ed Warnicke @ Cisco Jokul Li @ Intel Dave Barach @ Cisco Yu Liu @ Intel Damjan Marion @ Cisco Ping Yu @ Intel Andreas Schultz @ Travelping Zhihong Wang @ Intel Mathias Gumz @ Travelping Qi Zhang @ Intel *Other names and brands may be claimed as the property of others.

Agenda • UDPI Project Overview and Scope • Flow-based Reference Solution • HW Offloading

Agenda • UDPI Project Overview and Scope • Flow-based Reference Solution • HW Offloading and SW Flow Processing • TCP Segments Reassembly • Hyperscan Block and Stream Mode • Identifying Layer 7 Applications for HTTPS • Key Takeaway *Other names and brands may be claimed as the property of others.

UDPI Project Overview UDPI (Universal Deep Packet Inspection): A new project in FD. io

UDPI Project Overview UDPI (Universal Deep Packet Inspection): A new project in FD. io https: //wiki. fd. io/view/UDPI Provides a reference framework to build a high performance solution for DPI. Integrates with the general purpose FD. io VPP stack. Leverages industry regex matching library (Hyperscan) to provide rich features. Can be used in IPS/IDS, Web Firewall, Layer 7 LB and similar applications. Initial code contributions are from Intel and Travelping (OEM for DT). 4

UDPI Founders and Committers Intel leads this project. 13 organizations joined and 20 initial

UDPI Founders and Committers Intel leads this project. 13 organizations joined and 20 initial committers. 5

UDPI Architecture 1. Flow Classification 2. Application Detection 3. Application-based Action 6

UDPI Architecture 1. Flow Classification 2. Application Detection 3. Application-based Action 6

UDPI Project Scope • Flow Classification & Expiration o HW flow offloading leveraging rte_flow

UDPI Project Scope • Flow Classification & Expiration o HW flow offloading leveraging rte_flow on DPDK o SW flow classification • Application Detection o Leverage Hyperscan Stream Mode o Reassembly TCP segments on the fly • Application-based Actions o HQos, Rate Limiting, Policy Routing, SDWAN, etc. • Support Hundreds of Protocols & Applications o TLS/HTTPS, HTTP, DNS, QUIC, etc. 7

Intel E 810 NIC • More HW Resources • Flexible Pipeline (FXP) • Advanced

Intel E 810 NIC • More HW Resources • Flexible Pipeline (FXP) • Advanced Tx HQo. S

Flow Configuration • Configures static flows with 5 -tuple and BD/VRF-aware. • Creates SW

Flow Configuration • Configures static flows with 5 -tuple and BD/VRF-aware. • Creates SW flows dynamically. • Supports both ipv 4 and ipv 6 flows. • Firstly try offloading flow to NIC leveraging rte_flow mechanism. • If failed, then creates SW flow mappings.

Bi-directional Flow Handling • Each flow creates two HW or SW flow mappings, i.

Bi-directional Flow Handling • Each flow creates two HW or SW flow mappings, i. e. bi-directional. • Both flow mappings will be mapped to the same flow. • Need to identify flow directions when parsing packets

HW Flow Offloading • Leverages rte_flow mechanism from DPDK: • • • Supports ipv

HW Flow Offloading • Leverages rte_flow mechanism from DPDK: • • • Supports ipv 4 -n-tuple, ipv 6 -n-tuple, ipv 4 -vxlan, ipv 6 -vxlan, etc. Supports rte_flow_item, rte_flow_action, etc. Using rte_flow_create to create a HW flow. If failed, then create a SW flow session. HW flow matching: • • • If one packet matches a flow, then flow ID is marked by HW. DPDK drivers sets flow ID to metadata of packet descriptor. Subsequent features could retrieve flow ID from packet descriptor.

SW Flow Processing • Leverages RSS mechanism. • Supports ipv 4 -n-tuple, ipv 6

SW Flow Processing • Leverages RSS mechanism. • Supports ipv 4 -n-tuple, ipv 6 -n-tuple, etc. • Calculate hash value and look up SW flow table. • • If matched, set flow ID to metadata of packet descriptor. Subsequent features could retrieve flow ID from packet descriptor. • If failed to match, then creates a SW flow entry dynamically.

NICs & Flows One flow is mapping to many NICs. • Same flow from

NICs & Flows One flow is mapping to many NICs. • Same flow from all NICs are sent to same core to handle. One NIC contains many flows. • Different flows from one NICs are sent to different core to handle.

BD/VRF Aware If having same 5 -tuple packet matching on 2 different VLANs or

BD/VRF Aware If having same 5 -tuple packet matching on 2 different VLANs or even on 2 different interfaces, they should be treated as different flows.

DPI Flow Processing • When HW flow offloading matched, packets will be redirected to

DPI Flow Processing • When HW flow offloading matched, packets will be redirected to DPI plugin with dpi flow id in packet descriptor. • If not hit, packets will be bypassed to DPI plugin from ip-input, and then lookup SW flow mapping table entry. • If failed to match, then creates a SW flow table entry dynamically.

TCP Connection Track • Tracks TCP three-way handshakes. • Identify TCP traffic direction. •

TCP Connection Track • Tracks TCP three-way handshakes. • Identify TCP traffic direction. • Tracks TCP send sequence and ack sequence for TCP segments reassembly. • TCP FIN/ACK will expire the flow.

App. ID Database typedef struct dpi_app_match_rule_ { char *host; char *pattern; char *app_name; u

App. ID Database typedef struct dpi_app_match_rule_ { char *host; char *pattern; char *app_name; u 32 app_id; } dpi_app_match_rule; typedef enum { DPI_APP_CISCO = 1, … DPI_APP_INTEL = 7, … } dpi_application_id_t; dpi_app_match_rules[] = { {"www. cisco. com", NULL, "Cisco", DPI_APP_CISCO} , … , {"www. intel. com", NULL, "Intel", DPI_APP_INTEL} , … }

How Hyperscan Works Rulesets User Defined #1 #2 Runtime Init (Once) Mode (Block, Stream)

How Hyperscan Works Rulesets User Defined #1 #2 Runtime Init (Once) Mode (Block, Stream) Predefined Flags Allocate Scratch hs_alloc_scratch /foo. *bar/s /[a-f]{6, 12}/i Hyperscan Compiler hs_compile_multi Match callback Match forcallback ID callback n Match for ID n at Offset i for Match at Offset i ID n at Offset i Scratch space hs_scratch_t … #3 /^GETs. *HTTP/m Phase 1: Compilation at system initialization phase Bytecode (database) Hyperscan Runtime Scan (Block mode) hs_scan Phase 2: Data Searching & Match Data Blocks

Block Mode and TCP Segments Reassembly • • Block mode can scan rules only

Block Mode and TCP Segments Reassembly • • Block mode can scan rules only in a complete payload. If defining a rule "abcde", then for Block Mode, "abcde“ should be in a complete PDU payload. • Requirements for TCP Segments process: o o o • • Reassembly TCP segments first to a complete PDU payload. Scan PDU payload through Block mode. Fragment TCP segments again. This degrades the performance. Most Open Source DPI projects using Hyperscan performs in this way.

Stream Mode and TCP Segments Reassembly • • Stream mode can scan rules straddling

Stream Mode and TCP Segments Reassembly • • Stream mode can scan rules straddling into different TCP segments. If defining a rule "abcde", then for Stream Mode, then "abc" can be reside in packet 1, and "de" can be in packet 2. • Requirements for TCP Segments process: o Reassemble TCP segments reassembly on the fly. o Can handle out-of-order tcp segments. o Can handle overlapping segments. • • This helps to improve the performance. UDPI project is implemented in this way.

Example: Identifying Layer 7 Applications for HTTPS • Identify SSL/TLS certificate message and subsequent

Example: Identifying Layer 7 Applications for HTTPS • Identify SSL/TLS certificate message and subsequent segments. • Scan SSL/TLS certificate message through hyperscan, and get application ID if hit. • App ID will be stored, and all subsequent packets will leverage it directly. • If maximum packets for this flow are scanned and not matched, the detection will end up.

Test Stream 1. TCP three-way handshake 2. TLSv 1. 2 handshake 3. TLSv 1.

Test Stream 1. TCP three-way handshake 2. TLSv 1. 2 handshake 3. TLSv 1. 2 Application Data 22

Test Results 1. Packet 1 to 8 failed to detect 2. Packet 9 detected

Test Results 1. Packet 1 to 8 failed to detect 2. Packet 9 detected successfully 23

Key Takeaway • Provides a flow-based framework for advanced packet processing. • Flexible HW

Key Takeaway • Provides a flow-based framework for advanced packet processing. • Flexible HW flow offloading and SW flow processing. • Supports TCP segments reassembly on the fly. • Leverages Hyperscan Stream Mode to achieve high performance. 24

Welcome to join us! https: //wiki. fd. io/view/UDPI 25

Welcome to join us! https: //wiki. fd. io/view/UDPI 25