Packet Prefiltering for Network Intrusion Detection Ioannis Sourdis

Packet Pre-filtering for Network Intrusion Detection Ioannis Sourdis, Vasilis Dimopoulos, Dionisios Pnevmatikatos and Stamatis Vassiliadis CE, TU Delft, the Netherlands ECE, TU Crete, Greece ICS-FORTH, Greece 5. 12. 2006 Technical University of Crete

Introduction Intrusion Detection System (IDS): n Packet Classification (Header Matching) n Payload Scan (Pattern Matching) Related work on Pattern matching & Packet Classification Is that enough for the core of a high-speed IDS? ? ? TU Crete 5. 12. 2006 © Ioannis Sourdis 2

Motivation IDS rulesets contain thousands of rules (SNORT IDS: 3, 000 -4, 000) IDS rule syntax becomes more complicated: payload H E A D E R P 1 depth offset Significant Cost Performance limitation TU Crete The high-speed Pattern matching techniques are required but are not adequate to accommodate the new IDS syntax features 5. 12. 2006 © Ioannis Sourdis H E A D E R P 1 min Distance P 2 within 3

What do we need? Need for different levels of processing packets 1 st Level: use a simplified version of the rules n n It’s a Fact: Not ALL rules will match every single packet Exclude the majority of the rules 2 nd Level: use advanced attack descriptions for the remaining rules TU Crete Reduce the cost and maintain high performance 5. 12. 2006 © Ioannis Sourdis 4

How to accomplish this? We are looking for a fast-inexpensive way to exclude/ “filter-out” for each incoming packet most of the IDS rules + point out a small subset of possibly matching rules? Partially match each rule of the set Possibly matching rules Considering only header info per rule groups of hundreds of rules. Many rules may have the same header description! Suggestion: header + a small portion of payload pattern in the filter IDS ruleset Rule #1 Rule #2 Rule #3. . . . Rule #N+1. . Rule #M TU Crete 5. 12. 2006 © Ioannis Sourdis 5

Packet Pre-filtering How: Put a small FIRST part of each rule in the pre-filtering n n Match (if any) the header description of the rule (Source and Destination Address, protocol) Match a prefix of the first payload pattern of the rule (constant number of bytes) IF for a packet, this part of the rule matches the rule needs to be fully matched ELSE the rule is excluded/filtered-out TU Crete 5. 12. 2006 © Ioannis Sourdis 6

Example IDS Ruleset: 1. Rule(1): header: H(1) payload: P(1)+[pattern_suffix] 2. Rule(2): header: H(1) payload: P(2)+[pattern_suffix] H(1) P(1) … P(2) … P(3) … P(N) … N. Rule(N): header: H(1) payload: P(N)+[pattern_suffix] … P(1)…P(N): prefixes of payload patterns (i. e. 4 -10 characters long) Candidate: Rule(1) Rule(2) Rule(3) ……. . Rule(N) The pre-filtering won’t be efficient if incoming packets look like this: TU Crete 5. 12. 2006 © Ioannis Sourdis 7

Example IDS Ruleset: 1. Rule(1): header: H(1) payload: P(1)+[pattern_suffix] 2. Rule(2): header: H(1) payload: P(2)+[pattern_suffix] … N. Rule(N): header: H(1) payload: P(N)+[pattern_suffix] P(1)…P(N): prefixes of payload patterns (i. e. 4 -10 characters long) The pre-filtering will be efficient if incoming packets look like this: H(1) P(1) Candidate: … Rule(1) P(2) … Rule(2) The less rules activated per single packet the most efficient the -filtering pre TU Crete 5. 12. 2006 © Ioannis Sourdis 8

HW Implementation Field extractor: n Header matching: Extract header and payload Payload Matching: n n n Prefix of the 1 st payload pattern (i. e. 2 -10 chars) Implementation DCAM Regular Expressions may also be used (matching again Reg. Expr. prefixes) TU Crete 5. 12. 2006 © Ioannis Sourdis n Src/Dest IP+Port, Protocol n Implementation simple comparators Bitmask, each bit corresponds to a rule Priority Encoder: Pipelined, encodes/outputs every SET bit of the bitmask. 9

2 nd Level: Complete Match Engine HW or SW? SW assign in multiple threads to match the candidate rules Network processor multiple processing engines. . . Fully customized HW system We propose (not excluding other solutions): Reconfigurable Hardware: HW performance, flexibility, reconfiguration to update the ruleset TU Crete 5. 12. 2006 © Ioannis Sourdis 10

Reconfigurable IDS core Organization TU Crete Pre-filtering points out the rules to be fully matched Specialized Engines: For each candidate rule: n A PE is reserved n A firmware is transferred to the PE n PE released Eo. P or rule mismatch Coprocessors (Static patterns & Regular expression matching) perform payload scan PEs select the coprocessor info and decide whether a rule matches or not 5. 12. 2006 © Ioannis Sourdis 11

What if (Candidate rules > PEs) ? What does (Candidate rules>PEs) mean? A single packet partially matches more than x rules (e. g. x=32 or 64) Can such a packet be a normal packet? What happens when (Candidate rules>PEs)? In order to guarantee performance, the packet is reported, Admin policies determine the next step (i. e. drop) TU Crete 5. 12. 2006 © Ioannis Sourdis 12

Experimental Results Defcon 11 traces 9 trace files ~10 millions packets 4. 6 million packets have payload length: n Mean 698 bytes n Max 1460 bytes SNORT v 2. 4 3, 191 rules n 2, 271 rules with payload description n 920 only header TU Crete 5. 12. 2006 © Ioannis Sourdis 13

Simulation Results Pre-Filtering setup: n Header matching Scr/dest IP+Port, Protocol n Payload Pattern match 2 -10 chars prefix match For prefix>2 chars: Average Candidate rules per packet=[1, 3] per trace Overall average: 1. 8 rules per packet Only header match ~45 rules per packet TU Crete 5. 12. 2006 © Ioannis Sourdis 14

Simulation Results Payload prefix match= 2 chars: max 63 candidate rules per packets Payload prefix match>=4 chars: max 32 candidate rules per packets What does this mean: Max number of rules for further processing 1% or 32 out of 3, 200 rules The Max degree of parallelism needed (processing engines, threads etc. ) TU Crete 5. 12. 2006 © Ioannis Sourdis 15

Implementation Results Datapath 8 bits/cycle: Virtex 2: 2. 7 Gbps Virtex 4: 4 Gbps Area 11 K slices (medium -small FPGA) Datapath 32 bits/cycle: Virtex 2: 9. 7 Gbps Virtex 4: 14 Gbps Area 15 K slices (medium -small FPGA) Priority encoder takes most of the area TU Crete 5. 12. 2006 © Ioannis Sourdis 16

Conclusions Packet Pre-filtering n Points out a small rule subset per incoming packet for further processing n Offloads an IDS core n Allows to utilize more sophisticated attack descriptions on the 2 nd phase of the system (for a few rules) We include in the filter (per rule) n Header matching (Source/Destination Address & Protocol) n 4 -8 characters payload pattern match TU Crete 5. 12. 2006 © Ioannis Sourdis 17

Conclusions Performance: 99% of the IDS rules per incoming packet do not need further processing (in Defcon 11 traces), without loosing detection precision. Throughput: 2. 7 -10 Gbps (Virtex 2) or 4 -14 Gbps (Virtex 4), 8 or 32 bits datapaths Requirements: Lightweight system, requires 10 -15 K slides, can fit in a medium-sized FPGA Can be integrated in both HW or SW based systems TU Crete 5. 12. 2006 © Ioannis Sourdis 18

Questions? TU Crete 5. 12. 2006 © Ioannis Sourdis 19