Adaptive Intrusion Detection and Mitigation Systems for Wi

Adaptive Intrusion Detection and Mitigation Systems for Wi. MAX Networks Yan Chen Northwestern Lab for Internet and Security Technology (LIST) Dept. of Computer Science Northwestern University http: //list. cs. northwestern. edu Motorola Liaisons Gregory W. Cox, Z. Judy Fu, Philip R. Roberts Motorola Labs

The Spread of Sapphire/Slammer Worms

Current Intrusion Detection Systems (IDS) • Mostly host-based – not scalable to high-speed networks • Mostly simple signature-based – Can’t deal with unknown attacks, polymorphic worms • Statistical detection – Unscalable for flow-level detection – Overall traffic based: inaccurate, high false positives • Cannot differentiate malicious events with unintentional anomalies

Adaptive Intrusion Detection and Mitigation for Wi. MAX Networks (WAIDM) User s 802. 16 BS User s Internet Users Inter net Switch/ BS controller scan port WAIDM system • Attached to a switch connecting BS as a black box • Enable the early detection and mitigation of global scale attacks • Could be differentiator for Motorola’s 802. 16 products Switch/ BS controller 802. 16 BS Users (a) Original configuration (b) WAIDM deployed

Adaptive Intrusion Detection and Mitigation for Wi. MAX Networks (WAIDM) • Scalability – Online traffic recording » Reversible sketch for data streaming computation » Record millions of flows (GB traffic) in a few hundred KB – Online sketch-based flow-level anomaly detection » Adaptively learn the traffic pattern changes • Accuracy Integrated approach for false positive reduction – Automatic Polymorphic Worm signature generation (Hamsa) – Network element fault Diagnostics with Operational Determinism (ODD)

WAIDM Architecture Normal flows Reversible sketch monitoring Streaming packet data Filtering Remote aggregated sketch records Sent out for aggregation Local sketch records Sketch based statistical anomaly detection (SSAD) Keys of suspicious flows Part I Sketchbased monitoring & detection Keys of normal flows Suspicious flows Per-flow monitoring Signature -based detection Polymorphic worm detection (Hamsa) Network fault diagnosis (ODD) Intrusion or anomaly alarms Data path Control path Modules on the critical path Modules on the non-critical path Part II Per-flow monitoring & detection

Accomplishments • Motorola Interactions – – – • Patents being filed through Motorola – – • The first two components of WAIDM are ready for field test on Motorola Wi. MAX networks or testbed Product teams interested to use as differentiator (Networks security service director: Randall Martin) Close collaboration/interaction with Motorola Labs (Judy Fu, Phil Roberts, Steve Gilbert) Reverse Hashing for High-speed Network Monitoring: Algorithms, Evaluation, and Applications. Hamsa: Fast Signature Generation for Zero-day Polymorphic Worms with Provable Attack Resilience. Students involved – – Three Ph. D. students: Yan Gao, Zhichun Li, & Yao Zhao One M. S. student: Prasad Narayana

Accomplishments on Publications • Four conference papers and one journal papers (with another four under submission) – A Do. S Resilient Flow-level Intrusion Detection Approach for High-speed Networks, to appear in IEEE International Conference on Distributed Computing Systems (ICDCS), 2006 (14%). – Hamsa: Fast Signature Generation for Zero-day Polymorphic Worms with Provable Attack Resilience, to appear in IEEE Symposium on Security and Privacy, 2006 (about 8%). – Reverse Hashing for High-speed Network Monitoring: Algorithms, Evaluation, and Applications, Proc. of IEEE INFOCOM, 2006 (18%). – IDGraphs: Intrusion Detection and Analysis Using Stream Compositing, to appear in IEEE Computer Graphics & Applications, special issue on Visualization for Cyber Security, 2006. » An earlier version also in Proc. of the IEEE Workshop on Visualization for Computer Security (Viz. SEC), 2005

Hamsa: Fast Signature Generation for Zero-day Polymorphic Worms with Provable Attack Resilience

Desired Requirements for Polymorphic Worm Signature Generation • Network based, no host-level info • Noise tolerant – Most network flow classifiers suffer false positives. – Even host based IDSes, such as honeynets, can be injected with noise. • Attack resilience – Attackers always try to evade the IDS • Efficient signature matching for high-speed links No existing work satisfies these requirements !

Hamsa Architecture

Choice of Signatures Two classes of signatures • Content based – Invariant content » Protocol Frame » Control Data: leading to control flow hijacking » Worm Executable Payload – Token: a substring with reasonable coverage to the suspicious traffic – Signatures: conjunction and/or sequence of tokens • Behavior based Our choice: content based – Fast signature matching » ASIC based approach can achieve 6 ~ 8 Gbps – Generic, not depend upon any protocol or server

Hamsa Design • Key idea: model the uniqueness of worm invariants – Greedy algorithm for finding token conjunction signatures • Highly accurate while much faster – Both analytically and experimentally – Compared with the latest work, polygraph – Suffix array based token extraction • Provable attack resilience guarantee – Propose an adversary model • Noise tolerant

Hamsa Signature Generator • Core part: Model-based Greedy Signature Generation • Iterative approach for multiple worms • Signature refinement for better specificity – False positive is worse than false negative

Experiment Methodology • Experiential setup: – Suspicious pool: » » – • Three pseudo polymorphic worm based on real exploits (Code. Red II, Apache-Knacker and ATPhttpd), Two polymorphic engine from Internet (CLET and TAPi. ON). Normal pool: 2 hour departmental http trace (326 MB) Signature evaluation: – – False negative: 5000 generated worm samples per worm False positive: » » » 4 -day departmental http trace (12. 6 GB) 3. 7 GB web crawling including. mp 3, . rm, . ppt, . pdf, . swf etc. /usr/bin of Linux Fedora Core 4

Results on Signature Quality Worms Training FN Training FP Evaluation FN Evaluation FP Binary evaluation FP 0 0 Signature Code-Red II 0 0 0 {'. ida? ': 1, '%u 780': 1, ' HTTP/1. 0rn': 1, 'GET /': 1, '%u': 2} CLET 0 0. 109% 0 0. 06236% 0. 268% {'0x 8 b': 1, 'xffxff': 1, 'tx 07xeb': 1} • Single worm with noise – – • Suspicious pool size: 100 and 200 samples Noise ratio: 0%, 10%, 30%, 50% Noise samples randomly picked from the normal pool Always get above signature and accuracy Multiple worms with similar results

Speed and Attack Resilience Results • Implementation with hybrid of C++/Python – • 500 samples with 20% noise, 326 MB normal traffic pool, 15 seconds on an XEON 2. 8 Ghz Provable attack resilience – – – We propose a new attack, token-fit It fails the existing state-of-the-art, Polygraph BUT We still can generate correct signature!

Ongoing Work • Semantics Aided Signature Generation for Zero-day Polymorphic Worms – Some worms do not have any content invariant – Incorporate semantic information for more accurate detection • Vulnerability Analysis for 802. 16 Wi. MAX Network Protocols – Use formal verification methods to automatically search for vulnerabilities in 802. 16 specs. – Completeness and correctness