Introduction to Intrusion Detection Magnus Almgren Chalmers University

Introduction to Intrusion Detection Magnus Almgren Chalmers University of Technology

Outline of Presentation l My background l Justification for intrusion detection l Types of intrusion detection systems l Figures of merit l What to expect when deploying an IDS l State of the art l Discussion: Designing an IDS

My Background l Master of Science in Engineering Physics, Uppsala University Ä Thesis: Building an IDS for IBM Research, Zurich l l l Master of Science in Computer Science with Distinction in Research, Stanford University (Fulbright Scholar) Ä Report: Web Server Security l l l A Lightweight Tool for Detecting Web Server Attacks, NDSS 2000 Prototype for later security product Best Research Report, MS level at Stanford University Application-Integrated Data Collection for Security Monitoring, RAID 2001 Computer Scientist, SRI International Ä Research projects for mostly US military (DARPA, etc) for building better intrusion detection systems. Involved with intrusion detection research for about six years.

Justification for intrusion detection l Premise ¡ We will never be able to have completely secure systems l l l An Intrusion Detection System monitors computer systems ¡ ¡ l Bugs in newly built software, and legacy systems (db) used Compare with a traditional surveillance camera enforces your local security policy IDS and Fraud Detection Systems similar ¡ This talk mostly about IDS but will mention FDS

What is an intrusion detection system? l IDS are used for ¡ ¡ ¡ Detect intrusions and intrusion attempts Give alarms Stop on-going attacks Trace attackers Investigate and assess the damage Gather information for recovery actions IDS key components Audit Source Models Detection Response

Where does intrusion detection fit into the security landscape? Security Prevention Encryption Detection Firewalls Network IDS l Recovery Intrusion Detection Systems (IDS) Virus Scanners … Principle: Defense in Depth (layered mechanisms) Prevent Detect React/Survive

Features of an IDS l An IDS is ¡ Closely integrated into the system it is monitoring: l l ¡ ¡ ¡ concerning current threats, and the implementation of the monitored system. Almost always designed and built after the monitored system is deployed. Sometimes seemed as ad hoc. A necessary complement to other types of protection mechanisms. l Even if you do not succeed to stop the intrusion, it is of value to know that an intrusion has indeed occurred, how it occurred and which damage that has been caused.

History of Intrusion Detection l First non-secret report, 1980 ¡ Anderson, James P, Computer Security Threat Monitoring and Surveillance l Seminal Paper, 1987, SRI International ¡ Denning, l Field D E, An Intrusion-Detection Model is still relatively new ¡ Key questions remain to be answered For example: l How do you measure security?

Types of IDS l Traditionally ¡ Based l on the placement (implying input/audit data) Host-based versus network-based IDS ¡ Based l divided in two different ways on the type of detection paradigm Model normal versus anomalous behavior l Modern ¡ Hybrid systems are a mixture system using both normal and anomalous models, and ¡ application-based, etc.

Traditional Host IDS l IDS deployed on the host it monitors ¡ ¡ l Data sources ¡ ¡ l Reliable source of local events (processes, uid, paths), but goes down with the monitored host. Audit data – useful, system call information dominates, but limited insight into application data Application/system log files – limited content, disk space management Usually, data produced after the fact Blind to most network-level attacks Administrative overhead ¡ With many resources to protect, also many IDS need to be deployed and administered.

Traditional Network IDS l Advantages: ¡ ¡ ¡ Passive, non-invasive Hidden Can monitor multiple hosts from one location l Problem areas: ¡ ¡ ¡ Encrypted traffic Evasion tricks Network speed Session/transaction reconstruction and statefulness Timely preemption difficult

Detection Models l Misuse detection (signature-based) ¡ l Define what is wrong and give alarms for such behavior. (default permit) Anomaly Detection Define what is correct and give alarms for everything else. (default deny) Normal behavior Number ¡ Sound alarm, if payload contains /. . %c 0%af. . /

Detection Coverage l The detection coverage of the same type of IDS is similar. ¡ NIDS examples l l Snort, ISS, Dragon, SRI’s e. Xpert-Net Different IDS types complement each other somewhat. NIDS HIDS

IDS Survey: Some Research Systems Ref Model Generation Process selflearning self-organizing maps hybrid NN ripper e. Bayes programmed e. Xpert-BSM bottleneck snort web. IDS verification misuse anomaly NN e. Stat specbased anomaly Ref Model Type

Figures of Merit for IDS No alarms should be given in the absence of intrusions. l Intrusion (attempts) must be detected. l Probability of detection hit rate l Rate of false positives false alarm rate l Rate of false negatives miss rate l intrusion no alarm Miss OK OK False Alarm

Theoretical Baseline: Base Rate Fallacy l Bayesian l Intrusions Detection Rate are uncommon, which means that even if the false alarm rate is low you will still have many false alarms.

Evasion Techniques: Web example l Using lower protocol levels [Ptacek and Newsham] l Crafting ambiguous HTTP request 1. 2. GET Tab /cgi-bin/phf GET /%00 cgi-bin/phf 3. GET / HTTP 1. 1 Host: victim Content-Length: 3 123 GET /cgi-bin/phf l The evasion techniques work because Web servers and NIDS decode them differently

Other Problems: Alert Inundation l SYN flood attack (Denial-of-service) ¡ Early systems: 1 alert per SYN packet ¡ Today better … l Port Scans ¡ Happens l Ineffective ¡ Code all the time, but we do not care Attacks Red against my Apache server Low Abstraction Level of Alerts: Several SYN packets received We are under a Do. S attack

Popular research area: Alert Handling l Alert Aggregation ¡ l Alert Correlation ¡ ¡ l More complex correlations, often between different types of sensors. Interoperability with IDMEF Topology Vetting ¡ l Simple alerts aggregated within a sensor By knowing the topology of the network and which services are installed, we can avoid harmless alerts. Alert Prioritizing ¡ Prioritize alerts based on the interest of the operator.

Event Stream Models Alert Handling Maturity: Sensors Commercial Filtering & Aggregation Research Correlation Plan Recognition

Deploying an IDS l IDS != FW ¡ ¡ ¡ Not something you just drop into your network and then leave there Extensive tuning needed Some regular analysis of all alarms should be expected l l Compare with a surveillance camera. Not much use without a guard watching it! One difference IDS versus FDS ¡ Usually more people handling FDS alerts l Economic cost easier to calculate, and that justifies people’s salary

State of the Art Gartner: IDS is dead. Long live IPS. l Catches well-known attacks and l ¡ l may even catch simple variants. Do not detect ¡ ¡ ¡ new attacks, larger variants, and the Insider (basically ignored). l Especially true, if masquerades: uses a little bit more than he should of all resources. False alarms are a major problem. l Mostly misuse systems, and very few anomaly detection systems among the commercial systems. l

DARPA’s Four Generations of Security and Survivability Technology l Trusted 1 st Prevent intrusions 2 nd Detect, block intrusions 3 rd Operate through attacks Predict, diagnose, 4 th heal, and improve Computing Base l Authentication and access control l Encryption l Multi-level security l Boundary Controllers l Intrusion Detection Systems l Public l Real-time l Error execution monitoring l Intrusion tolerance l Self-regenerative Key Infrastructure l Virtual Private Networks Detection l Error Compensation l Graceful degradation systems l Self-optimization, selfdiagnosis, self-healing l Reliability is continually improved l Critical services never lost

Future Directions of Research l AI & IDS Revival in the US ¡ More self-aware systems that can self-heal ¡ Anomaly-based systems l My paper: Active Learning and IDS l Formally define systems and their detection capabilities l Handling gigabit speeds l Alert handling l Less of black box solutions

Relevant Papers for Intrusion Detection l l l l time Lightweight Tool for Detecting Web Server Attacks Application-Integrated Data Collection for Security Monitoring Malicious Code Outbreak Discovery: Issues and Approaches An Architecture for an Adaptive Intrusion Tolerant Server Dependable Intrusion Tolerance: Technology Demo Implications of IDS Classification on Attack Detection Using Active Learning for Intrusion Detection (most found on my home page) http: //www. ce. chalmers. se/staff/almgren

Further Reading l l l Amoroso: Intrusion Detection Intrusion. net Books, ISBN 0 -9666700 -7 -8 Escamilla: Intrusion Detection Wiley, ISBN 0 -471 -29000 -9 (mainly about networking and firewalls) Bace: Intrusion Detection, MTP, ISBN 1 -57870 -185 -6 Lindqvist: On the Fundamentals of Analysis and Detection of Computer Misuse (phd thesis), Department of Computer Engineering, Chalmers, ISBN 91 -7197 -832 -1 Northcutt et al ¡ Network Intrusion Detection ¡ Intrusion Signatures and Analysis ¡ Inside Network Perimeter Security: The Definitive Guide to Firewalls, Virtual Private Networks (VPNs), Routers, and Intrusion Detection Systems ISBN 0735712654 ISBN 0735710635 ISBN 0735712328