The Roles of Intrusion Detection and Data Fusion

The Roles of Intrusion Detection and Data Fusion in Cyber Security Situational Awareness A Review of the Published Literature and Discussion of Future Research Plans Nicklaus A. Giacobe

Cyber Security Situational Awareness Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Intrusion Detection (ID) Plays and Important Role in Developing Situational Awareness Cyber Situational Awareness = Network Security Situational Awareness Activities Performed on Behalf of an Organization – “Network Security Office” Activities Performed by Computer/Network Security Analysts Difficult, Complex Work – Lots of Data from IDS, Antivirus Systems, Firewall Logs, Server Security Logs, etc. Ever-Changing Landscape - New Threats, New Technologies, New Software, New Vulnerabilities

Cyber Security Situational Awareness Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion This Introduction Part 1: What is the Current State of ID Technology? Part 2: What are We Trying to Accomplish? Part 3: Future Research Recommendations Conclusion/Discussion

Part 1: The Current State of Technology in ID Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion History of ID Alert Correlation and Data Fusion Techniques Visualizations

Part 1: The Current State of Technology in ID Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion History of ID Alert Correlation and Data Fusion Techniques Visualizations

History of Intrusion Detection Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Two Different Locations to Monitor Host-Based IDS (Denning) Log Files (C 2 compliance) on Unix Machines (Denning 1987) IDES/NIDES – Baseline “normal” user behavior (Javitz et al. 1994) Network-Based IDS (Mukherjee/Heberlein) NSM (LAN Monitor) – history of previous connections, known bad actors lists, signatures of attack types (Mukherjee et al. 1994) NIDS (Multiple Network IDS and Host) (Snapp et al, 1991) (interesting JDL comparison)

History of Intrusion Detection Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Two Different Methods of Analysis Pattern-Matching (Misuse) Detection (Spafford) Match activity to patterns of known undesired behavior (Kumar et al. 1994, 1995) Tripwire – MD Hashing of files (Kim et al. 1994) DDo. S prevention /SYN Floods / Active Do. S prevention (Schuba et al. 1997) Anomaly Detection (Stolfo) Looking for abnormalities in network traffic (Lee et al. 1999) Qualitative evaluation of the data stream (statistical methods) (Portnoy, et al. 2001) – alert on infrequent types of data Statistical Payload Evaluations – for Worm Detection (Wang et al. 2004, 2006 a, 2006 b) and mitigation (Locasto et al. , 2006)

History of Intrusion Detection Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Testing and Evaluation of IDSs DARPA IDS Data Sets from 1998 -2000 1999 Data Set Contained 2 Weeks of “training data” with labeled known intrusions 7 Weeks of unlabeled data Evaluate IDSs under design or in production Over-fit problem IDSs could be developed that find all of the problems in the “training data”, but could be very poor at alerting on novel intrusion methods

Part 1: The Current State of Technology in ID Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion History of ID Alert Correlation and Data Fusion Techniques Visualizations

Alert Correlation and Data Fusion Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Correlate by Source, Destination or Attack Method Non-Trivial port-number vs. service name, IP address vs. hostname, etc. (Cuppens 2001) Need Adaptors – Different systems not designed for fusion (Debar et al. 2001) Promise of better understanding… see next slide

Understanding Through Correlation Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Adapted from (Debar et al. 2001)

Alert Correlation and Data Fusion Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion JDL Fusion Model (Hall and Mc. Mullen 2004)

Alert Correlation and Data Fusion Introduction Current State of ID Technology Theory and Background Future Research JDL Fusion Model (Hall and Mc. Mullen 2004) Conclusions & Discussion Source Pre. Processing Level 1 Object Refinement Level 2 Situation Refinement Level 3 Threat Refinement

Part 1: The Current State of Technology in ID Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion History of ID Alert Correlation and Data Fusion Techniques Visualization of Underlying and Fused Data

Data Fusion Techniques Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Bayesian Inference Complete list of all possible states of the system Probabilities of current state Need for accurate historical data (Holsopple et al. 2006) D-S Theory No need for exact knowledge Sort out independent evidence and combine it using the Dempster Rule Very human-like logical combination Can combine evidence of non-similar sources/data types

Data Fusion Techniques Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Data Mining Algorithms Support Vector Machines (SVMs) (Liu et al. 2007 x 3) Neural Networks (Wang et al. 2007) May be helpful in rapidly combining multiple sources of similar data Thomas and Balakrishnan (2008) Combined alert data from 3 different IDSs (PHAD, ALAD, Snort) using MLFF-NN Tested vs. DARPA 1999 data set Showed improved detection rates of the known data over each individual IDS (68% vs. 28%, 32%, 51%)

Part 1: The Current State of Technology in ID Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion History of ID Alert Correlation and Data Fusion Techniques Visualizations

Visualizations Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Based on Network Topology Based on Geopolitical Topology Network Traffic Representations Alert and Track-Based Displays

Hierarchical Network Map from Mansmann and Vinnik (2006) Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

Representation of Threats and Actors on a Geopolitical Map from (Pike et al. 2008) Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

Representation of host to port to remote host of network traffic from (Fink et al. 2004) Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

Panel Displaying Network Connections from a Single Host from (Fischer et al. 2008) Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

Representing the Three Ws from (Foresti et al. 2007) Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion

Part 2: What are We Trying to Accomplish? Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Definition of Computer Security Theory of Situational Awareness Cognitive Load Theory Cognitive Task Analysis

Part 2: What are We Trying to Accomplish? Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Definition of Computer Security Theory of Situational Awareness Cognitive Load Theory Cognitive Task Analysis

Definitions… Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion (Computer) Security is… Manunta (1999) Security is interaction of Asset (A), Protector (P) and Threat (T) in a given Situation (Si) CIA Triad (Tipton et al. 2007) Confidentiality Integrity Availability Bishop (2003) Only authorized actions can be executed by authorized users

Part 2: What are We Trying to Accomplish? Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Definition of Computer Security Theory of Situational Awareness Cognitive Load Theory Cognitive Task Analysis

Theory of Situational Awareness Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Endsley (1995) State of Knowledge Elements Situation Future Projection “Awareness Machine” unlikely Focus instead on “awareness support technologies”

Theory of Situational Awareness Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Endsley (1995)

Higher Levels of Fusion = Situational Awareness Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Mapping of IDS Fusion tasks between JDL Model and Endsley SA Model. From Yang et al. (2009)

Higher Levels of Fusion Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion INFERD Level 2 Fusion Engine – Based on a priori knowledge from system experts – pattern matching attack methods and known vulnerabilities of the system TANDI Level 3 Fusion – Projection of future attacks based on knowledge of vulnerabilities of the system (Yang et al. 2009)

Part 2: What are We Trying to Accomplish? Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Definition of Computer Security Theory of Situational Awareness Cognitive Load Theory Cognitive Task Analysis

Cognitive Load Theory Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Sweller et al. (1998) Working Memory (limited capacity) Long Term Memory (unlimited capacity, based on schemas to represent complex, related information) Split Attention Conflicting, Repetitive Modality Effect

Part 2: What are We Trying to Accomplish? Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Definition of Computer Security Theory of Situational Awareness Cognitive Load Theory Cognitive Task Analysis

Cognitive Task Analysis Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Biros and Eppich (2001) – CTA of IDS Analysts in the USAF - 5 capabilities required ID non-local addresses ID source addresses Develop mental image of “normal” behavior Create and maintain SA Knowledge sharing Killcrece et al. (2003) – CTA of gov’t/military security specialists – 3 general categories Reactive Work (majority of the work) Proactive Work Quality Management (training, etc)

Cognitive Task Analysis Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion D’Amico et al. (2007) – CTA of Network Security Professionals in the Department of Defense

Part 3: Where Do We Go From Here? Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Model Building To understand the contributions of the algorithm builders CTA To understand the needs of the analyst Visualization Recommendations Based on the work above

Conclusion Introduction Current State of ID Technology Theory and Background Future Research Conclusions & Discussion Current State of ID History of ID Alert Correlation and Data Fusion Data fusion techniques Visualization of underlying and fused data Theoretical Basis for Understanding SA in the Cyber Security Domain Definition of Computer Security Theory of Situational Awareness Cognitive Load Theory Cognitive Task Analysis Recommendations for Future Work Model Building - To understand the contributions of the algorithm builders CTA - To understand the needs of the analyst Visualization Recommendations – Based on Needs and Cognitive Capabilities of Analysts

Discussion and Questions Just in case you needed a prompt to ask questions … here it is