CSC 382 Computer Security Network Monitoring CSC 382

CSC 382: Computer Security Network Monitoring CSC 382: Computer Security 1

Topics 1. 2. 3. 4. 5. 6. 7. Principles Models of Intrusion Detection False Positives Architecture of an IDS Deployment Active Response (IPS) Host-based IDS and IPS CSC 382: Computer Security 2

Principles of Intrusion Detection Characteristics of systems not under attack 1. User, process actions conform to statistically predictable pattern. 2. User, process actions do not include sequences of actions that subvert the security policy. 3. Process actions correspond to a set of specifications describing what the processes are allowed to do. Systems under attack do not meet at least one. CSC 382: Computer Security 3

Example Goal: insert a back door into a system – Intruder will modify system configuration file or program. – Requires privilege; attacker enters system as an unprivileged user and must acquire privilege. • Nonprivileged user may not normally acquire privilege (violates #1). • Attacker may break in using sequence of commands that violate security policy (violates #2). • Attacker may cause program to act in ways that violate program’s specification (violates #3). CSC 382: Computer Security 4

Goals of IDS 1. Detect wide variety of intrusions – Previously known and unknown attacks. – Need to adapt to new attacks or changes in behavior. 2. Detect intrusions in timely fashion – May need to be be real-time, especially when system responds to intrusion. • Problem: analyzing commands may impact response time of system. – May suffice to report intrusion occurred a few minutes or hours ago. CSC 382: Computer Security 5

Goals of IDS 3. Present analysis in easy-to-understand format. – – – Ideally a binary indicator. Usually more complex, allowing analyst to examine suspected attack. User interface critical, especially when monitoring many systems. 4. Be accurate – – Minimize false positives, false negatives. Minimize time spent verifying attacks, looking for them. CSC 382: Computer Security 6

Deep Packet Inspection • IDS requires, some firewalls do too. • DPI = Analysis of Application Layer data • Protocol Standard Compliance – Is port 53 traffic DNS or a covert shell session? – Is port 80 traffic HTTP or tunneled IM or P 2 P? • Protocol Anomaly Detection – Traffic is valid HTTP. – But suspicious URL contains directory traversal. CSC 382: Computer Security 7

Models of Intrusion Detection 1. Anomaly detection – What is usual, is known. – What is unusual, is bad. 2. Misuse detection – What is bad is known. – Look for what is bad, hope it doesn’t change. CSC 382: Computer Security 8

Anomaly Detection Analyzes a set of characteristics of system, and compares their values with expected values; report when computed statistics do not match expected statistics. – Threshold metrics – Statistical moments – Markov model CSC 382: Computer Security 9

Threshold Metrics • Counts number of events that occur – Between m and n events (inclusive) expected – If number falls outside this range, anomalous. • Example – Windows: lock user out after k failed sequential login attempts. Range is (0, k– 1). • k or more failed logins deemed anomalous • Threshold depends on typing skill. CSC 382: Computer Security 10

Sequences of System Calls • Define normal behavior in terms of sequences of system calls. • Example normal trace: open read write open write close • Doesn’t normally run other programs. • Attack trace: open read write open exec write close CSC 382: Computer Security 11

Finding Features • Which features best show anomalies? – CPU use may not, but I/O use may. • Use training data – Anomalous data marked. – Feature selection program picks features, clusters that best reflects anomalous data. • Use compiler techniques to build program model. CSC 382: Computer Security 12

Misuse Detection • Determines whether a sequence of instructions being executed is known to violate the site security policy. – Descriptions of known or potential exploits grouped into rule sets. – IDS matches data against rule sets; on match, potential attack found. • Cannot detect new attacks: – No rules to cover them. CSC 382: Computer Security 13

Example: snort Network Intrusion Detection System – Sniffs packets off wire. – Checks packets for matches against rule sets. – Logs detected signs of misuse. – Alerts adminstrator when misuse detected. CSC 382: Computer Security 14

Snort Rules • Rule Header – – Action: pass, log, alert Network Protocol Source Address (Host or Network) + Port Destination Address (Host or Network) + Port • Rule Body – Content: packet ASCII or binary content – TCP/IP flags and options to match – Message to log, indicating nature of misuse detected CSC 382: Computer Security 15

Snort Rule Example: rule for ssh shell code exploit alert tcp $EXTERNAL_NET any -> $HOME_NET 22 (msg: "EXPLOIT ssh CRC 32 overflow NOOP"; flow: to_server, established; content: "|90 90 90 90 90|"; reference: bugtraq, 2347; reference: cve, CVE-20010144; classtype: shellcode-detect; sid: 1326; rev: 3; ) CSC 382: Computer Security 16

Comparison and Contrast • Misuse detection: if all policy rules known, easy to construct rulesets to detect violations. – Usual case is that much of policy is unspecified, so rulesets describe attacks, and are not complete. • Anomaly detection: detects unusual events, but these are not necessarily security problems. CSC 382: Computer Security 17

False Positives • A new test for a disease that is 95% accurate • Assume 1 in 1000 people have disease. • Should everyone get the test? – Sample size: 1000 – Expect 0. 95 + (999 * 0. 05) positives – Ergo, 50 people will be told they have disease – If you test positive, only 2% chance you have it. CSC 382: Computer Security 18

IDS Architecture An IDS is essentially a sophisticated audit system – Agent gathers data for analysis. – Director analyzes data obtained from the agents according to its internal rules. – Notifier acts on director results. • May simply notify security officer. • May reconfigure agents, director to alter collection, analysis methods. • May activate response mechanism. CSC 382: Computer Security 19

Agents Obtain information and sends to director. Preprocessing – Simplifying and reformatting of data. Push vs Pull – Agents may push data to Director, or – Director may pull data from Agents. CSC 382: Computer Security 20

Host-Based Agents 1. Obtain information from logs – – – May use many logs as sources. May be security-related or not. May use virtual logs if agent is part of the kernel. 2. Agent generates its information – – Analyzes state of system. Treats results of analysis as log data. CSC 382: Computer Security 21

Network-Based Agents • Sniff traffic from network. – Use hubs, SPAN ports, or taps to see traffic. – Need agents on all switches to see entire network. • Agent needs same view of traffic as destination – TTL tricks, fragmentation may obscure this. • End-to-end encryption defeats content monitoring – Not traffic analysis, though. CSC 382: Computer Security 22

Aggregation of Information Agents produce information at multiple layers of abstraction. – Application-monitoring agents provide one view of an event. – System-monitoring agents provide a different view of an event. – Network-monitoring agents provide yet another view (involving many packets) of an event. CSC 382: Computer Security 23

Director • Reduces information from agents – Eliminates unnecessary, redundent records. • Analyzes information to detect attacks – Analysis engine can use any of the modelling techniques. • Usually run on separate system – Does not impact performance of monitored systems. – Rules, profiles not available to ordinary users. CSC 382: Computer Security 24

Example • Jane logs in to perform system maintenance during the day. • She logs in at night to write reports. • One night she begins recompiling the kernel. • Agent #1 reports logins and logouts. • Agent #2 reports commands executed. – Neither agent spots discrepancy. – Director correlates log, spots it at once. CSC 382: Computer Security 25

Adaptive Directors • Modify profiles, rulesets to adapt their analysis to changes in system – Usually use machine learning or planning to determine how to do this. • Example: use neural nets to analyze logs – Network adapted to users’ behavior over time. – Used learning techniques to improve classification of events as anomalous. • Reduced number of false alarms. CSC 382: Computer Security 26

Notifier • Accepts information from director • Takes appropriate action – Notify system security officer – Respond to attack • Often GUIs – Use visualization to convey information. CSC 382: Computer Security 27

Example Architecture: snort CSC 382: Computer Security 28

IDS Deployment IDS deployment should reflect your threat model. Major classes of attackers: 1. External attackers intruding from Internet. 2. Internal attackers intruding from your LANs. Where should you place IDS systems? 1. 2. 3. 4. Perimeter (outside firewall) DMZ Intranet Wireless CSC 382: Computer Security 29

IDS Deployment CSC 382: Computer Security 30

Sguil NSM Console CSC 382: Computer Security 31

Intrusion Prevention Systems • What else can you do with IDS alerts? – Identify attack before it completes. – Prevent it from completing. • How to prevent attacks? – Directly: IPS drops attack packets. – Indirectly: IPS modifies firewall rules. • Is IPS a good idea? – How do you deal with false positives? CSC 382: Computer Security 32

IPS Deployment Types Inline IPS Non-Inline Intranet IPS Intranet CSC 382: Computer Security 33

Active Responses by Network Layer • Data Link: Shut down a switch port. Only useful for local intrusions. Rate limit switch ports. • Network: Block a particular IP address. – Inline: can perform blocking itself. – Non-inline: send request to firewall. • Transport: Send TCP RST or ICMP messages to sender and target to tear down TCP sessions. • Application: Inline IPS can modify application data to be harmless: /bin -> /ben CSC 382: Computer Security 34

Host IDS and IPS • Anti-virus and anti-spyware – AVG anti-virus, Spy. Bot S&D • Log monitors – swatch, logwatch • Integrity checkers – tripwire, osiris, samhain • Application shims – mod_security CSC 382: Computer Security 35

Key Points • Intrusion detection is a form of auditing. • Models of IDS: – Anomaly detection: unexpected events. – Misuse detection: violations of policy. • The problem of false positives. • IDS Architecture: – Agents. – Director. – Notifiers. • Active Response: Intrusion Prevention Systems CSC 382: Computer Security 36

References 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. Richard Bejtlich, The Tao of Network Security Monitoring, Addison-Wesley, 2004. Matt Bishop, Computer Security: Art and Science, Addison-Wesley, 2003. Brian Caswell, et. al. , Snort 2. 0 Intrusion Detection, Snygress, 2003. William Cheswick, Steven Bellovin, and Avriel Rubin, Firewalls and Internet Security, 2 nd edition, 2003. The Honeynet Project, Know Your Enemy, 2 nd edition, Addison-Wesley, 2004. Richard A. Kemmerer and Giovanni Vigna, “Intrusion Detection: A Brief History and Overview, ” IEEE Security & Privacy, v 1 n 1, Apr 2002, pp 27 -30. Steven Northcutt and Julie Novak, Network Intrusion Detection, 3 rd edition, New Riders, 2002. Michael Rash et. al. , Intrusion Prevention and Active Response, Syngress, 2005. Rafiq Rehman, Intrusion Detection Systems with Snort: Advanced IDS Techniques Using Snort, Apache, My. SQL, PHP, and ACID, Prentice Hall, 2003. Ed Skoudis, Counter Hack Reloaded 2/e, Prentice Hall, 2006. Ed Skoudis and Lenny Zeltser, Malware: Fighting Malicious Code, Prentice Hall, 2003. CSC 382: Computer Security 37