Networkbased Intrusion Detection Prevention and Forensics System Yan
Network-based Intrusion Detection, Prevention and Forensics System Yan Chen Department of Electrical Engineering and Computer Science Northwestern University Lab for Internet & Security Technology (LIST) 1 http: //list. cs. northwestern. edu
The Spread of Sapphire/Slammer Worms 2
Current Intrusion Detection Systems (IDS) • Mostly host-based and not scalable to highspeed networks – Slammer worm infected 75, 000 machines in <10 mins – Host-based schemes inefficient and user dependent • Have to install IDS on all user machines ! • Mostly simple signature-based – Cannot recognize unknown anomalies/intrusions 3 – New viruses/worms, polymorphism
Current Intrusion Detection Systems (II) • Cannot provide quality info forensics or situational-aware analysis – Hard to differentiate malicious events with unintentional anomalies • Anomalies can be caused by network element faults, e. g. , router misconfiguration, link failures, etc. , or application (such as P 2 P) misconfiguration – Cannot tell the situational-aware info: attack scope/target/strategy, attacker (botnet) size, etc. 4
Network-based Intrusion Detection, Prevention, and Forensics System • Online traffic recording [SIGCOMM IMC 2004, INFOCOM 2006, To. N 2007, INFOCOM 2008] – – Reversible sketch for data streaming computation Record millions of flows (GB traffic) in a few hundred KB Small # of memory access per packet Scalable to large key space size (232 or 264) • Online sketch-based flow-level anomaly detection [IEEE ICDCS 2006] [IEEE CG&A, Security Visualization 2006] – Adaptively learn the traffic pattern changes – As a first step, detect TCP SYN flooding, horizontal and vertical scans even when mixed • Online stealthy spreader (botnet scan) detection [IWQo. S 2007] 5
Network-based Intrusion Detection, Prevention, and Forensics System (II) • Polymorphic worm signature generation & detection [IEEE Symposium on Security and Privacy 2006, IEEE ICNP 2007] • Accurate network diagnostics [ACM SIGCOMM 2006] [IEEE INFOCOM 2007 (2)] • Scalable distributed intrusion alert fusion w/ DHT [SIGCOMM Workshop on Large Scale Attack Defense 2006] • Large-scale botnet and P 2 P misconfiguration event 6 forensics [work in progress]
System Deployment • Attached to a router/switch as a black box • Edge network detection particularly powerful LAN Switch Router Splitter Inter net Switch Splitter scan port Router Switch LAN HPNAIDM system (b) Original configuration LAN Switch LAN (a) RAND system Router Switch LAN Inter net RAND system Inter net LAN scan port RAND system Monitor each port separately (c) Monitor aggregated traffic from all ports 7
P 2 P Doctor: Measurement and Diagnosis of Misconfigured Peerto-Peer Traffic Anup Goyal, Zhichun Li, Yan Chen and Aleksandar Kuzmanovic Lab for Internet and Security Technology (LIST) Northwestern Univ.
What is P 2 P Misconfiguration Ø P 2 P file sharing accounted for > 60% of traffic in USA and > 80% in Asia Ø Thousands of peers send P 2 P file downloading requests to a “random” target on the Internet ü possibly triggered by bugs or by malicious reasons ü generates large amount of unwanted traffic Ø It contributes on an average of about 30% of the “Internet background radiation”
Motivations Ø On Dec. 6 th, 2006, 5, 047 sources generated >31, 000 packets/sec and 11 MB/s of traffic to a single unused IP in Northwestern University Ø P 2 P software DC++ has already been exploited by attackers for Do. S Ø direct gigabit “junk” data per second to a victim host from more than 150, 000 peers Ø Currently, little is known about the characteristics or root causes of P 2 P misconfiguration events 11 MB/s
Outline • • • Motivation Passive measurement results P 2 P Doctor system design Root cause diagnosis and analysis Conclusion
Peer Classification Poisoned Peers (Intentional) Unintentionally Misconfigured peers All the peers Normal Peers Bogus Peers Not in the P 2 P Network Anti-P 2 P Peers In the P 2 P Network
Passive Measurement • Honeynet/honeyfarm datasets • Events: # of unique sources > 100 in 6 hours – After filtering scan traffic • Event characteristics: – Mostly target a single IP – Duration: A few hours to up to a month LBL NU GQ Sensor 5 /24 10 /24 4 /16 Traces 883 GB 287 GB 49 GB Duration 37 7 26 months days e. Mule LBL NU 106 Bit. Torrent 242 90 Gnutella 1 1 Soribada 4 0 Xunlei 18 0 VAgaa 1 0
Popularity 30%! • Growth Trend: The average total connections of P 2 P misconfiguration events per month. • IP space – Observed in three sensors in five different /8 IP prefixes
Further Diagnosis • Problems with passive measurement on archived data – Events have gone – Hard to backtrack the propagation – Root cause? • Need a real-time backtracking and diagnosis system!
Outline • • • Motivation Passive measurement results P 2 P Doctor system design Root cause diagnosis and analysis Conclusion
Design of P 2 P Doctor System P 2 P-enabled Honeynet Backtracking system Root cause inference 10100101011101 infohash; ‘abc. avi’ P 2 P payload signature based responder Event identification Protocol parsing for metadata
Design of P 2 P Doctor System P 2 P-enabled Honeynet Backtracking system Index Server (tracker) Crawling BT: top 100, e. Mule: 185 Peer Exchange Protocol Crawling Root cause inference DHT Crawling
Design of P 2 P Doctor System P 2 P-enabled Honeynet • • Backtracking system Root cause inference What is the root cause? Which peers spread misconfigurtion? How is misconfiguration disseminated? What is the percentage of bogus peers in the misconfigured P 2 P networks?
Deployment and Data Collection • Deployed the P 2 P doctor system on NU honeynet (10 /24 networks in three /8) • Real-time events – Previous passive measurement data referred as historical events Bit. Torrent e. Mule # of events 20 42 Duration 23 days 08/23/2007 to 09/15/2007
Outline • • • Motivation Passive measurement results P 2 P Doctor system design Root cause diagnosis and analysis Conclusion
Root Cause Analysis • Methodology – Track how honeynet IPs propagated in P 2 P systems – Use unroutable IP space as a big honeynet (66. 8% of IPv 4 Space) – Hypothesis formulation and testing • Classification of measured peers – Misconfigured peers: Passively observed from honeynet – Backtracked peers: actively observed through backtracking – Reverse honeynet peers: the IP obtained by reversing the target IP from the honeynets • Results – Data plane traffic radiation – Detailed results focus on e. Mule and Bit. Torrent
Data Plane Traffic Radiation 1. 2. 3. 4 Resource mapping Peer Exchange Who has Beowulf. avi? DHT Index Server 1. 2. 3. 4
e. Mule – Root Cause • Byte ordering is the problem! 4. 3. 2. 1 1. 2. 3. 4 4. 3. 2. 1
e. Mule – Root Cause • Byte ordering is the problem! – Hypothesis from the historical data • In 80% of events, the reverse target IPs are alive – Verified with real-time events • 61% of the reverse honeynet peers indeed running e. Mule with the port number reported • For the backtracked peers which is in the unroutable IP space, 69. 6% of them having reverse IPs run e. Mule
e. Mule – Peers & Dissemination • Which peers spread misconfiguration? – 99. 24% of misconfigured peers are normal peers • How is the misconfiguration disseminated? – Index Server? No – Peer exchange? Yes • Percentage of bogus peers in e. Mule network? – [12. 7%, 25. 0%] w/ a total of 37, 079 backtracked peers
Bit. Torrent – Responsible Peers Ø Both anti-P 2 P and normal peers are responsible Ø Events classified to two types with diagonally different sets of characteristics Ø For anti-P 2 P peers events Ø All the sources are from the IP range owned by anti-p 2 p companies like Media Defender, Media Sentry, Net Sentry etc. Ø Seen 6 out of 7 major anti-P 2 P companies sources in our honeynet. Anti-P 2 P peers Normal peers Number of Events 127 (39%) 205 (61%) Client Software 100% - Azureus 90% - UTorrent (NU) 88% - Bit. Comet+Bit. Spirit (LBL) 400 25 All together Poisson 4. 5 hours 106. 1 hours Avg. number of Connections / src Arrival & Departure Avg. Duration
Bit. Torrent – Root Cause Ø Refuted Byte Ordering Hypothesis – For 20 real-time events, no reverse honeynet peers runs Bit. Torrent Ø For normal peer events, culprit is Peer Exchange (PEX) protocol implemented by u. Torrent-compatible clients Ø For anti-P 2 P peer events – Possibly related to Azureus system – Still an open question (No real-time events)
Bit. Torrent – Dissemination Ø How is misconfiguration disseminated? – Index server? - No – Peer exchange? - Yes Ø Percentage of bogus peers in Bit. Torrent network? Ø Out of a total of 9, 000 backtracked peers, only 13 IPs are unroutable and 3, 150 IPs gave connection timeout Ø 0. 14% < bogus Peers < 35%
Conclusions • The first study to measure and diagnose largescale P 2 P misconfiguration events • Found 30% Internet background radiation is caused by P 2 P misconfiguration – Popular in various P 2 P systems, exponential growth trend, and scattered in the IPv 4 space • For e. Mule, we found it is caused by network byte order problem • For Bit. Torrent, classified to anti-P 2 P peer events and normal peer events with diagonally different sets of characteristics – Found the u. Torrent PEX causes the problem in normal peer events
Backup Slides
Motivation Ø Given unprecedented amount of traffic, even a slight misconfiguration of the P 2 P system can result in a DDo. S kind of situation Ø Prevalence in time, space, and across a number of distinct P 2 P systems with a temporal increasing trend is alarming. Ø P 2 P miscongurations can cause innocent people to get involved in the above “war” between P 2 P and anti-P 2 P systems. Ø Presently, nothing is known about the causes or overall effects of P 2 P mis-configurations Ø Our goal is to determine the root cause(s) of each type of mis-configuration
Related Work • Misconguration is widely spread across different networked and distributed systems like BGP [Labovitz et al. ] and firewalls [Cuppens et al. ]. • Measurement studies of normal P 2 P traffic [ACM SOSP (2003), MCN (2002)], while we measure the abnormal P 2 P traffic observed in honeynets. • In [INFOCOM (2005)], Content pollution including intentional and unintentional pollution is widespread for popular titles. • P 2 P systems like Fasttrack and Overnet are vulnerable to the index poisoning attack [INFOCOM (2006)] • All of the above studies focus on the content pollution or index poisoning while our focus is the index misconfiguration. • First large-scale measurement study on the root causes for both intentional/unintentional index misconfiguration.
What is P 2 P Misconfiguration Ø More than 50% of the traffic in the Internet today is P 2 P traffic ü By Symantec Corporation’s recent report Ø P 2 P file sharing accounted for > 60% of traffic in USA and > 80% in Asia
- Slides: 35