15 744 Computer Networking Network Intrusion Detection Systems

15 -744: Computer Networking Network Intrusion Detection Systems (NIDS)

NIDS • Background • Bro: A NIDS • NIDS Traffic Normalization • Honeycomb: NIDS signature generation 2

Recall: Network “ 101” vs. Reality Traditional view: “Dumb” network Reality: Lots of in-network processing Appliances or Middleboxes: IDS, Firewall, Proxies, Load balancers…. 3

Recall: Middleboxes Galore! Data from a large enterprise Type of appliance Survey across 57 network operators Number Firewalls 166 NIDS 127 Media gateways 110 Load balancers 67 Proxies 66 VPN gateways 45 WAN Optimizers 44 Voice gateways 11 Total Middleboxes 636 Total routers ~900 APLOMB (SIGCOMM’ 13) 4

Firewalls • Network-level firewalls: • Limit access to the network • Installed at perimeter of the network • Allows traffic specified in the policy • Drops everything else Internal Network Firewall Internet 5

Typical Firewall Configuration • Internal hosts can access DMZ and Internet • External hosts can access DMZ only, not Intranet • DMZ hosts can access Internet only DMZ X X • Advantages? • If a service gets compromised in DMZ it cannot affect internal hosts Intranet 6

Intrusion Detection Systems (IDS) • Firewalls allow traffic only to legitimate hosts and services • Traffic to the legitimate hosts/services may contain attacks • Solution? • Intrusion Detection Systems • Monitor data and behavior • Report when identify attacks 7

Firewall vs. NIDS • Firewall • Active filtering • Fail-close Internet NIDS • Network IDS • Passive monitoring • Fail-open FW Internet • Advantages and disadvantages? 8

Types of IDS Signaturebased Anomalybased Host-based Network-based 9

Signature-based IDS • Characteristics: • Uses known pattern matching to signify attack • Can identify intrusions from packet header/data • May use Boolean operators in rule set • ‘this_string’ • ‘this_variable’ AND ‘that_number’ • ‘this_string’ AND ‘that_variable’ NOT ‘that_tcp_flag’ 10

Signature-based IDS • Advantages • Widely available • Easy to implement • Easy to update • Disadvantages • • Cannot detect attacks for which it has no signature Must be updated for each new attack and attack variant Lag time from new exploit to update can be dangerous ‘New’ attack variant can be created by changing a single string • May be resource intensive 11

Anomaly-based IDS • Characteristics • Uses statistical models or a machine learning engine to characterize normal usage behaviors • Recognizes departures from normal as potential intrusions 12

Anomaly-based IDS • Advantages • Can detect attempts to exploit new and unforeseen vulnerabilities • Can recognize unusual traffic based on a number of characteristics: • Payload • Source address • Time • Can recognize authorized usage that falls outside the normal pattern • Disadvantages • Generally slower, more resource intensive compared to signaturebased IDS • Greater complexity, difficult to configure • Higher percentages of false alerts • Link between abnormal and intrusive may be weak 13

Host-based IDS • Characteristics: • Runs on a single host • Can analyze audit-trails, logs, integrity of files and directories, etc. • Tripwire, Fcheck • Real. Secure® Server Sensor • Research systems such as Emerald • May report to central administrative console 15

Host-based IDS • Advantages: • Relatively easy to deploy and to manage • Only one machine is involved • May require only one administrator • Creates single source of log and alert information • Generally not resource intensive - in most cases • Often will not require CPU, memory, etc. beyond what is needed for OS and applications • Disadvantages: • Works well for a single machine; extremely labor-intensive to monitor multiple machines each running a host-based IDS • If the host is compromised, the IDS may cease to function and thus no more alerts will be generated 16

Network-based IDS • Network monitor • Passively captures traffic and inspects it • Can also function in a client-server model • Sensors are located on multiple machines across the network • All sensors feed data to console • Console machine handles logging and alerting 18

Network-based IDS • Advantages • Positioned properly, can test effectiveness of firewalls, router access lists, etc. • Can monitor multiple machines from one physical and logical location • Console can generate an alert if a monitored machine/network has ceased to send information • Disadvantages: • Since it is capturing all network packets, can produce large log/alert files • Can be difficult to cull through vast amount of information • Console machine generally must be quite powerful, similar to a workgroup server • If console machine goes down then multiple machines may be left unmonitored • Communication from sensors to console may increase overall network traffic levels 19

NIDS • Background • Bro: A NIDS • NIDS Traffic Normalization • Honeycomb: NIDS signature generation 20

Bro: Detecting intruders in real-time • Bro is a standalone NIDS developed by Vern Paxson • Designed to keep LBL an open environment (to resist the need to install a firewall) • Goals • • • High-speed monitoring, no packet drops Real-time notification Separate mechanism from policy Extensibility Simple to use, guard against mistakes Tolerate attacks on NIDS Paxon, 1998 21

Bro system architecture Policy script Alerts/notifications Policy Script Interpreter Event control Complexity of operations increases Event stream Event Engine tcpdump filters Volume of data decreases Filtered packet stream libpcap Packet stream Network 22

libpcap Layer • Only passes relevant packets to Event Engine • Uses BSD Packet Filter (BPF) to efficiently filter packets • Filter rules • tcp port finger or tcp port ftp or tcp port telnet or port 111 or tcp[13] & 7 != 0 23

Reminder: TCP header format 24

Event engine • State for each connection, based on <Src. IP, Src. Port, Dst. IP, Dst. Port> • If state not present, allocate fresh state • TCP processing • Update state based on SYN/FIN/RST flags • Process acknowledgment • SYN generates a timer event, if nothing happens after 5 min, generate connection_attempt event • UDP processing • Initial packets generate udp_request and udp_reply events 25

Policy script interpreter • Clear separation of event generation and response to achieve clear separation between mechanism and policy • Advantage: extensibility (adding a new protocol analyzer and new event handler usually separate from other components) • Event are stored in a FIFO queue and processed sequentially • Policy script interpreter executes event handlers 26

Bro policy scripts • Goal: A clear and error-free language • Written in a specialized language: • • • Network types (IP addresses, connections, protocol, etc. ) Typed constants, variables Network operators (comparison, ranges, etc. ) Control statements (IF/THEN, etc. ) Regular expressions • It can • Generate alerts • Call exterior programs 27

Bro policy scripts: variables and operators if ([H, S] in allowed_services) … it’s okay … 28

Offline Analysis • Checkpointing in order to • Reclaim memory of dormant connections • Offline-analysis • Weakness: no state transitioned across checkpoint • Logs maintained for a long time for • Forensics on past break-ins • Complex analysis that would be too expensive to be real -time 30

Attacks on Bro • Overload attack • Send packets that match filters • Send packet streams that generate events • Try to generate events that lead to recording to disk • Defense strategy 1: Assume monitor capabilities are secret. • Good assumption? • Defense strategy 2: Lower the load (e. g. , stop capturing HTTP traffic) • Effective if attacker does not know how Bro lowers load 31

Attacks on Bro monitor • Crash attack • Find packet sequence that crashes monitor • Exhaust memory, disk resources • Defense strategy 1: Careful code analysis • Defense strategy 2: OS-level watchdog timer with subsequent packet capturing (sacrificing real-time detection) 32

Attacks on Bro monitor • Subterfuge attack • Mislead Bro: find traffic patterns that Bro and the end system interpret differently • Example: Carefully setting TTL field • Defense strategy: Traffic normalization 33

NIDS • Background • Bro: A NIDS • NIDS Traffic Normalization • Honeycomb: NIDS signature generation 35

NIDS: Evasion and Normalization • Problems • NIDS only has partial knowledge of what traffic the host sees (e. g. , TTL expires, MTU) • Ambiguities in TCP/IP (e. g. , Overlapping IP & TCP fragments) • Different OS implement standard differently • Approach: traffic normalization Handley et al. , 2001 36

Small TTL attack NIDS sees: A T T I A C K NIDS Host Internet End-host sees: Attacker’s data stream A T T I A C K A T T A C K same TCP seq #, “I” has short TTL 37

Fragmentation overlap attack NIDS sees: A T T A I C K NIDS Internet Host End-host sees: Attacker’s data stream A T T A I C K same TCP seq # or same IP frag offset 38

Approach: traffic normalizer • Introduce “bump in the wire”: traffic normalizer to evade protocol ambiguities NIDS Internet Normalizer Host 39

Alternative approaches • Host-based IDS • Loses the advantages of monitoring the entire site cheaply • Major deployment and management efforts • Detailed Intranet map • Required knowledge of every OS and app • Bifurcating analysis • If the NIDS does not know which of the two interpretations the end system may apply to an input packet, split the analysis context • State explosion? 40

NIDS • Background • Bro: A NIDS • NIDS Traffic Normalization • Honeycomb: NIDS signature generation 49

Honeycomb: Motivation • NIDS work based on signatures • How to generate signatures to begin with? • Common practice is manual and expertisebased • Can we do better? Kreibich and Crowcroft, 2004 50

Honeycomb: Background • Good NIDS signatures should be • Narrow enough; otherwise, high false positives • Flexible enough; otherwise, high false negatives • Honeypots: decoy computer resources to detect or counteract computer resources • Examples: dummy database items, dummy web servers, … • Key Idea behind Honeycomb: • Traffic sent to a honeypot should be malicious • We can extract its pattern and use it as a NIDS signature 51

Honeycomb: Architecture • Good NIDS signatures: 52

Honeycomb – Connection Tracking • Goal: Find attack signatures by • finding deviation from protocols • finding inbound traffic patterns 56

Honeycomb: Pattern matching in flow content • Horizontal pattern detection: happens every nth message and applies longest common substring (LCS) algorithm 58

Honeycomb: Pattern matching in flow content • Vertical detection: concatenates messages then applies LCS algorithm • Advantage? 59

Honeycomb - Results Net. Bios web MSSQL (Slammer worm) Distribution of TCP and UDP traffic destination ports in packets directed at the honeypot. 61

Honeycomb - Results Signature Honeycomb created for the Slammer Worm 62

Honeycomb - Results Signature Honeycomb created for the Code. Red II Worm 63

Summary • Network intrusion detection systems is a complement to a firewall • Goal: Finding malicious traffic both low false positives and false negatives • Decoupling policy from analysis is important • Protocols are ambiguous • Unclear how end-hosts implement the ambiguous parts • Can be used to evade NIDS • Possible solution: Protocol normalization (a. k. a. protocol scrubbing) • Opportunities for automatic signature generation 64