Intrusion Detection Somesh Jha University of Wisconsin J

  • Slides: 59
Download presentation
Intrusion Detection Somesh Jha University of Wisconsin J. Giffin and S. Jha

Intrusion Detection Somesh Jha University of Wisconsin J. Giffin and S. Jha

Intrusion Detection Goal: Discover attempts to maliciously gain access to a system 12/16/2021 J.

Intrusion Detection Goal: Discover attempts to maliciously gain access to a system 12/16/2021 J. Giffin and S. Jha 2

Network Intrusion Detection Systems (NIDS) • Inspects packets at certain vantage points – For

Network Intrusion Detection Systems (NIDS) • Inspects packets at certain vantage points – For example, behind the routers • Look for malicious or anomalous behavior • Much more fine-grained than firewalls – Example: drop a packet whose payload “matches” a certain string • Called “deep packet inspection” 12/16/2021 J. Giffin and S. Jha 3

Network Intrusion Prevention System (IPS) • NIDS are generally “passive” – Raise alerts if

Network Intrusion Prevention System (IPS) • NIDS are generally “passive” – Raise alerts if something suspicious happens • IPS are active – Drop suspicious looking packages – Route certain packets for further inspection – Main challenge: have to work at line speeds 12/16/2021 J. Giffin and S. Jha 4

Classification of NIDS • Signature-based – Also called misuse detection – Establish a database

Classification of NIDS • Signature-based – Also called misuse detection – Establish a database of malicious patterns – If a sequence of packets “matches” one of the patterns, raise an alarm – Positives • Good attack libraries • Easy to understand the results – Negatives • Unable to detect new attacks or variants of old attacks • Example – Cisco, Snort, Bro, Tippingpoint, NFR, … 12/16/2021 J. Giffin and S. Jha 5

Classification of NIDS • Anomaly-based – Establish a statistical profile of normal traffic –

Classification of NIDS • Anomaly-based – Establish a statistical profile of normal traffic – If monitored traffic deviates “sufficiently” from the established profile, raise an alarm • Positives – Can detect new attacks • Negatives – High false alarm rate • High variability in normal traffic – Intruder can go under the “radar” • Examples – Mostly research systems 12/16/2021 J. Giffin and S. Jha 6

Classification of NIDS • Stateless – Need to keep no state – Example: raise

Classification of NIDS • Stateless – Need to keep no state – Example: raise an alarm if you see a packet that contains the pattern “mellissa” • Positives – Very fast • Negatives – For some attacks need to keep state 12/16/2021 J. Giffin and S. Jha 7

Classification of NIDS • Stateful – Keeps state – Sometime need to do reassembly

Classification of NIDS • Stateful – Keeps state – Sometime need to do reassembly • Reassemble packets that belong to the same connection, e. g. , packets that belong to the same ssh session • Quite hard! (out-of-order delivery) – Positives • Can detect more attacks – Negatives • Requires too much memory 12/16/2021 J. Giffin and S. Jha 8

Snort Filtered packet stream 12/16/2021 J. Giffin and S. Jha 9

Snort Filtered packet stream 12/16/2021 J. Giffin and S. Jha 9

libpcap • Takes the “raw” packet stream • Parses the packets and presents them

libpcap • Takes the “raw” packet stream • Parses the packets and presents them as a – Filtered packet stream • Library for packet capture • Website for more details – http: //www-nrg. ee. lbl. gov/. 12/16/2021 J. Giffin and S. Jha 10

Malicious Pattern Example alert tcp any -> 10. 1. 1. 0/24 80 (content: “/cgi-bin/phf”;

Malicious Pattern Example alert tcp any -> 10. 1. 1. 0/24 80 (content: “/cgi-bin/phf”; msg: “PHF probe!”; ) action pass log alert 12/16/2021 protocol source address source port J. Giffin and S. Jha destination address destination port 11

Malicious Patterns Example • content: “/cgi-bin/phf” – Matches any packet whose payload contains the

Malicious Patterns Example • content: “/cgi-bin/phf” – Matches any packet whose payload contains the string “/cgi-bin/phf” – Look at http: //www. cert. org/advisories/CA 1996 -06. html • msg: “PHF probe!” – Generate this message if a match happens 12/16/2021 J. Giffin and S. Jha 12

More Examples alert tcp any -> 10. 1. 1. 0/24 6000: 6010 (msg: “X

More Examples alert tcp any -> 10. 1. 1. 0/24 6000: 6010 (msg: “X traffic”; ) alert tcp !10. 1. 1. 0/24 any -> 10. 1. 1. 0/24 6000: 6010 (msg: “X traffic”; ) 12/16/2021 J. Giffin and S. Jha 13

How to generate new patterns? • Buffer overrun found in Internet Message Access Protocol

How to generate new patterns? • Buffer overrun found in Internet Message Access Protocol (IMAP) – http: //www. cert. org/advisories/CA-199709. html • Run exploit in a test network and record all traffic • Examine the content of the attack packet 12/16/2021 J. Giffin and S. Jha 14

Notional "IMAP buffer overflow" packet 052499 -22: 27: 58. 403313 192. 168. 1. 4:

Notional "IMAP buffer overflow" packet 052499 -22: 27: 58. 403313 192. 168. 1. 4: 1034 -> 192. 168. 1. 3: 143 TCP TTL: 64 TOS: 0 x 0 DF ***PA* Seq: 0 x 5295 B 44 E Ack: 0 x 1 B 4 F 8970 Win: 0 x 7 D 78 90 90 90 90 EB 3 B. . . . ; 5 E 89 76 08 31 ED 31 C 9 31 C 0 88 6 E 07 89 6 E 0 C ^. v. 1. 1. 1. . n. B 0 0 B 89 F 3 8 D 6 E 08 89 E 9 8 D 6 E 0 C 89 EA CD 80. . . n. . 31 DB 89 D 8 40 CD 80 90 90 90 1. . . @. . . 90 90 90 E 8 C 0 FF FF FF. . . . 2 F 62 69 6 E 2 F 73 68 90 90 90 /bin/sh. . 12/16/2021 J. Giffin and S. Jha 15

Alert rule for the new buffer overflow alert tcp any -> 192. 168. 1.

Alert rule for the new buffer overflow alert tcp any -> 192. 168. 1. 0/24 143 (content: "|E 8 C 0 FFFF FF|/bin/sh"; msg: "New IMAP Buffer Overflow detected!"; ) Can mix hex formatted bytecode and text 12/16/2021 J. Giffin and S. Jha 16

Advantages of Snort • Lightweight – Small footprint – Focused monitoring: highly tuned Snort

Advantages of Snort • Lightweight – Small footprint – Focused monitoring: highly tuned Snort for the SMTP server • Malicious patterns easy to develop – Large user community – Consider the IRDP denial-of-service attack – Rule for this attack available on the same day the attack was announced • Commercial company (Sourcefire) behind it 12/16/2021 J. Giffin and S. Jha 17

Disadvantages • Does not perform stream reassembly – Attackers can use that to “fool”

Disadvantages • Does not perform stream reassembly – Attackers can use that to “fool” Snort – Break one attack packet into a stream • Pattern matching is expensive – Matching patterns in payloads is expensive (avoid it!) • Rule development methodology is adhoc 12/16/2021 J. Giffin and S. Jha 18

Host-based ID • Monitor interaction between a specific program and OS • Raise an

Host-based ID • Monitor interaction between a specific program and OS • Raise an alarm if suspicious “system calls” are observed • Unlike NIDS monitoring happens at the end hosts • Need to model – Unusual behavior – Normal behavior 12/16/2021 J. Giffin and S. Jha 19

Goal: Discover attempts to maliciously gain access to a system Misuse Detection Specification-Based Monitoring

Goal: Discover attempts to maliciously gain access to a system Misuse Detection Specification-Based Monitoring Anomaly Detection • Specify patterns of attack or misuse • Specify constraints upon • Learn typical behavior of application program behavior • Ensure misuse patterns do not arise at runtime • Variations indicate • Ensure execution does not violate specification potential intrusions • Snort • Our work; Ko, et. al. • IDES • Rigid: cannot adapt to novel attacks • Specifications can be cumbersome to create • High false alarm rate 12/16/2021 J. Giffin and S. Jha 20

Specification-Based Monitoring • Two components: – Specification: Indicates constraints upon program behavior – Enforcement:

Specification-Based Monitoring • Two components: – Specification: Indicates constraints upon program behavior – Enforcement: How the specification is verified at runtime or from audit data 12/16/2021 J. Giffin and S. Jha 21

Analyst or Administrator Training Sets Static Source Code Analysis Static Binary Code Analysis Specification

Analyst or Administrator Training Sets Static Source Code Analysis Static Binary Code Analysis Specification Enforcement Execution Matches Model of Application Execution Obeys Static Ruleset 12/16/2021 J. Giffin and S. Jha 22

Representative Work by Ko, et al. • Specification: Programmers or administrators specify correct program

Representative Work by Ko, et al. • Specification: Programmers or administrators specify correct program behavior PROGRAM fingerd read(X) : - worldreadable(X); bind(79); write(“/etc/log”); exec(“/usr/ucb/finger”); END • Enforcement: At runtime, only allow actions that match the specified policy 12/16/2021 J. Giffin and S. Jha 23

Analyst or Administrator Training Sets Static Source Code Analysis Static Binary Code Analysis Specification

Analyst or Administrator Training Sets Static Source Code Analysis Static Binary Code Analysis Specification Enforcement Execution Matches Model of Application Execution Obeys Static Ruleset 12/16/2021 J. Giffin and S. Jha 24

Representative Work by Forrest, et al • Specification: Learn correct program behavior with training

Representative Work by Forrest, et al • Specification: Learn correct program behavior with training – Record sequences of system calls • Enforcement: Only accept behaviors similar to learned patterns • Example system: STIDE 12/16/2021 J. Giffin and S. Jha 25

Training • Repeatedly run the program, varying the input • For some n, record

Training • Repeatedly run the program, varying the input • For some n, record all sequences of n system calls observed – n depends upon the program • End result: database of n-tuples of system calls 12/16/2021 J. Giffin and S. Jha 26

cat (print file contents) geteuid, getegid, getgid, fstat, open, fstat, lseek, mmap, read, memcntl,

cat (print file contents) geteuid, getegid, getgid, fstat, open, fstat, lseek, mmap, read, memcntl, write, lseek, munmap, lseek, close, exit 12/16/2021 geteuid, getuid, getegid, getgid, fstat, open / lseek open, fstat lseek, mmap / munmap / close mmap, read, memcntl, write, lseek munmap, lseek close, close / exit J. Giffin and S. Jha 27

Enforcement • Monitor system calls generated by application • Ensure that the last n

Enforcement • Monitor system calls generated by application • Ensure that the last n calls match a sequence in the database • Option: Allow slight deviation from database – Training set may have been incomplete 12/16/2021 J. Giffin and S. Jha 28

cat (print file contents) geteuid, getegid, getgid, fstat, open, fstat, lseek, mmap, read, memcntl,

cat (print file contents) geteuid, getegid, getgid, fstat, open, fstat, lseek, mmap, read, memcntl, write, lseek, munmap, lseek, close, exit Accepts incorrect system call sequences geteuid, getegid, getgid, fstat, lseek, close, exit 12/16/2021 geteuid, getuid, getegid, getgid, fstat, open / lseek open, fstat lseek, mmap / munmap / close mmap, read, memcntl, write, lseek munmap, lseek close, close / exit J. Giffin and S. Jha 29

Drawbacks • Accepts incorrect call sequences – Due to window-based approach with ambiguity –

Drawbacks • Accepts incorrect call sequences – Due to window-based approach with ambiguity – Opportunity for attack sequence to go undetected • Only learn behaviors exercised in training set – Not all execution paths followed • Users must construct valid training sets • Users must determine window size 12/16/2021 J. Giffin and S. Jha 30

Drawbacks • Specification may over-fit the data • If training on real data, training

Drawbacks • Specification may over-fit the data • If training on real data, training set may contain exploits – Learn exploit pattern as normal 12/16/2021 J. Giffin and S. Jha 31

Analyst or Administrator Training Sets Static Source Code Analysis Static Binary Code Analysis Specification

Analyst or Administrator Training Sets Static Source Code Analysis Static Binary Code Analysis Specification Enforcement Execution Matches Model of Application Execution Obeys Static Ruleset 12/16/2021 J. Giffin and S. Jha 32

Our Approach function: save %sp, 0 x 96, %sp cmp %i 0, 0 bge

Our Approach function: save %sp, 0 x 96, %sp cmp %i 0, 0 bge L 1 mov 15, %o 1 call read mov 0, %o 0 call line nop b L 2 nop L 1: call read mov %i 0, %o 0 call close mov %i 0, %o 0 L 2: ret restore 12/16/2021 Specification: Static analysis of binary code • Specifications are automatically generated • Not reliant upon programmers to produce accurate specifications • Analyzes all execution paths • Source code may be unavailable J. Giffin and S. Jha 33

Our Approach read Enforcement: Operate an automaton modeling correct system call sequences • Dynamic

Our Approach read Enforcement: Operate an automaton modeling correct system call sequences • Dynamic ruleset close 12/16/2021 line • More expressive than static ruleset of Ko, et. al. J. Giffin and S. Jha 34

Non-Deterministic Finite Automaton (NFA) • Structure read close 12/16/2021 read line – States –

Non-Deterministic Finite Automaton (NFA) • Structure read close 12/16/2021 read line – States – Labeled edges between states • Edge labels are input symbols – call names • Path to any accepting state defines valid sequence of calls J. Giffin and S. Jha 35

Our Approach read Enforcement: Operate an automaton modeling correct system call sequences • Dynamic

Our Approach read Enforcement: Operate an automaton modeling correct system call sequences • Dynamic ruleset close 12/16/2021 line • More expressive than static ruleset of Ko, et. al. J. Giffin and S. Jha 36

The Binary View (SPARC) function: save %sp, 0 x 96, %sp cmp %i 0,

The Binary View (SPARC) function: save %sp, 0 x 96, %sp cmp %i 0, 0 bge L 1 mov 15, %o 1 call read mov 0, %o 0 call line nop b L 2 nop L 1: call read mov %i 0, %o 0 call close mov %i 0, %o 0 L 2: ret restore 12/16/2021 function (int a) { if (a < 0) { read(0, 15); line(); } else { read(a, 15); close(a); } } J. Giffin and S. Jha 37

Control Flow Graph Generation CFG ENTRY function: save %sp, 0 x 96, %sp cmp

Control Flow Graph Generation CFG ENTRY function: save %sp, 0 x 96, %sp cmp %i 0, 0 bge L 1 mov 15, %o 1 call read mov 0, %o 0 call line nop b L 2 nop L 1: call read mov %i 0, %o 0 call close mov %i 0, %o 0 L 2: ret restore 12/16/2021 bge call read call close call line ret CFG EXIT J. Giffin and S. Jha 38

Control Flow Graph Translation CFG ENTRY bge read close read line call read call

Control Flow Graph Translation CFG ENTRY bge read close read line call read call close call line ret CFG EXIT 12/16/2021 J. Giffin and S. Jha 39

Control Flow Graph Translation CFG ENTRY bge read close read line call read call

Control Flow Graph Translation CFG ENTRY bge read close read line call read call close call line ret CFG EXIT 12/16/2021 J. Giffin and S. Jha 40

A Interprocedural Model Generation read close line 12/16/2021 J. Giffin and S. Jha 41

A Interprocedural Model Generation read close line 12/16/2021 J. Giffin and S. Jha 41

A Interprocedural Model Generation read close line 12/16/2021 line write J. Giffin and S.

A Interprocedural Model Generation read close line 12/16/2021 line write J. Giffin and S. Jha 42

A Interprocedural Model Generation read close line 12/16/2021 line write J. Giffin and S.

A Interprocedural Model Generation read close line 12/16/2021 line write J. Giffin and S. Jha B line close 43

A read close 12/16/2021 Interprocedural Model Generation read line write J. Giffin and S.

A read close 12/16/2021 Interprocedural Model Generation read line write J. Giffin and S. Jha B line close 44

A read close 12/16/2021 Interprocedural Model Generation read B line write J. Giffin and

A read close 12/16/2021 Interprocedural Model Generation read B line write J. Giffin and S. Jha close 45

Possible Paths A read close 12/16/2021 read B line write J. Giffin and S.

Possible Paths A read close 12/16/2021 read B line write J. Giffin and S. Jha close 46

Possible Paths A read close 12/16/2021 read B line write J. Giffin and S.

Possible Paths A read close 12/16/2021 read B line write J. Giffin and S. Jha close 47

Impossible Paths A read close 12/16/2021 read B line write J. Giffin and S.

Impossible Paths A read close 12/16/2021 read B line write J. Giffin and S. Jha close 48

Impossible Paths A read close 12/16/2021 read B line write J. Giffin and S.

Impossible Paths A read close 12/16/2021 read B line write J. Giffin and S. Jha close 49

A read Adding Context Sensitivity read line B Y X write close Y close

A read Adding Context Sensitivity read line B Y X write close Y close X 12/16/2021 J. Giffin and S. Jha 50

PDA State Explosion • ε-edge identifiers maintained on a stack – Stack may grow

PDA State Explosion • ε-edge identifiers maintained on a stack – Stack may grow to be unbounded X • Solution: – Bound the maximum size of the runtime stack – A regular language overapproximation of the context-free language of the PDA 12/16/2021 J. Giffin and S. Jha 51

Data Flow Analysis function: save %sp, 0 x 96, %sp cmp %i 0, 0

Data Flow Analysis function: save %sp, 0 x 96, %sp cmp %i 0, 0 bge L 1 mov 15, %o 1 call read mov 0, %o 0 call line nop b L 2 nop L 1: call read mov %i 0, %o 0 call close mov %i 0, %o 0 L 2: ret restore 12/16/2021 Argument recovery • Statically known arguments constrain remote calls • Reduces opportunity given to attackers J. Giffin and S. Jha 52

Call Site Renaming function: save %sp, 0 x 96, %sp cmp $i 0, 0

Call Site Renaming function: save %sp, 0 x 96, %sp cmp $i 0, 0 bge L 1 mov 15, %o 1 call read mov 0, %o 0 call line nop b L 2 nop L 1: call read mov %i 0, %o 0 call close mov %i 0, %o 0 L 2: ret restore 12/16/2021 • Give each monitored call site a unique name • Associates arguments with call sites • Obfuscation • Reduces nondeterminism J. Giffin and S. Jha 53

Call Site Renaming function: save %sp, 0 x 96, %sp cmp $i 0, 0

Call Site Renaming function: save %sp, 0 x 96, %sp cmp $i 0, 0 bge L 1 mov 15, %o 1 call _638 mov 0, %o 0 call line nop b L 2 nop L 1: call read mov %i 0, %o 0 call close mov %i 0, %o 0 L 2: ret restore 12/16/2021 • Give each monitored call site a unique name • Associates arguments with call sites • Obfuscation • Reduces nondeterminism J. Giffin and S. Jha 54

Call Site Renaming function: save %sp, 0 x 96, %sp cmp $i 0, 0

Call Site Renaming function: save %sp, 0 x 96, %sp cmp $i 0, 0 bge L 1 mov 15, %o 1 call _638 mov 0, %o 0 call line nop b L 2 nop L 1: call _83 mov %i 0, %o 0 call close mov %i 0, %o 0 L 2: ret restore 12/16/2021 • Give each monitored call site a unique name • Associates arguments with call sites • Obfuscation • Reduces nondeterminism J. Giffin and S. Jha 55

Call Site Renaming function: save %sp, 0 x 96, %sp cmp $i 0, 0

Call Site Renaming function: save %sp, 0 x 96, %sp cmp $i 0, 0 bge L 1 mov 15, %o 1 call _638 mov 0, %o 0 call line nop b L 2 nop L 1: call _83 mov %i 0, %o 0 call _1920 mov %i 0, %o 0 L 2: ret restore 12/16/2021 • Give each monitored call site a unique name • Associates arguments with call sites • Obfuscation • Reduces nondeterminism J. Giffin and S. Jha 56

Call Site Renaming read close line 12/16/2021 • Give each monitored call site a

Call Site Renaming read close line 12/16/2021 • Give each monitored call site a unique name • Associates arguments with call sites • Obfuscation • Reduces nondeterminism J. Giffin and S. Jha 57

Call Site Renaming _638 _83 _1920 line 12/16/2021 • Give each monitored call site

Call Site Renaming _638 _83 _1920 line 12/16/2021 • Give each monitored call site a unique name • Associates arguments with call sites • Obfuscation • Reduces nondeterminism J. Giffin and S. Jha 58

Technical Challenges • Integrating other specification sources • Optimal null call insertion • C++

Technical Challenges • Integrating other specification sources • Optimal null call insertion • C++ vtable analysis 12/16/2021 J. Giffin and S. Jha 59

Intrusion Detection Somesh Jha University of Wisconsin IntrusionIntrusion Detection Somesh Jha University of Wisconsin Intrusion
Intrusion Detection Somesh Jha University of Wisconsin JIntrusion Detection Somesh Jha University of Wisconsin J
Intrusion Detection Somesh Jha University of Wisconsin JIntrusion Detection Somesh Jha University of Wisconsin J
Network Intrusion Detection System NIDS Somesh Jha NIDSNetwork Intrusion Detection System NIDS Somesh Jha NIDS
Intrusion Detection System 1 Intrusion Detection Intrusion detectionIntrusion Detection System 1 Intrusion Detection Intrusion detection
IPSecAn Overview Somesh Jha University of Wisconsin 1IPSecAn Overview Somesh Jha University of Wisconsin 1
IPSecAn Overview Somesh Jha University of Wisconsin 1IPSecAn Overview Somesh Jha University of Wisconsin 1
Intrusion Detection Arun Hodigere Intrusion and Intrusion DetectionIntrusion Detection Arun Hodigere Intrusion and Intrusion Detection
Intrusion Detection Arun Hodigere Intrusion and Intrusion DetectionIntrusion Detection Arun Hodigere Intrusion and Intrusion Detection
Two Techniques for Analyzing Malware Somesh Jha UniversityTwo Techniques for Analyzing Malware Somesh Jha University
Intrusion Detection System Intrusion Prevention System Topics IntrusionIntrusion Detection System Intrusion Prevention System Topics Intrusion
Malicious Code for Fun and Profit Somesh JhaMalicious Code for Fun and Profit Somesh Jha
Secure Programs via Gamebased Synthesis Somesh Jha TomSecure Programs via Gamebased Synthesis Somesh Jha Tom
IPSecAn Overview Somesh Jha 1 Outline why IPSecIPSecAn Overview Somesh Jha 1 Outline why IPSec
Software Security Issues in Embedded Systems Somesh JhaSoftware Security Issues in Embedded Systems Somesh Jha
Intrusion Detection Methods Intrusion detection is the processIntrusion Detection Methods Intrusion detection is the process
TEMA Intrusion Detection EBITEMA TEMA Intrusion Detection TemalineTEMA Intrusion Detection EBITEMA TEMA Intrusion Detection Temaline
Intrusion Detection Outline p Intrusion detection and computerIntrusion Detection Outline p Intrusion detection and computer
Intrusion Detection Systems IDS IDS Intrusion Detection SystemIntrusion Detection Systems IDS IDS Intrusion Detection System
Intrusion Detection Systems Network Intrusion Detection System NIDSIntrusion Detection Systems Network Intrusion Detection System NIDS
Intrusion Detection Systems Intrusion Detection Systems 1980 PaperIntrusion Detection Systems Intrusion Detection Systems 1980 Paper
HONEYPOTS An Intrusion Detection System Index Intrusion DetectionHONEYPOTS An Intrusion Detection System Index Intrusion Detection
INTRUSION DETECTION PREVENTION Iliandra Gonzalez INTRUSION DETECTION SYSTEMINTRUSION DETECTION PREVENTION Iliandra Gonzalez INTRUSION DETECTION SYSTEM
Intrusion Detection Methods Intrusion detection is the processIntrusion Detection Methods Intrusion detection is the process