Automated Worm Fingerprinting Sumeet Singh Cristian Estan George
- Slides: 88
Automated Worm Fingerprinting Sumeet Singh, Cristian Estan, George Varghese, and Stefan Savage
Introduction n n Problem: how to react quickly to worms? Code. Red 2001 ¡ n Infected ~360, 000 hosts within 11 hours Sapphire/Slammer (376 bytes) 2002 ¡ Infected ~75, 000 hosts within 10 minutes
Existing Approaches n Detection ¡ n Ad hoc intrusion detection Characterization ¡ Manual signature extraction n Isolates and decompiles a new worm Look for and test unique signatures Can take hours or days
Existing Approaches n Containment ¡ Updates to anti-virus and network filtering products
Earlybird n n Automatically detect and contain new worms Two observations ¡ ¡ Some portion of the content in existing worms is invariant Rare to see the same string recurring from many sources to many destinations
Earlybird n Automatically extract the signature of all known worms ¡ n Also Blaster, My. Doom, and Kibuv. B hours or days before any public signatures were distributed Few false positives
Background and Related Work n Almost all IPs were scanned by Slammer < 10 minutes ¡ Limited only by bandwidth constraints
The SQL Slammer Worm: 30 Minutes After “Release” - Infections doubled every 8. 5 seconds - Spread 100 X faster than Code Red - At peak, scanned 55 million hosts per second.
Network Effects Of The SQL Slammer Worm n At the height of infections ¡ ¡ ¡ Several ISPs noted significant bandwidth consumption at peering points Average packet loss approached 20% South Korea lost almost all Internet service for period of time Financial ATMs were affected Some airline ticketing systems overwhelmed
Signature-Based Methods n Pretty effective if signatures can be generated quickly ¡ ¡ For Code. Red, 60 minutes For Slammer, 1 – 5 minutes
Worm Detection n Three classes of methods ¡ ¡ ¡ Scan detection Honeypots Behavioral techniques
Scan Detection n n Look for unusual frequency and distribution of address scanning Limitations ¡ Not suited to worms that spread in a nonrandom fashion (i. e. emails, IM, P 2 P apps) n n Based on a target list Spread topologically
Scan Detection n More limitations ¡ ¡ Detects infected sites Does not produce a signature
Honeypots n Monitored idle hosts with untreated vulnerabilities ¡ n Used to isolate worms Limitations ¡ ¡ Manual extraction of signatures Depend on quick infections
Behavioral Detection n Looks for unusual system call patterns ¡ ¡ n Sending a packet from the same buffer containing a received packet Can detect slow moving worms Limitations ¡ ¡ Needs application-specific knowledge Cannot infer a large-scale outbreak
Characterization n n Process of analyzing and identifying a new worm Current approaches ¡ ¡ Use a priori vulnerability signatures Automated signature extraction
Vulnerability Signatures n Example ¡ Slammer Worm n n UDP traffic on port 1434 that is longer than 100 bytes (buffer overflow) Can be deployed before the outbreak ¡ Can only be applied to well-known vulnerabilities
Some Automated Signature Extraction Techniques n Allows viruses to infect decoy programs ¡ ¡ Extracts the modified regions of the decoy Uses heuristics to identify invariant code strings across infected instances
Some Automated Signature Extraction Techniques n Limitation ¡ Assumes the presence of a virus in a controlled environment
Some Automated Signature Extraction Techniques n Honeycomb ¡ n Autograph ¡ n Find longest common subsequences among sets of strings found in messages Uses network-level data to infer worm signatures Limitations ¡ Scale and full distributed deployments
Containment n Mechanism used to deter the spread of an active worm ¡ Host quarantine n ¡ ¡ Via IP ACLs on routers or firewalls String-matching Connection throttling n On all outgoing connections
Host Quarantine n Preventing an infected host from talking to other hosts ¡ Via IP ACLs on routers or firewalls
Defining Worm Behavior n Content invariance ¡ n Content prevalence ¡ n Portions of a worm are invariant (e. g. the decryption routine) Appears frequently on the network Address dispersion ¡ Distribution of destination addresses more uniform to spread fast
Finding Worm Signatures n Traffic pattern is sufficient for detecting worms ¡ ¡ ¡ Relatively straightforward Extract all possible substrings Raise an alarm when n Frequency. Counter[substring] > threshold 1 Source. Counter[substring] > threshold 2 Dest. Counter[substring] > threshold 3
Practical Content Sifting n Characteristics ¡ ¡ ¡ Small processing requirements Small memory requirements Allows arbitrary deployment strategies
Estimating Content Prevalence n Finding the packet payloads that appear at least x times among the N packets sent ¡ During a given interval
Estimating Content Prevalence n Table[payload] ¡ n 1 GB table filled in 10 seconds Table[hash[payload]] ¡ ¡ ¡ 1 GB table filled in 4 minutes Tracking millions of ants to track a few elephants Collisions. . . false positives
Multistage Filters stream memory Array of counters Hash(Pink) [Singh et al. 2002]
Multistage Filters packet memory Array of counters Hash(Green)
Multistage Filters packet memory Array of counters Hash(Green)
Multistage Filters packet memory
Multistage Filters Collisions are OK packet memory
Multistage Filters Reached threshold packet memory packet 1 1 Insert
Multistage Filters packet memory packet 1 1
Multistage Filters packet memory packet 1 1 packet 2 1
Multistage Filters Stage 1 packet memory packet 1 1 No false negatives! (guaranteed detection) Stage 2
Conservative Updates Gray = all prior packets
Conservative Updates Redundant
Conservative Updates
Detecting Common Strings n n Cannot afford to detect all substrings Maybe can afford to detect all strings with a small fixed length
Detecting Common Strings n n Cannot afford to detect all substrings Maybe can afford to detect all strings with a small fixed length A horse is a horse, of course F 1 = (c 1 p 4 + c 2 p 3 + c 3 p 2 + c 4 p 1 + c 5) mod M
Detecting Common Strings n n Cannot afford to detect all substrings Maybe can afford to detect all strings with a small fixed length F 2 = (c 2 p 4 + c 3 p 3 + c 4 p 2 + c 5 p 1 + c 6) mod M A horse is a horse, of course F 1 = (c 1 p 4 + c 2 p 3 + c 3 p 2 + c 4 p 1 + c 5) mod M
Detecting Common Strings n n Cannot afford to detect all substrings Maybe can afford to detect all strings with a small fixed length F 2 = (c 2 p 4 + c 3 p 3 + c 4 p 2 + c 5 p 1 + c 6) mod M = (c 1 p 5 + c 2 p 4 + c 3 p 3 + c 4 p 2 + c 5 p 1 + c 6 - c 1 p 5) mod M = (p. F 1 + c 6 - c 1 p 5) mod M A horse is a horse, of course F 1 = (c 1 p 4 + c 2 p 3 + c 3 p 2 + c 4 p 1 + c 5) mod M
Detecting Common Strings n n Cannot afford to detect all substrings Maybe can afford to detect all strings with a small fixed length ¡ Still too expensive…
Estimating Address Dispersion n Not sufficient to count the number of source and destination pairs ¡ e. g. send a mail to a mailing list n n n Two sources—mail server and the sender Many destinations Need to count the unique source and destination traffic flows ¡ For each substring
Bitmap counting – direct bitmap Set bits in the bitmap using hash of the flow ID of incoming packets HASH(green)=10001001 [Estan et al. 2003]
Bitmap counting – direct bitmap Different flows have different hash values HASH(blue)=00100100
Bitmap counting – direct bitmap Packets from the same flow always hash to the same bit HASH(green)=10001001
Bitmap counting – direct bitmap Collisions OK, estimates compensate for them HASH(violet)=10010101
Bitmap counting – direct bitmap HASH(orange)=11110011
Bitmap counting – direct bitmap HASH(pink)=11100000
Bitmap counting – direct bitmap As the bitmap fills up, estimates get inaccurate HASH(yellow)=01100011
Bitmap counting – direct bitmap Solution: use more bits HASH(green)=10001001
Bitmap counting – direct bitmap Solution: use more bits Problem: memory scales with the number of flows HASH(blue)=00100100
Bitmap counting – virtual bitmap Solution: a) store only a portion of the bitmap b) multiply estimate by scaling factor
Bitmap counting – virtual bitmap HASH(pink)=11100000
Bitmap counting – virtual bitmap Problem: estimate inaccurate when few flows active HASH(yellow)=01100011
Bitmap counting – multiple bmps Solution: use many bitmaps, each accurate for a different range
Bitmap counting – multiple bmps HASH(pink)=11100000
Bitmap counting – multiple bmps HASH(yellow)=01100011
Bitmap counting – multiple bmps Use this bitmap to estimate number of flows
Bitmap counting – multiple bmps Use this bitmap to estimate number of flows
Bitmap counting – multires. bmp OR OR Problem: must update up to three bitmaps per packet Solution: combine bitmaps into one
Bitmap counting – multires. bmp HASH(pink)=11100000
Bitmap counting – multires. bmp HASH(yellow)=01100011
Multiresolution Bitmaps n n Still too expensive to scale Scaled bitmap ¡ ¡ Recycles the hash space with too many bits set Adjusts the scaling factor according n E. g. , 1 bit represents 2 flows as opposed to a single flow
Too CPU-Intensive n A packet with 1, 000 bytes of payload ¡ n Needs 960 fingerprints for string length of 40 Prone to Denial-of-Service attacks
CPU Scaling n Obvious approach: sampling - Random sampling may miss many substrings n Solution: value sampling ¡ Track only certain substrings n ¡ e. g. last 6 bits of fingerprint are 0 P(not tracking a worm) = P(not tracking any of its substrings)
CPU Scaling n Example ¡ ¡ ¡ Track only substrings with last 6 bits = 0 s String length = 40 1, 000 char string n n ¡ 960 substrings 960 fingerprints ‘ 11100… 101010’…‘ 10110… 000000’… Use only ‘xxxxx…. 000000’ as signatures n Probably 960 / 26 = 15 signatures
CPU Scaling n n n P(finding a 100 -byte signature) = 55% P(finding a 200 -byte signature) = 92% P(finding a 400 -byte signature) = 99. 64%
Putting It Together Address Dispersion Table key header src cnt dest cnt payload substring fingerprints AD entry exist? substring fingerprints update counters else update counter key cnt counters > dispersion threshold? report key as suspicious worm cnt > prevalence threshold? create AD entry Content Prevalence Table
Putting It Together n n n Sample frequency: 1/64 String length: 40 Use 4 hash functions to update prevalence table ¡ Multistage filter reset every 60 seconds
System Design n Two major components ¡ Sensors n n ¡ Sift through traffic for a given address space Report signatures An aggregator n n Coordinates real-time updates Distributes signatures
Implementation and Environment n n Written in C and My. SQL (5, 000 lines) rrd-tools library for graphical reporting PHP scripting for administrative control Prototype executes on a 1. 6 Ghz AMD Opteron 242 1 U Server ¡ Linux 2. 6 kernel
Early. Bird n n Processes 1 TB of traffic per day Can keep up with 200 Mbps of continuous traffic
Parameter Tuning n Prevalence threshold: 3 ¡ n Very few signatures repeat Address dispersion threshold ¡ ¡ ¡ 30 sources and 30 destinations Reset every few hours Reduces the number of reported signatures down to ~25, 000
Parameter Tuning n Tradeoff between and speed and accuracy ¡ Can detect Slammer in 1 second as opposed to 5 seconds n With 100 x more reported signatures
Performance n n 200 Mbps Can be pipelined and parallelized for achieve 40 Gbps
Memory Consumption n Prevalence table ¡ 4 stages n ¡ n 2 MB total Address dispersion table ¡ ¡ n Each with ~500, 000 bins (8 bits/bin) 25 K entries (28 bytes each) < 1 MB Total: < 4 MB
Trace-Based Verification n Two main sources of false positives ¡ 2, 000 common protocol headers n n ¡ ¡ e. g. HTTP, SMTP Whitelisted SPAM e-mails Bit. Torrent n Many-to-many download
False Negatives n n So far none Detected every worm outbreak
Inter-Packet Signatures n n An attacker might evade detection by splitting an invariant string across packets With 7 MB extra, Early. Bird can keep per flow states and fingerprint across packets
Live Experience with Early. Bird n Detected precise signatures ¡ ¡ Code. Red variants My. Doom mail worm Sasser Kibvu. B
Variant Content n Polymorphic viruses ¡ ¡ Semantically equivalent but textually distinct code Invariant decoding routine
Extensions n n Self configuration Slow worms
Containment n How to handle false positives? ¡ ¡ If too aggressive, Early. Bird becomes a target for Do. S attacks An attacker can fool the system to block a target message
Coordination n Trust of deployed servers Validation Policy
Conclusions n Early. Bird is a promising approach ¡ ¡ ¡ n To detect unknown worms real-time To extract signatures automatically To detect SPAMs with minor changes Wire-speed signature learning is viable
- Dr frost compound measures
- Veptr complications
- Sarwan singh kundan singh
- Pharynx function earthworm
- Dna fingerprinting lesson plan
- Vntr vs str in dna fingerprinting
- Os fingerprinting
- Flaviano c. guerrero
- Gcat dna
- Dna fingerprinting minilab answers
- Fingerprinting
- Who ate the cheese dna fingerprinting lab
- What is sperm dna fingerprinting?
- Dhcp fingerprinting
- Basic and composite ridge characteristics
- Crossover ridge fingerprint
- How many bands does luna have in her fingerprint
- Dna fingerprinting
- Example 2
- Passive os fingerprinting daemon
- Lausd fingerprinting
- Dna fingerprinting zanichelli
- Biorad dna fingerprinting
- Conclusion of dna fingerprinting
- Jla badge
- Intellectually astute
- First fundamental principle of fingerprints
- Venn diagram ghent
- George washington vs king george iii
- Cristian duarte edad
- Mira fusil
- Cristian manzoni polimi
- Instrumente optice care dau imagini virtuale
- Cristian massimi unibo
- Cristian valle ronceros
- App1store
- Posicion del estomago
- Cristian solano
- Cristian david ocampo
- Cristian viscione
- Cristian david ocampo
- Dr gagiu cristian
- Cristian secchi
- Physical clocks
- Cristian fugaciu
- Cawarfidae
- Cristian varela psicologo
- Berkeley algorithm example
- Cristian presura inventii
- Hans cristian andersen
- Imagina cristian castro letra
- Cristian ignat
- Cristian nacu
- Hola amor que haces
- Sidicoweb
- Cristian torti
- Dr cristian lara
- Cristian cadar
- Cristian aste
- Hans cristian andersen
- Cristian garbin
- Cristian dragos
- Desinencia de correr
- Leon que palabra es aguda grave o esdrujula
- De que estan fets els ossos
- Las mariposas estan tristes
- Los seres vivos están formados por
- Es el día del examen final. pablo y rosa están
- ¿qué haces tú con el libro de viajes? (leer)
- Formas.no personales del verbo
- Como estan clasificados los valores
- En una rueda dela fortuna las canastillas estan numeradas
- Proteinas como estan formadas
- Estoy estas estamos están
- Funciones de la proteina
- Ejemplos de metas de comprensión
- Estar en el futuro
- No creo en las personas
- Donde estan los otros nueve
- Hay un manzano muy
- Lista de dones de una persona
- Sistema oseo
- A ti que habitas en el cielo
- Como estan distribuidos los electrones
- Donde hay dos o tres reunidos en mi nombre
- Les amistats estan fetes de trossets
- Bioarticion
- Eteres
- Que estan haciendo?