Data Streaming in Computer Networking Cristian Estan George
- Slides: 30
Data Streaming in Computer Networking Cristian Estan, George Varghese University of California, San Diego
Talk structure l Traditional streaming in networking Rules of the game u Iteration paradigm: packet scheduling example u l New streaming problems Detecting malicious traffic u Understanding networkloads u June 8, 2003 Data streaming in computer networking - MPDS 2003 2
Internet service model Source port Destination port Source IP address Destination IP address Data Header Flow Internet June 8, 2003 Data streaming in computer networking - MPDS 2003 3
Traditional router functions ? IP Lookup Incoming 1 Outgoing 1 Incoming 2 Outgoing 2 Incoming 3 Outgoing 3 June 8, 2003 Data streaming in computer networking - MPDS 2003 4
Traditional router functions Out 2 IP Lookup Incoming 1 Outgoing 1 Incoming 2 Outgoing 2 Incoming 3 Outgoing 3 June 8, 2003 Data streaming in computer networking - MPDS 2003 5
Traditional router functions Incoming 1 Incoming 2 Incoming 3 June 8, 2003 Out 2 Out 3 Switching Out 3 Out 1 Out 2 Data streaming in computer networking - MPDS 2003 Outgoing 1 Outgoing 2 Outgoing 3 6
Traditional router functions Scheduling Incoming 1 Outgoing 1 Flow 2 Incoming 2 Flow 3 Incoming 3 June 8, 2003 Outgoing 2 Outgoing 3 Data streaming in computer networking - MPDS 2003 7
Traditional router functions Scheduling Incoming 1 Outgoing 1 Flow 2 Flow 3 Incoming 2 Outgoing 2 Incoming 3 Outgoing 3 June 8, 2003 Data streaming in computer networking - MPDS 2003 8
Rules of the game l Wire speed processing u u l What does this mean for algorithms? u u l At 40 gigabits/s 8 nanoseconds per packet - need fast SRAM Limited SRAM (say 32 megabits) but millions of flows Low worst case complexity bounds Low bounds on the amount of memory used Differences from databases u u u June 8, 2003 One pass vs. multiple passes Worst case vs. average case Small constants vs. asymptotic complexity Data streaming in computer networking - MPDS 2003 9
Talk structure l Traditional streaming in networking Rules of the game u Iteration paradigm: packet scheduling example u l New streaming problems Detecting malicious traffic u Understanding networkloads u June 8, 2003 Data streaming in computer networking - MPDS 2003 10
Iteration paradigm l l Many networking algorithms use iteration in time Way to allow multi-pass algorithms without storing input by assuming inputs do not change quickly Many examples (MULTOPS for Do. S detection [Gil 01], CSFQ for scheduling [Stoica 98]) Would be nice to formalize tradeoff between quality of results and drift rate of input June 8, 2003 Data streaming in computer networking - MPDS 2003 11
Example: Core Stateless FQ R R Mark rate R June 8, 2003 R If R>F drop with probability 1 -F/R Iteratively compute fair share F Data streaming in computer networking - MPDS 2003 12
Talk structure l Traditional streaming in networking Rules of the game u Iteration paradigm: packet scheduling example u l New streaming problems Detecting malicious traffic u Understanding networkloads u June 8, 2003 Data streaming in computer networking - MPDS 2003 13
New streaming problems l Detecting malicious activity u u u l Flooding (denial of service attacks) Worms Scans looking for vulnerable servers Understanding workloads u u u June 8, 2003 Billing Planning network growth Application mix Data streaming in computer networking - MPDS 2003 14
Detecting malicious traffic l Well defined building blocks u Detecting large aggregates » u Counting active flows in an aggregate » l Similar to iceberg queries Similar to counting distinct values Many open problems: e. g. detect worms and Do. S attacks (not clear what is right formal problem statement) June 8, 2003 Data streaming in computer networking - MPDS 2003 15
Talk structure l Traditional streaming in networking Rules of the game u Iteration paradigm: packet scheduling example u l New streaming problems Detecting malicious traffic u Understanding networkloads u June 8, 2003 Data streaming in computer networking - MPDS 2003 16
Informal problem definition Analysis Traffic reports Applications: 50% of traffic is Kazaa Terabytes of measurement data Sources: 20% of traffic comes from Steve’s PC June 8, 2003 Data streaming in computer networking - MPDS 2003 17
Informal problem definition Analysis Traffic reports 20% is Kazaa from Steve’s PC Terabytes of measurement data June 8, 2003 50% is Kazaa from the dorms Data streaming in computer networking - MPDS 2003 18
Formal problem definition l Define clusters: u u u l Threshold clusters: u l Atoms: fields 1 to n with hierarchies in each field including * Cluster: intersection of one set from each field hierarchy Example: Source=*, Destination=CS Net, App= Email Report traffic clusters above threshold T (e. g. 1% of traffic) Omit redundant clusters: u June 8, 2003 Compression rule: remove general clusters from report when its traffic can be inferred (up to error T) from on nonoverlapping more specific clusters Data streaming in computer networking - MPDS 2003 19
Solution status l The good: u u u l The bad: u u l Offline tool Auto. Focus; SIGCOMM 2003 paper Detected worm, busy servers, squid cache, etc. Network managers like it Takes long: 3 hours at T=0. 5% for one day trace Needs much memory 300 Mbytes The wanted: u June 8, 2003 Streaming algorithm - we invite improvements Data streaming in computer networking - MPDS 2003 20
Conclusions l l New rules: strict constraints on algorithms running in routers Iteration in time: can give simple algorithms, but needs more formalization as to quality of results General open problems: many challenges in detecting malicious traffic such as worms and Do. S attacks Specific open problem: computing traffic cluster reports in streaming fashion June 8, 2003 Data streaming in computer networking - MPDS 2003 21
Thank you! Alg ori thm s ? Databases g Ne June 8, 2003 in ork tw Data streaming in computer networking - MPDS 2003 22
Unidimensional clusters 15 10. 8. 0. 2 35 30 40 10. 8. 0. 3 10. 8. 0. 4 10. 8. 0. 5 June 8, 2003 160 35 110 10. 8. 0. 9 75 10. 8. 0. 10 10. 8. 0. 14 Data streaming in computer networking - MPDS 2003 23
Unidimensional clusters 10. 8. 0. 0/28 500 10. 8. 0. 0/29 120 10. 8/29 380 10. 8. 0. 0/30 50 10. 8. 0. 4/30 70 10. 8/30 305 75 10. 8. 0. 12/30 10. 8. 0. 2/31 50 10. 8. 0. 4/31 70 10. 8/31 270 10. 8. 0. 1 35 0/31 75 10. 8. 0. 14/31 15 10. 8. 0. 2 35 30 40 10. 8. 0. 3 10. 8. 0. 4 10. 8. 0. 5 June 8, 2003 160 35 110 10. 8. 0. 9 75 10. 8. 0. 10 10. 8. 0. 14 Data streaming in computer networking - MPDS 2003 24
Unidimensional clusters 10. 8. 0. 0/28 500 10. 8. 0. 0/29 120 10. 8/29 380 10. 8. 0. 0/30 50 10. 8. 0. 4/30 70 10. 8/30 305 75 10. 8. 0. 12/30 10. 8. 0. 2/31 50 10. 8. 0. 4/31 70 10. 8/31 270 10. 8. 0. 1 35 0/31 75 10. 8. 0. 14/31 15 10. 8. 0. 2 35 30 40 10. 8. 0. 3 10. 8. 0. 4 10. 8. 0. 5 June 8, 2003 160 35 110 10. 8. 0. 9 75 10. 8. 0. 10 10. 8. 0. 14 Data streaming in computer networking - MPDS 2003 25
Unidimensional clusters 10. 8. 0. 0/28 500 10. 8. 0. 0/29 120 10. 8/29 380 10. 8/30 305 10. 8/31 270 160 110 10. 8. 0. 9 June 8, 2003 Data streaming in computer networking - MPDS 2003 26
Unidimensional clusters 10. 8. 0. 0/28 500 10. 8. 0. 0/29 120 10. 8/29 380 10. 8/30 305 10. 8/31 270 160 110 10. 8. 0. 9 June 8, 2003 Data streaming in computer networking - MPDS 2003 27
Multidimensional clusters l Two dimensions u u Source network Protocol (traffic type) l Trees turn into lattice l Multiple parents l Nodes overlap June 8, 2003 Data streaming in computer networking - MPDS 2003 28
Offline solution June 8, 2003 Data streaming in computer networking - MPDS 2003 29
Sample report June 8, 2003 Data streaming in computer networking - MPDS 2003 30
- Traditional network vs sdn
- Konsep stream
- Network streaming telemetry
- Reno computer networking
- How to calculate subnets and hosts
- Lan basics
- An engineering approach to computer networking
- Computer networking terms
- Evolution of computer networking
- Hotmail
- Analog and digital signals in computer networking
- Computer networks kurose
- Computer networking
- "pearson aida"
- An engineering approach to computer networking
- "computer networking"
- George washington x king george iii
- Venn diagram ghent
- Dr gagiu cristian
- Dr cristian lara
- Aparatul de proiectie instrument optic
- Christian secchi
- Cristi ignat
- Cristian dragos
- Authentication certificate format
- Cristian varela psicologo
- Cristian torti
- Phase matching
- Cristian david ocampo
- Imagina cristian castro
- Cristian garbin