Network Traffic Measurement and Modeling Carey Williamson Department

Network Traffic Measurement and Modeling Carey Williamson Department of Computer Science University of Calgary 1

Network Traffic Measurement u. A recent focus of networking research (mid to late 1980’s, early 1990’s) u Collect data or packet traces showing packet activity on the network for different network applications 2

Purpose u Understand the traffic characteristics of existing networks u Develop models of traffic for future networks u Useful for simulations, planning studies 3

Requirements u Network measurement requires hardware or software measurement facilities that attach directly to network u Allows you to observe all packet traffic on the network, or to filter it to collect only the traffic of interest u Assumes broadcast-based network technology, superuser permission 4

Measurement Tools u Can be classified into hardware and software measurement tools u Hardware: specialized equipment u Examples: HP 4972 LAN Analyzer, Data. General Network Sniffer, others. . . u Software: special software tools u Examples: tcpdump, xtr, SNMP, others. . . 5

Measurement Tools (Cont’d) u Measurement tools can also be classified as intrusive or non-intrusive u Intrusive: the monitoring tool generates traffic of its own during data collection u Non-intrusive: the monitoring tool is passive, observing and recording traffic info, while generating none of its own 6

Measurement Tools (Cont’d) u Measurement tools can also be classified as real-time or non-real-time u Real-time: collects traffic data as it happens, and may even be able to display traffic info as it happens u Non-real-time: collected traffic data may only be a subset (sample) of the total traffic, and is analyzed off-line (later) 7

Potential Uses of Tools u Protocol debugging u Network debugging and troubleshooting u Changing network configuration u Designing, testing new protocols u Designing, testing new applications u Detecting network weirdness: broadcast storms, routing loops, etc. 8

Potential Uses of Tools (Cont’d) u Performance evaluation of protocols and applications u How protocol/application is being used u How well it works u How to design it better 9

Potential Uses of Tools (Cont’d) u Workload characterization u What traffic is generated u Packet size distribution u Packet arrival process u Burstiness u Important in the design of networks, applications, interconnection devices, congestion control algorithms, etc. 10

Potential Uses of Tools (Cont’d) u Workload modeling u Construct synthetic workload models that concisely capture the salient characteristics of actual network traffic u Use as representative, reproducible, flexible, controllable workload models for simulations, capacity planning studies, etc. 11

Measurement Environments u Local Area Networks (LAN’s) u e. g. , u Wide Area Networks (WAN’s) u e. g. , u ATM Ethernet LANs the Internet Networks 12

Some References u Raj Jain, ‘‘Packet Trains”, 1986 u Cheriton and Williamson, “VMTP”, 1987 u Chiu and Sudama, “DECNET Applications and Protocols”, 1988 u Gusella, “Diskless Workstations”, 1990 u Caceres, Danzig, Jamin, Mitzel, “Wide Area TCP/IP Traffic”, 1991 13

Some References (Cont’d) u Paxson, “Measurements and Models of Wide Area TCP Traffic”, 1991 u Leland et al, “Self-Similarity”, 1993 u Garrett, Willinger, “VBR Video”, 1994 u Paxson, “Failure of Poisson Modeling”, 1994 14

Summary of Measurement Results u The following represents my own synopsis of the “Top 10 Observations” from network measurement and monitoring research in the last 10 years u Not an exhaustive list, but hits most of the highlights u For more detail, see papers (or ask!) 15

Observation #1 u The traffic model that you use is extremely important in the performance evaluation of routing, flow control, and congestion control strategies u Have to consider application-dependent, protocol-dependent, and networkdependent characteristics u The more realistic, the better (GIGO) 16

Observation #2 u Characterizing aggregate network traffic is difficult u Lots of (diverse) applications u Just a snapshot: traffic mix, protocols, applications, network configuration, technology, and users change with time 17

Observation #3 u Packet arrival process is not Poisson u Packets travel in trains u Packets travel in tandems u Packets get clumped together (ack compression) u Interarrival times are not exponential u Interarrival times are not independent 18

Observation #4 u Packet traffic is bursty u Average utilization may be very low u Peak utilization can be very high u Depends on what interval you use!! u Traffic may be self-similar: bursts exist across a wide range of time scales u Defining burstiness (precisely) is difficult 19

Observation #5 u Traffic is non-uniformly distributed amongst the hosts on the network u Example: 10% of the hosts account for 90% of the traffic (or 20 -80) u Why? Clients versus servers, geographic reasons, popular ftp sites, web sites, etc. 20

Observation #6 u Network traffic exhibits ‘‘locality’’ effects u Pattern is far from random u Temporal locality u Spatial locality u Persistence and concentration u True at host level, at gateway level, at application level 21

Observation #7 u Well over 80% of the byte and packet traffic on most networks is TCP/IP u By far the most prevalent u Often as high as 95 -99% u Most studies focus only on TCP/IP for this reason (as they should!) 22

Observation #8 u Most conversations are short u Example: 90% of bulk data transfers send less than 10 kilobytes of data u Example: 50% of interactive connections last less than 90 seconds u Distributions may be ‘‘heavy tailed’’ (i. e. , extreme values may skew the mean and/or the distribution) 23

Observation #9 u Traffic is bidirectional u Data usually flows both ways u Not JUST acks in the reverse direction u Usually asymmetric bandwidth though u Pretty much what you would expect from the TCP/IP traffic for most applications 24

Observation #10 u Packet size distribution is bimodal u Lots of small packets for interactive traffic and acknowledgements u Lots of large packets for bulk data file transfer type applications u Very few in between sizes 25

Summary u There has been lots of interesting network measurement work in the last ten years u We will take a look at some of it soon u LAN, WAN, and Video measurements u Network traffic self-similarity 26