Flow Scan A Network Traffic Reporting and Visualization

Flow. Scan A Network Traffic Reporting and Visualization Tool Dave Plonka plonka@doit. wisc. edu

Presentation Overview Introduction " " Flow. Scan's Functionality " Hardware & Software Components Sample Graphs " " " Short & Long Term Analyses, Events " Graphs by Autonomous Systems, Top ASNs " Sub. Net. IO graphs References

Flow. Scan A Network Traffic Reporting and Visualization Tool " " Flow. Scan is a software package for open systems that is freely available under the terms of the GNU General Public License. Flow. Scan analyzes and reports on flow data exported by Internet Protocol routers. Flow. Scan produces graph images which provide a continuous, near real-time view of the network traffic across a network's border. Development since December 1998. Beta release in September 1999. Released March 2000.

Background on Flows & Cisco Net. Flow " The notion of flow profiling was introduced by the research community " Today, for performance and accounting reasons, flow profiling is built into some networking devices " Not yet standards-based " Flow. Scan utilizes flows defined and exported by Cisco's Net. Flow feature. Essentially using the definition introduced by [Claffy. PB]. " By this definition, an IP flow is a unidirectional series of IP packets of a given protocol, traveling between a source and destination, within a certain period of time.

Sample Flows ncftp GET session

Background on Flows & Cisco Net. Flow " Diagram by Daniel W. Mc. Robb, from the cflowd configuration documentation, 1998 -1999.

Flow. Scan's Functionality " Flow. Scan examines each flow and maintains counters based upon that flow's classification " Flow. Scan periodically reports what it finds into databases. Each database contains packet, byte, and flow counters " Counters are maintained based on these flow attributes: " IP protocol such as ICMP, TCP, and UDP " well-known service or application such as ftp-data, ftp, smtp, nntp, http, Real. Media, Quake, and Napster " the class A, B, C network, or CIDR block in which a "local" IP address resides " the AS (Autonomous System) pair between which the represented traffic was exchanged

Flow. Scan's Functionality

Flow. Scan Hardware Components " Works with most Cisco routers " Compatibility with Juniper's routers and River. Stone's Switch Router (formerly Cabletron's SSR) is being developed " Most Flow. Scan systems are Sun SPARC Solaris machines or Intel GNU/Linux or BSD machines " The fastest Flow. Scan machines appear to be multiprocessor Intel PIII machines " GIF or PNG image files suitable for any web server, we use Apache

Flow. Scan Hardware Components

Flow. Scan Software Components " Perl " Flowscan script " Perl modules " Campus. IO report " Patched cflowd " Sub. Net. IO report " RRDtool " Unix or GNU/Linux " Cron " Make

Software

Short Term Analysis " Graphs over a short, recent time frame are based upon five-minute intervals. " Network abuse, such as flood-based Denial of Service attacks, are easily visible as "stalagmites" and "stalactites". These would be hidden in coarser-grained long-term graphs " This Example: " Flood of outbound 40 -byte TCP RST reply packets " Flood of inbound 40 -byte TCP ACK packets " Resulted in as much as 10, 000 flows per second

Short Term Analysis

Short Term Analysis Bits, Packets, Flows Graphs 48 hours, 4 -6 Nov 2000 " 2000/11/05 ~0200 -> ~1000 Apparently peering w/Abilene was down. (This was due to changes at AADS) " 2000/11/05 ~0415 -> ~1100 outbound flood of UDP packets ~10, 000 packets per second " 2000/11/05 ~0800, ~0830 inbound flood of 1500 byte ICMP ECHO and ECHOREPLY packets destined for a campus dial-up user. This amounted to as much as 25 Mb/s. " 2000/11/05 ~1400 -> ? Apparently peering w/Abilene was down again. Star. TAP too. (More problems at AADS) " 2000/11/06 ~0730 AADS got things back together connectivity to Abilene and Star. TAP restored.

Campus. IO ISP Traffic, 10 -11 NOV 2000 " Graph by Alexander Kunz <Alexander. Kunz@nextra. de>, 2000.

Campus. IO University of Wisconsin - Parkside 10 -11 Nov 2000 " Graph by Steven Premeau <premeau@uwp. edu>, 2000.

Long Term Analysis " Daily average graphs aid capacity planning and traffic shaping efforts. " This example: " Graph produced 2000/09/21 over past 550 days " academic calendar dramatically influences the traffic levels, but only to and from Res. Net. " increase in outbound ftp traffic from the Computer Sciences department within the past year. " outbound traffic has consistently exceeded our inbound traffic level, the discrepancy between the two appears to be increasing.

Campus. IO Long Term Analysis 550 days prior to 21 Sep 2000

Campus. IO Napster Daily Averages March Through September 2000 " Note that these are daily averages, five minute peak Napster traffic would be higher " Note two "horns" or spikes in late March and Septemember. These represent some of the highest outbound daily averages observed and will be explored in the subsequent slides.

Campus. IO Napster Daily Averages March Through September 2000

Campus. IO Events Red. Hat 6. 2 Release C. Wednesday 29 Mar 2000 " Spent an hour or two investigating increased CS traffic before coming in that morning " Found traffic to be TCP on ports >1024, host addresses indicated that it was likely to be PASV mode ftp data " Jump was from ~5 Mb/s to ~30 Mb/s " David Parter of CS informed me that their Red. Hat mirror was made active about that

Campus. IO Events Red. Hat 6. 2 Release c. Wednesday 29 Mar 2000

Campus. IO Events Red. Hat 7 Release "Black" Monday, 25 Sep 2000 " PASV mode ftp detection built-into Campus. IO by this time " Jump from 5 -10 Mb/s to 50 -60 Mb/s for CS; another Red. Hat mirror is in the "blue", Student Information Technology " Notice flat-topping in daily peaks. This is due to the hitting capacity of Wisc. Net's commodity internet connectivity to Chicago " at capacity of upstream links for nearly entire days

Campus. IO Events Red. Hat 7 Release "Black" Monday, 25 Sep 2000

Campus. IO Events "All in 2 day's work" Monday & Tuesday, 23 -24 Oct 2000 " Note arrow of time and events occur left to right: " 2000/10/03 0500 peer router upgrade, RSP 4 -> RSP 8, OC 3 -> OC 12 " 2000/10/03 1525 campus to peer cutover from OC 3 to OC 12 " 2000/10/03 1915 experimenting with rate-limits " 2000/10/04 1100 napster. com outage? " 2000/10/04 1615 48 -byte TCP inbound Do. S flood " 2000/10/04 1830 Res. Net -> world rate-limit applied " 2000/10/04 2100 40 -byte TCP SYN outbound Do. S flood

Campus. IO Events "All in 2 day's work" Monday & Tuesday, 23 -24 Oct 2000

Campus. IO Events "All in 2 day's work" Monday & Tuesday, 23 -24 Oct 2000 " A method to visualize "events" and correlate real-world incidents with automated measurement " Working on a generalized approach for instrumenting the Internet to provide this sort of info to sites and researchers

Campus. IO Events "All in 2 day's work" Monday & Tuesday, 23 -24 Oct 2000

Campus. IO ASNs UW-Madison Peers " There is the need in large networks to determine the amount of traffic that each other Autonomous System (AS) sources, sinks, or carries for your institution " These information is used to make informed peering and provisioning decisions " UW-Madison peers with many others, most of our traffic is passed to Wisc. Net and Abilene

Campus. IO ASNs UW-Madison Peers Wednesday & Thursday, 1 -2 Nov 2000

Campus. IO ASNs Top Origin ASNs

Campus. IO ASNs Top "Path" ASNs

Sub. Net. IO Report " Sub. Net. IO is another "canned" Flow. Scan report " It is derived from Campus. IO; It reports traffic to and from campus done by individual subnets " These examples: " Wisc. World 33. 6 K and 56 K bps dial pool traffic; note inbound Do. S attack to at about 3 PM " Do. IT DSL service rivals the amount of traffic with only a fraction of the number of users; graphs is more erratic because of the smaller population of users

Sub. Net. IO Wednesday & Thursday, 1 -2 Nov 2000

Flow. Scan Credits & Thanks " Daniel Mc. Robb and CAIDA for cflowd " Tobi Oetiker and CAIDA for RRDtool " Perl authors and developers for perl and CPAN " Free Software Foundation for GNU " UW-Madison Do. IT's Network Operations and Network Engineering Technology groups for mentoring and support

Flow. Scan A Network Traffic Reporting and Visualization Tool http: //net. doit. wisc. edu /~plonka/Flow. Scan/