Monitoring very high speed links Gianluca Iannaccone Sprint

Monitoring very high speed links Gianluca Iannaccone Sprint ATL joint work with: Christophe Diot – Sprint ATL Ian Graham – University of Waikato Nick Mc. Keown – Stanford University November 2 nd, 2001 IMW 2001

Packet-level trace collection: the DAG example • Optical splitter to divert part of the signal toward the capture card. • The capture card stores a record with a timestamp for each packet. • The card uses the PCI bus to transfer the records to the host main memory. • It is used by the Sprint IP Monitoring Project. 2 November 2 nd, 2001 IMW 2001

Challenges for next generation capture devices • PCI bus throughput: – PCI 66 Mhz/64 bit is already challenged at OC-48 speed. • Storage capacity and data management: – Several terabytes per trace collection. • Memory access speed. • Disk array speed: – OC-192 would require roughly 250 Mbytes/sec (64 byte records, 300 byte average packet size). 3 November 2 nd, 2001 IMW 2001

Challenges for next generation capture devices (cont’d) • Deployment of extensive monitoring facilities inside the routers: – Backplane speeds are in the order of hundreds of Gbps; – Stringent power and space constraints; – Switched backplanes reduce the advantage of being inside the router. 4 November 2 nd, 2001 IMW 2001

Our Goals • Be capable of collecting packet at OC-192 speed with the current PCI technology. • Develop a 4: 1 compression technique. • Minimize the loss of information due to compression. 5 November 2 nd, 2001 IMW 2001

System requirements • Online processing required for compression. • At OC-192 speed (10 Gbps), a new packet arrives every 240 ns (300 -byte packets). – We can assume large average packet sizes because we can buffer incoming packets (once timestamped). 6 November 2 nd, 2001 IMW 2001

From packet traces to flow traces • Store information at the flow-level: – a flow is defined by the usual 5 -tuple. • Flow information are shared among all the packets belonging to the same flow. • Store few information per packet: – packet arrival time; – size; – sequence numbers –… 7 November 2 nd, 2001 IMW 2001

Flow and packet records TCP: 28 bytes/flow; 16 bytes/pkt; UDP: 20 bytes/flow; 12 bytes/pkt; Other: 16 bytes/flow; 12 bytes/pkt; 8 November 2 nd, 2001 IMW 2001

Flow termination • Records of terminated flows are dumped to the host memory (and then to disk). • Flow termination relies on a timeout: – A short timeout will require less memory; – A long timeout will give better compression rates; – Anyway, the timeout does not affect the accuracy of the system because we can postprocess the trace; 9 November 2 nd, 2001 IMW 2001

Flow fragmentation • Long-lived flows can be a problem: – Large records and high bursts of traffic on the PCI bus; • We set a limit to memory occupancy for a flow record (with relative packet records): – If the memory limit is exceeded, the Record Number is increased and the record is stored. – When the flow terminates the entire record is stored to disk with the Last Record flag set. 10 November 2 nd, 2001 IMW 2001

Memory required 11 November 2 nd, 2001 IMW 2001

Preliminary performance study • We have built a simple emulator to compress a packet-level trace. • 5 different traces with different link utilizations and traffic patterns. • Compression rates ranging between 3. 20 and 4. 02 (compared to DAG 3 packet traces). 12 November 2 nd, 2001 IMW 2001

Hardware architecture • Steps required in trace collection: – process framing information; – timestamp the packet; – classify the packet; – update flow/packet record; – store terminated flows to disk (background); • All these steps can be pipelined: – Each step can be performed during the interval between two packet arrivals. 13 November 2 nd, 2001 IMW 2001

Packet classification • May be very expensive in terms of computation time and resources: – One-to-one mapping between packets and flows. • But a simple hash function may be enough: – Requires a large amount of fast memory; – Collisions can be solved using a second hash function or a lookup trie. 14 November 2 nd, 2001 IMW 2001

Conclusion • New compression techniques are necessary to monitor OC-192/OC-768 links. • We have proposed a novel scheme for flowbased trace collection. • We have shown how to achieve compression ratio of 4: 1 without losing much information. 15 November 2 nd, 2001 IMW 2001

Open issues and future work • • • 16 Detailed cost analysis (chipsets, memory). Packet classification performance. Second stage of record compression. Techniques to cope with traffic anomalies. Sampling techniques. November 2 nd, 2001 IMW 2001