Peer Policy Policing with NETFLOW NANOG 25 June

Peer Policy Policing with NETFLOW NANOG 25 June 9, 2002

Matthew Meyer Traffic Engineering NANOG 25 June 9, 2002

The Global Crossing Network » 200 + On Net Cities » 27 On Net Countries » Nearly 100, 000 route miles » 17 Metro Networks

Peer Policy Policing With Netflow » Discovering and engaging the wayward packet flows that stumble onto your network » Giving default free networking a fighting chance » Get off my lawn » Bottom line: Just detecting a peer defaulting traffic us

Peer Policy Policing with Netflow Defining the problem » Telecom & Internet-space companies going into Ch 11 » Punctuated mass customer moves due to Ch 7 backbone liquidations » Peering less flexible » Some will resort to uncouth methods to mitigate the congestion and sidestep potential costs

Peer Policy Policing with Netflow Defining the problem » Fewer players, larger peerings » Peering inherits more flux and less flexibility to deal with it » Some more liberal peering channels may dry up or become heavily utilized

Peer Policy Policing with Netflow Addressing the Problem » Time to think like a bean counter » Is peering being abused? » Effect: Lower capex due to longer upgrade cycles » End goal: Knowing that we run a tight ship and being alerted when uninvited traffic enters the network

Peer Policy Policing with Netflow Measurement » Not rocket science » 1: 100 Netflow sampling » Sampling points: All traffic arriving on our border routers » Currently set to do peer-as type flow export

Peer Policy Policing with Netflow Measurement » One centrally located collector » Collector handling approximately 20 selected routers » Collector i. BGP peers with border routers » Records route table changes every 5 minutes » Dual Pentium III, 1 G memory, multiple Ultra-160 SCSI drives, directly connected to backbone

Peer Policy Policing with Netflow Measurement DEFAULTING PEER REPORT: Rec'd Peer Bytes percentage of total router interface destined for peer Bytes for interface br 2. HUB 1. gblx. net_so-2/1/3. 0 0. 011 M 0. 006 <-Peer A br 2. HUB 1. gblx. net_so-2/1/0. 026 M 0. 008 <-Peer B br 2. HUB 1. gblx. net_so-3/1/0. 087 M 0. 008 <-Peer C br 2. HUB 1. gblx. net_so-2/1/2. 0 0. 145 M 0. 011 <-Peer D br 2. HUB 1. gblx. net_at-2/2/0. 0 0. 167 M 0. 024 <-Peer E br 2. HUB 1. gblx. net_so-1/2/3. 0 0. 339 M 0. 017 <-Peer F br 2. HUB 1. gblx. net_so-3/1/2. 0 2. 464 M 0. 246 <-Peer G br 2. HUB 1. gblx. net_so-0/0/0. 0 3319. 615 M 56. 722 <-uplink br 2. HUB 1. gblx. net_so-1/0/0. 0 3381. 523 M 61. 515 <-uplink

Peer Policy Policing with Netflow Measurement EXAMPLE OF FLOWDATA /Ixia/See. Flow/bin/rseeas 2 as -S '20020603 00: 00' br 2. w 00 t 1. gblx. net Facets: Time. Interval : 06/04/2002 16: 50: 49. 217018 - 06/04/2002 19: 31: 52. 879363 UTC Router. Ipv 4 Addr : 10. 10. 10 Input. If. Index : 67 Input. If. Ipv 4 Addr : 10. 0. 0. 1 Input. If. Name Router. Name : so-1/2/3. 0 : br 2. w 00 t 1. gblx. net Src AS Dst AS Packets Pkts/sec Bytes Bits/sec -------------------- 1111 2222 654. 061 K 67. 683 321. 386 M 266. 058 K 1111 3333 177. 794 K 18. 398 130. 125 M 107. 723 K 99 44444 139. 861 K 14. 473 91. 889 M 76. 070 K 1111 3549 257. 006 K 26. 595 78. 603 M 65. 071 K 1111 5555 72. 634 K 7. 516 65. 807 M 54. 478 K [~300 more lines clipped]

Peer Policy Policing with Netflow Manipulating the Data » Extracted with Ixia tools » 24 hour cumulative byte count per interface + dest-as key pair » Created a peer-as list » Ignored incorrectly reported Netflow data according to routing policy

Peer Policy Policing with Netflow Where to Look » Our design is hierarchical » Peers tend to be on dedicated peering routers » Our peering in consistent and rich » Collecting closer to the core would not catch this behavior universally

Peer Policy Policing with Netflow Analysis » BGP import policy gets in the way of trusting source AS » Trace levels of false peer to peer traffic associated with most peering interfaces » In initial beta, no peers have been found blatantly defaulting to us

Peer Policy Policing with Netflow So Far So Good » For the moment peer defaulting does not seem to be a problem » We can move forward and easily complete a detection system » Feeling more confident about possible tighter peering ahead

Peer Policy Policing with Netflow What’s Next » Change flow export style from peer-as to originas » Putting the discovery ‘on cron’ » Long term: » Distribute collection » Build some visualization » Integrate with RRDtool

Peer Policy Policing with Netflow Retrospect » Good exercise in ‘Netflow 101’ » Sampling capability excellent » Data quality excellent » Restored confidence in Netflow reliability

SEAMLESS NETWORK. GLOBAL REACH.

THANK YOU