aarnet 3 radb and rpsl APAN NOC Taipei

AARNet 3 Network Highlights • • STM-64 c (10 Gbps) Backbone Dual STM-1 to NT & Tasmania Replacing Procket 8812 with Juniper M 320 Deploying DWDM from Adelaide to Brisbane – Providing multiple Gig. E to regional areas – Rolling our backbone onto our DWDM kit • Multiple trans Pacific circuits – 2 x STM-64 c for research and education – 2 x STM-4 c (2 x 622 Mbps) for commodity – 2 x STM-1 • Looking to expand footprint to Asia 2 © 2005, AARNet Pty Ltd

AARNet 3 Network 3 © 2005, AARNet Pty Ltd

AARNet 3 International Connectivity 4 © 2005, AARNet Pty Ltd

Commodity Transit Provision • International commodity transit from – Palo Alto – Los Angeles – Seattle etc • Domestic commodity transit in 5 © 2005, AARNet Pty Ltd – – – Sydney Melbourne Adelaide Canberra Brisbane Perth etc

AARNet Po. Ps • Domestic existing – – – – Sydney (3) Melbourne (2) Brisbane (2) Adelaide (2) Perth (3) Canberra (2) Hobart (1) Darwin (1) • Coming soon – Alice Springs (1) 6 © 2005, AARNet Pty Ltd • International existing – – – Seattle Palo Alto Los Angeles Hawai’i Suva • Coming soon – Singapore – Frankfurt

The AARNet 3 environment • Currently over 60 routers deployed – This will expand to over 80 by the end of 2005 • A mix of Juniper, Cisco and Procket routers – Currently Procket at the core – migrating to Juniper – Cisco routers at the customer edge – Link speeds varying from STM-64 c to STM-4 s and STM-1 s for long haul – 10 Gb. E intra Po. Ps and Gb. E connections from Po. Ps but still some managed services and legacy ATM 7 © 2005, AARNet Pty Ltd

The BGP environment 8 • 17 commodity transit connections • Over 163 peers both commodity and R&E • Most peerings are bilateral, a few are multilateral • Some 16 peerings with external international R&E networks • Over 200 i. BGP peerings • Over 250 IPv 4 prefixes advertised and growing… • IPv 6 enabled © 2005, AARNet Pty Ltd multicast enabled • IPv 4

BGP policy complexity • • • 9 7575: 1 Export external to AARNet with "no-export" 7575: 2 No export beyond AARNet 7575: 3 Prepend AS 7575 once 7575: 4 Prepend AS 7575 twice 7575: 5 Prepend AS 7575 thrice 7575: 6 Blackhole traffic 7575: 7 Regional only 7575: 70 AARNet local preference 70 7575: 80 AARNet local preference 80 7575: 90 AARNet local preference 90 …and much more… © 2005, AARNet Pty Ltd

How do we manage this complexity? • Very hard to manage on an ad-hoc basic with such diversity • Needs an overall policy that manages router BGP configurations • Needs cross vendor router support • Turn towards IRRs and RPSL to manage this 10 © 2005, AARNet Pty Ltd

What is RPSL? • Object oriented language • Structured whois objects • Refinement of RIPE 181 (and it’s predecessors) based on operational experience • Describes things interesting to routing policy – – 11 Prefixes AS Numbers Relationships between BGP peers Management responsibility © 2005, AARNet Pty Ltd

How we went about it • Need to identify which IRR to use – AARNet uses RADB. – Others run their own for control • Need to decide what degree of filtering is desired – Prefix filters – AS path filters – Both! • Register a maintainer object at chosen IRR 12 – Usually a “manual” process and could be multistage if PGP key authentication required © 2005, AARNet Pty Ltd

Maintainer Object Maintainer objects used for authentication Multiple authentication methods NONE, MAIL-FROM, CRYPT-PW, PGPKEY mntner: MAINT-ASAARNET descr: Maintainers for AARNet and AARNet member objects admin-c: CS 3692 tech-c: GT 342 -AU upd-to: irrcontact@aarnet. edu. au mnt-nfy: irrcontact@aarnet. edu. au auth: PGPKEY-FAD 8 C 612 auth: PGPKEY-23 B 7 F 8 EF remarks: Australian Academic and Research Network http: //www. aarnet. edu. au/ mnt-by: MAINT-ASAARNET changed: nobody@aarnet. edu. au 20040113 source: RADB 13 © 2005, AARNet Pty Ltd

Route Object Use CIDR length format Specifies origin AS for a route Can indicate membership of a route set route: descr: origin: mnt-by: changed: source: 14 © 2005, AARNet Pty Ltd 134. 7. 0. 0/16 Curtin University of Technology AS 7575 MAINT-ASAARNET nobody@aarnet. edu. au 20050818 RADB

Route Set Object • Collects routes together with similar properties route-set: AS 7575: RS-UNSW descr: University of New South Wales members: 129. 94. 0. 0/16, 149. 171. 0. 0/16, 203. 10. 48. 0/24, 203. 20. 160. 0/19 remarks: List of routes accepted from AS 7570 admin-c: MP 151 tech-c: ANOC-AP mnt-by: MAINT-ASAARNET changed: nobody@aarnet. edu. au 20050427 source: RADB 15 © 2005, AARNet Pty Ltd

AS Set Object (1) • Collect together Autonomous Systems with shared properties • Can be used in policy in place of AS as-set: AS 7575: AS-EDGE descr: AARNet 3 customers AS set members: AS 1851, AS 4822, AS 6262, AS 7575, AS 7645, AS 10148, AS 17498, AS 23654, AS 23719, AS 24101, AS 24390, AS 24431, AS 24433, AS 24434, AS 24436, AS 24437 remarks: List of customers on AARNet 3 using public AS numbers remarks: http: //www. aarnet. edu. au admin-c: MP 151 tech-c: ANOC-AP mnt-by: MAINT-ASAARNET changed: nobody@aarnet. edu. au 20050819 source: RADB 16 © 2005, AARNet Pty Ltd

AS Set Object (2) • RPSL has hierarchical names as-set: descr: members: remarks: admin-c: tech-c: mnt-by: changed: source: 17 AS 7575: AS-CUSTOMER AARNet 3 customers AS set AS 7575: AS-EDGE, AS 7575: AS-RNO List of customers on AARNet 3 using public AS numbers http: //www. aarnet. edu. au MP 151 ANOC-AP MAINT-ASAARNET nobody@aarnet. edu. au 20050819 RADB © 2005, AARNet Pty Ltd

Autonomous System Object • Routing Policy Description object • Most important components are – import – export • These define the incoming and outgoing routing announcement relationships • Instant Documentation! • whois –h whois. ra. net AS 7575 18 © 2005, AARNet Pty Ltd

Whois queries • whois –h whois. ra. net AS 7575: CUSTOMER – members: AS 7575: AS-EDGE, AS 7575: AS-RNO • whois –h whois. ra. net AS 7575: AS-EDGE – members: AS 1851, AS 4822, AS 6262, AS 7575, AS 7645, AS 10148, AS 17498, AS 23654, AS 23719, AS 24101, AS 24390, AS 24431, AS 24433, AS 24434, AS 24436, AS 24437 • whois –h whois. ra. net !g. AS 1851 – 192. 43. 227. 0/24 192. 43. 229. 0/24 192. 43. 228. 0/24 192. 43. 227. 0/24 192. 43. 229. 0/24 19 © 2005, AARNet Pty Ltd 129. 127. 0. 0/16 203. 9. 156. 0/24 129. 127. 0. 0/16 192. 43. 228. 0/24 203. 9. 156. 0/24

Whois (2) • whois –h whois. ra. net AS 7575: AS-PEER – members: AS 24, AS 42, AS 174, AS 226, AS 297, AS 703, AS 1273, AS 1982, AS 2044, AS 2152, AS 2497, AS 2516, AS 3130, AS 3303, AS 3491, AS 3557, AS 3643, AS 3699, AS 3742, AS 3786, AS 3856, AS 4134, AS 4355, AS 4513, AS 4565, AS 4716, AS 4725, AS 4739, AS 4766, AS 4788, AS 5726, AS 6327, AS 6517, AS 6539, AS 6939, AS 7132, AS 8075, AS 8121, AS 8404, AS 9156, AS 9264, AS 9277, AS 9318, AS 9505, AS 10310, AS 10557, AS 11404, AS 11726, AS 11841, AS 12111, AS 12222, AS 14277, AS 14361, AS 15169, AS 15290, AS 15412, AS 16713, AS 18530, AS 21947, AS 22822, AS 23260, AS 23265, AS 23504, AS 25700, AS 25973, AS 26228, AS 27008, AS 27318, AS 29814, AS 30092, AS 31800, AS 33529 20 © 2005, AARNet Pty Ltd

Whois (3) • whois –h whois. ra. net !g. AS 8075 A 488 207. 46. 128. 0/18 207. 46. 192. 0/18 204. 95. 110. 0/23 207. 68. 128. 0/18 204. 255. 246. 0/23 198. 105. 232. 0/22 131. 107. 0. 0/16 207. 46. 32. 0/20 205. 248. 96. 0/19 204. 95. 96. 0/20 207. 68. 128. 0/18 207. 46. 0. 0/20 207. 46. 208. 0/20 192. 197. 157. 0/24 199. 60. 28. 0/24 199. 103. 122. 0/24 65. 55. 224. 0/19 199. 103. 90. 0/23 65. 54. 112. 0/20 65. 54. 96. 0/20 207. 46. 96. 0/19 207. 68. 160. 0/19 65. 54. 192. 0/19 65. 54. 128. 0/19 C • Can now build inbound prefix filters appropriately 21 © 2005, AARNet Pty Ltd

Use of RPSL • Use Rt. Config v 4 (part of RATool. Set from ISI) to generate filters based on information stored in our routing registry – Avoid filter errors (typos) – Filters consistent with documented policy (need to get policy correct though) – Currently we use RATool. Set v 4. 7. 1 – Need to script our own tools for Procket and Juniper 22 © 2005, AARNet Pty Ltd

Using RPSL to configure routers • Need to define “policy” for filtering – Inbound from customers & peers – Outbound to customers & peers • Need to be aware of shortcomings in router configuration and/or configuration generator – Command line length (on cisco this is 512 bytes) – Complexity of rules 23 © 2005, AARNet Pty Ltd

AARNet’s filtering philosophy • Inbound – – Filter customer by prefix and AS path Filter peer by prefix filter Filter providers for prefixes longer than a /24 Don’t accept martians from anyone • Outbound – Filter by BGP community, which indicates the class of the prefix (customer, peer, etc) 24 © 2005, AARNet Pty Ltd

Rt. Config & IRRTool. Set • Version 4. 0 supports RPSL • Generates cisco configurations • Contributed support for Bay’s BCC, Juniper’s Junos and Gated/RSd • Creates route and AS path filters. • Can also create ingress/egress filters 25 © 2005, AARNet Pty Ltd

RFC 1998 - Use of BGP communities import: { from AS-ANY action community. append(7575: 1000); } refine { from AS-ANY action pref=30; accept community. contains(7575: 70); from AS-ANY action pref=20; accept community. contains(7575: 80); from AS-ANY action pref=10; accept community. contains(7575: 90); from AS-ANY action pref=0; accept ANY; 26 © 2005, AARNet Pty Ltd

RFC 1998 (2) } refine { from AS 65510 at 202. 158. 192. 241 action community. append(7575: 2241, 7575: 3006, 7575: 5001); accept { 134. 7. 0. 0/16, 130. 116. 160. 0/21, 130. 116. 168. 0/24, 139. 230. 159. 0/24, 150. 229. 207. 128/25 } AND <^Peer. AS+$>; • Now the routes are correctly tagged and the RFC 1998 policy applied. 27 © 2005, AARNet Pty Ltd

Blackholes import: { from AS-ANY action community. append(7575: 1000); accept ANY; } refine { from AS-ANY action next-hop=192. 168. 1. 1; accept community. contains(7575: 6); } refine { from AS 65510 at 202. 158. 192. 241 action community. append(7575: 2241, 7575: 3006, 7575: 5001); accept { 134. 7. 0. 0/16, 130. 116. 160. 0/21, 130. 116. 168. 0/24, 139. 230. 159. 0/24, 150. 229. 207. 128/25 }^32 AND <^Peer. AS+$>; } from AS 24437 at 202. 158. 192. 250 action community. append(7575: 2250, 7575: 3006); accept Peer. AS^32 AND <^Peer. AS+$>; 28 © 2005, AARNet Pty Ltd

Rt. Config command line options • Defaults to using RADB – -h whois. ra. net – -p 43 – -protocol irrd • Defaults to “cisco” style output – -config cisco • -suppress_martian • -s <list of IRR sources> – -s CCAIR, RADB, CW 29 © 2005, AARNet Pty Ltd

Rt. Configuration Template (1) ! Rt. Config template for cpe-curtin-er 1 router in AS 7575 ! @Rt. Config set cisco_map_first_no = 10 @Rt. Config set cisco_map_increment_by = 10 @Rt. Config set cisco_prefix_acl_no = 100 @Rt. Config set cisco_aspath_acl_no = 130 @Rt. Config set cisco_pktfilter_acl_no = 130 @Rt. Config set cisco_community_acl_no = 30 @Rt. Config set cisco_max_preference = 100 ! no ip access-list extended DENY-BOGON-SOURCE @Rt. Config print. Prefix. Ranges " deny ip %p %K anyn" filter fltr-bogons permit ip any ! 30 © 2005, AARNet Pty Ltd

Rt. Configuration Template (2) ! Curtin University ! router bgp 7575 neighbor 202. 158. 198. 186 remote-as 65510 neighbor 202. 158. 198. 186 description Curtin University neighbor 202. 158. 198. 186 send-community neighbor 202. 158. 198. 186 soft-reconfiguration inbound neighbor 202. 158. 198. 186 ebgp-multihop 2 @Rt. Config set cisco_map_name = "AS%d-IPv 4 -1 -IMPORT" @Rt. Config import AS 7575 202. 158. 192. 241 AS 65510 202. 158. 198. 186 @Rt. Config set cisco_map_name = "AS%d-IPv 4 -1 -EXPORT" @Rt. Config export AS 7575 202. 158. 192. 241 AS 65510 202. 158. 198. 186 ! end 31 © 2005, AARNet Pty Ltd

Cisco Configuration (1) ip ip ip ! ip ip 32 prefix-list prefix-list pl 100 pl 100 seq seq seq 5 permit 130. 116. 160. 0/21 ge 32 10 permit 130. 116. 168. 0/24 ge 32 15 permit 134. 7. 0. 0/16 ge 32 20 permit 139. 230. 159. 0/24 ge 32 25 permit 150. 229. 207. 128/25 ge 32 30 deny 0. 0/0 le 32 prefix-list prefix-list pl 101 pl 101 seq seq 5 permit 130. 116. 160. 0/21 10 permit 130. 116. 168. 0/24 15 permit 134. 7. 0. 0/16 20 permit 134. 7. 230. 0/24 25 permit 134. 7. 254. 144/28 30 permit 139. 230. 159. 0/24 35 permit 150. 229. 207. 128/25 40 deny 0. 0/0 le 32 prefix-list pl 102 seq 5 permit 0. 0/0 prefix-list pl 102 seq 10 deny 0. 0/0 le 32 prefix-list pl 103 seq 5 permit 0. 0/0 le 24 prefix-list pl 103 seq 10 deny 0. 0/0 le 32 © 2005, AARNet Pty Ltd

Cisco Configuration (2) route-map AS 65510 -IPv 4 -1 -IMPORT permit 10 route-map AS 65510 -IPv 4 -1 -IMPORT permit 40 match ip address prefix-list pl 101 match as-path 130 match community 33 set ip next-hop 192. 168. 1. 1 set local-preference 90 set community 7575: 1000 7575: 2241 7575: 3006 7575: 5001 set community 7575: 1000 7575: 2241 7575: 3006 7575: 500 additive ! ! route-map AS 65510 -IPv 4 -1 -IMPORT permit 20 route-map AS 65510 -IPv 4 -1 -IMPORT permit 50 match ip address prefix-list pl 101 match as-path 130 match community 31 set local-preference 100 set local-preference 70 set community 7575: 1000 7575: 2241 7575: 3006 7575: 5001 additive ! route-map AS 65510 -IPv 4 -1 -IMPORT permit 30 match ip address prefix-list pl 101 match as-path 130 match community 32 set local-preference 80 set community 7575: 1000 7575: 2241 7575: 3006 7575: 5001 additive ! 33 © 2005, AARNet Pty Ltd

Using Rt. Config • Rt. Config –cisco_use_prefix_lists < cpe-curtin-er 1. rtconfig • Redirect output

Problems? • Policy can easily get very complex and result in even more complex router configuration • Line limit on cisco AS path filters (need to be careful when using as-sets) • Limited non-Cisco support • Need to develop scripts to implement on Procket and Juniper 35 © 2005, AARNet Pty Ltd

Where next? • • • RPSLng http: //www. radb. net/rpslng-08. html Adds IPv 6 and multicast extensions to RPSL RADB & RIPE have implemented support Implemented in recent releases of IRRTool. Set – ftp: //ftp. isc. org/isc/IRRTool. Set-4. 8. 2/ 36 © 2005, AARNet Pty Ltd

References • RPSL - RFC 2622 – http: //www. faqs. org/rfcs/rfc 2622. html • Using RPSL in Practice - RFC 2650 – http: //www. faqs. org/rfcs/rfc 2650. html • IRRTool. Set – ftp: //ftp. isc. org. net/isc/IRRTool. Set/ • RPSL Training Page – http: //www. isi. edu/ra/rps/training • RADB – http: //www. radb. net/ 37 © 2005, AARNet Pty Ltd