Corso di Reti di Calcolatori II Simon Pietro

Corso di Reti di Calcolatori II Simon Pietro Romano spromano@unina. it Inter-domain routing with BGP 4

Copyright notes… ● ● ● This is a shrinked version of a tutorial taught by Prof. Olivier Bonaventure from Universite catholique de Louvain (UCL), Belgium You can obtain an HTML or Open. Office version of this tutorial with the hypertext links by sending an email to the author. This work is licensed under a Creative Commons License: – ● The updated versions of the slides may be found on: – ● http: //creativecommons. org/licenses/by-sa/2. 0/ http: //totem. info. ucl. ac. be/BGP Tim Griffin maintains a very long and up to date list of references on BGP; see: – http: //www. cambridge. intel-research. net/~griffin/interdomain/

Outline ● Organization of the global Internet – Example of domains ● BGP basics ● BGP in large networks

How to route IP packets in the global Internet ? ● A map of the global Internet in 2000 (source: http: //research. lumeta. com/ches/map/gallery/index. html)

Organization of the Internet ● Internet is composed of more than 10. 000 autonomous routing domains (AS – Autonomous System) – A domain is a set of routers, links, hosts and local area networks under the same administrative control ● A domain can be very large. . . – ● A domain can be very small. . . – – AS 568: SUMNET-AS DISO-UNRRA contains 73154560 IP addresses AS 2111: IST-ATRIUM TE Experiment a single PC running Linux. . . Domains are interconnected in various ways ● ● The interconnection of all domains should in theory allow packets to be sent anywhere Usually a packet will need to cross a few ASes to reach its destination

Types of domains ● Transit domain – A transit domain allows external domains to use its own infrastructure to send packets to other domains S 1 S 2 ● T 2 T 1 T 3 S 4 S 3 Examples – UUNet, Open. Transit, GEANT, Internet 2, RENATER, EQUANT, BT, Telia, Level 3, . . .

Types of domains (2) ● Stub domain – A stub domain does not allow external domains to use its infrastructure to send packets to other domains ● A stub is connected to at least one transit domain – – Single-homed stub : connected to one transit domain Dual-homed stub : connected to two transit domains S 1 S 2 – T 3 S 4 S 3 Content-rich stub domain ● – T 2 T 1 Large web servers : Yahoo, Google, MSN, TF 1, BBC, . . . Access-rich stub domain ● ISPs providing Internet access via CATV, ADSL, . . .

A Stub domain : Belnet (http: //www. belnet. be) Note well: other maps of ISPs may be found at: http: //www. cs. washington. edu/research/networking/rocketfuel/interactive/

A transit domain : Easynet http: //www. easynet. be/home/index. cfm? id=15&l=1

A transit domain : GEANT (source http: //www. dante. net)

A transit domain : BT/IGnite Source : http: //www. ignite. net/info/maps. shtml

A large transit domain : UUNet Source: http: //www. uu. net

Outline ● Organization of the global Internet – Example of domains ● BGP basics ● BGP in large networks

Architecture of a normal IP router Routing protocol Routing table The "best" paths selected from the routing table built by the routing protocols are installed in the forwarding table Shap. IP packets Forwarding Table Control IP packets Class. Pol Forwarding Shap. Class. Pol Forwarding decision based on longest match Update of TTL and checksum fields in IP packets

Internet routing – Interior Gateway Protocol (IGP) ● Routing of IP packets inside each domain – Only knows topology of its domain Domain 4 Domain 2 Domain 1 – Domain 3 Exterior Gateway Protocol (EGP) ● Routing of IP packets between domains – Each domain is considered as a blackbox

Intra-domain routing ● Goal – Allow routers to transmit IP packets along the best path towards their destination ● best usually means the shortest path – ● Shortest measured in seconds or as number of hops sometimes best means the less loaded path Allow to find alternate routes in case of failures Behavior – All routers exchange routing information ● ● Each domain router can obtain routing information for the whole domain The network operator or the routing protocol selects the cost of each link

Outline ● Organization of the global Internet ● BGP basics – – – ● Routing policies The Border Gateway Protocol How to prefer some routes over others BGP in large networks

Inter-domain routing ● Goals – Allow to transmit IP packets along the best path towards their destination through several transit domains while taking into account the routing policies of each domain without knowing the detailed topology of those domains ● ● From an inter-domain viewpoint, best path often means cheapest path Each domain is free to specify inside its routing policy the domains for which it agrees to provide a transit service and the method it uses to select the best path to reach destination

Domains versus Autonomous Systems ● ● The BGP inter-domain routing protocol deals with Autonomous Systems (AS) – An AS is defined as <<a set of routers under a single technical administration. . . that presents a consistent picture of what destinations are reachable through it. >> – Each AS is identified by its AS number In practice – A domain is often equivalent to an AS – A domain may be composed of several ASes ● – Ex: Worldcom uses AS 701, AS 702, . . . Many domains do not have an AS number ● Ex: small networks connected to one provider without using BGP

Useful links ● ● Each AS on the Internet has been assigned a 16 bits number by the Regional Internet Registries For a current list of assigned AS numbers: – ● http: //www. cidr-report. org/autnums. html More information: – – http: //whois. ripe. net http: //www. radb. net

Types of inter-domain links ● Two types of inter-domain links – Private link ● Usually a leased line between two routers belonging to the two connected domains R 2 R 1 Domain B Domain A – Connection via a public interconnection point ● Usually Gigabit or higher Ethernet switch that interconnects routers belonging to different domains Physical link Interdomain link R 2 R 3 R 1 R 4

Routing policies ● ● In theory BGP allows each domain to define its own routing policy. . . In practice there are two common policies – customer-provider peering ● – Customer c buys Internet connectivity from provider P shared-cost peering ● Domains x and y agree to exchange packets by using a direct link or through an interconnection point

Customer-provider peering AS 1 $ AS 2 $ $ Customer AS 4 AS 3 $ Provider $ – Principle ● Customer sends to its provider its internal routes and the routes learned from its own customers – ● AS 7 Provider will advertise those routes to the entire Internet to allow anyone to reach the Customer Provider sends to its customers all known routes – Customer will be able to reach anyone on the Internet

Shared-cost peering AS 1 $ AS 2 $ $ AS 4 AS 3 $ Shared-cost Customer-provider $ – Principle ● Peer. X sends to Peer. Y its internal routes and the routes learned from its own customers – – ● AS 7 Peer. Y will use shared link to reach Peer. X and Peer. X's customers Peer. X's providers are not reachable via the shared link Peer. Y sends to Peer. X its internal routes and the routes learned from its own customers – – Peer. X will use shared link to reach Peer. Y and Peer. Y's customers Peer. Y's providers are not reachable via the shared link

Routing policies ● A domain specifies its routing policy by defining on each BGP router two sets of filters for each peer – Import filter ● – Export filter ● ● Specifies which routes can be accepted by the router among all the received routes from a given peer Specifies which routes can be advertised by the router to a given peer Filters can be defined in RPSL – Routing Policy Specification Language (RFC 2622) Note well: Internet Routing Registries contain the routing policies of various ISPs, see : http: //www. ripe. net/ripencc/pub-services/whois. html, http: //www. arin. net/whois/index. html, http: //www. apnic. net/apnic-bin/whois. pl

Routing policies Simple example with RPSL AS 1 $ AS 2 $ AS 3 $ AS 4 $ Shared-cost Customer-provider $ Import policy for AS 4 Import: from AS 3 accept AS 3 import: from AS 7 accept AS 7 import: from AS 1 accept ANY import: from AS 2 accept ANY Export policy for AS 4 export: to AS 3 announce AS 4 AS 7 export: to AS 7 announce ANY export: to AS 1 announce AS 4 AS 7 export: to AS 2 announce AS 4 AS 7 Import policy for AS 7 Import: from AS 4 accept ANY Export policy for AS 7 export: to AS 4 announce AS 7

Outline ● Organization of the global Internet ● BGP basics – – – ● Routing policies The Border Gateway Protocol How to prefer some routes over others BGP in large networks

The Border Gateway Protocol Principle ● – Path vector protocol ● BGP router advertises its best route to each destination AS 5 prefix: 1. 0. 0. 0/8 ●ASPath: AS 1 ● AS 1 1. 0. 0. 0/8 prefix: 1. 0. 0. 0/8 ●ASPath: AS 1 AS 2 ● – prefix: 1. 0. 0. 0/8 ●ASPath: : : AS 2: AS 4 AS 1 ● prefix: 1. 0. 0. 0/8 ●ASPath: AS 4: AS 1 ● AS 4 . . . with incremental updates ● Advertisements are only sent when their content changes

“Origin” of the routes announced by BGP ● Where do the routes announced by a BGP router come from ? – Learned from other BGP routers ● – Static configuration ● ● ● – BGP router only propagates the received routes BGP router is configured to advertise some prefixes Drawback : requires manual configuration Advantage : Stable set of advertised prefixes Learned from an Interior Gateway Protocol ● ● The prefixes received from the IGP are advertised by the BGP router usually as an aggregate Advantage – ● BGP advertisements follow network state, prefix is automatically withdrawn by BGP if it is not reachable via IGP Drawback – BGP announcements will be unstable if IGP is unstable. . .

Policies and BGP ● Two mechanisms to support policies in BGP – Each domain defines itself which is the best route to reach destination based on the routes learned from its peers ● ● – The chosen best route is not necessarily the ''shortest'' route as with IGPs Only the best route towards each destination can be announced to external peers Each domain determines, on its own, which routes can be advertised to each peer ● An AS does not necessarily advertise to all its neighbors all the routes that it knows

Conceptual model of a BGP router Legend: Adj-RIB-In Adjacency Routing Information Base for incoming messages Adj-RIB-Out Adjacency Routing Information Base for outgoing messages Loc-RIB Local Routing Information Base BGP Adj-RIB-In BGP Msgs from Peer[N] Peer[1] Import filter Attribute BGP Msgs manipulation from Peer[1] BGP Loc-RIB All acceptable routes BGP Decision Process One best route to each destination Import filter(Peer[i]) Determines which BGP Msgs are acceptable from Peer[i] BGP Routing Information Base Contains all the acceptable routes learned from all Peers + internal routes ● BGP decision process selects the best route towards each destination BGP Adj-RIB-Out Peer[N] BGP Msgs to Peer[N] Peer[1] Export filter Attribute manipulation BGP Msgs to Peer[1] Export filter(Peer[i]) Determines which routes can be sent to Peer[i]

BGP : Principles of operation ● Principles – BGP relies on the incremental exchange of path vectors BGP session established over TCP connection between peers Each peer sends all its active routes AS 3 R 1 BGP session BGP Msgs R 2 AS 4 As long as the BGP session remains up Incrementally update BGP routing tables

BGP : Principles of operation (2) ● Simplified model of BGP – – 2 types of BGP path vectors UPDATE ● ● Used to announce a route towards one prefix Content of UPDATE – – Destination address/prefix Inter-domain path used to reach destination (AS-Path) Next-hop (address of the router advertising the route) WITHDRAW ● ● Used to indicate that a previously announced route is not reachable anymore Content of WITHDRAW – Unreachable destination address/prefix

Events during a BGP session 1. Addition of a new route to RIB – A new internal route was added on local router ● ● – static route added by configuration Dynamic route learned from IGP Reception of UPDATE message announcing a new or modified route 2. Removal of a route from RIB – Removal of an internal route ● ● – Static route is removed from router configuration Intra-domain route declared unreachable by IGP Reception of WITHDRAW message 3. Loss of BGP session – All routes learned from this peer removed from RIB

The BGP messages ● Variable length messages with fixed size header OPEN used to establish BGP session ● UPDATE used to send new routes and to remove Marker ( 16 bytes ) : All 11. . . unusable routes ● NOTIFICATION used to inform the remote peer of Type Length : 16 bits an error BGP session is closed upon transmission or reception of NOTIFICATION message Max length of BGP messages : 4096 bytes ● KEEPALIVE one message must be sent at least every 30 seconds on each BGP session ● ROUTE_REFRESH used to support graceful restart 32 bits ●

The OPEN message ● Used to establish a BGP session between two BGP peers 32 bits Currently version 4 Version My AS Number Hold Time BGP Identifier Opt. Len Optional Parameters Variable Length Encoded in TLV Format AS # of the BGP peer sending the message Hold Time : maximum delay between successive KEEPALIVE, and/or UPDATE messages BGP Id : Usually IP v 4 loopback address of BGP peer Optional field : Used notably for capabilities negotiation

Establishment of a BGP session Usually, a BGP session can only be established between two manually configured peers. Each peer needs to be configured with the IP address and the AS number of the remote peer CONNECT. req SYN(port=179) CONNECT. ind CONNECT. resp CONNECT. conf SYN+ACK(port=179) TCP connection established DATA. req(OPEN) ACK(port=179) TCP connection established DATA(BGP OPEN) ACK DATA. req(OPEN) BGP session established DATA. req(OPEN) DATA(BGP OPEN) ACK BGP session established

The UPDATE message – Single message type used to carry both IPv 4 route announcements and route withdrawals 32 bits # Withdrawn routes Variable Length LEN Prefix length in bits Withdrawn prefix (1 -4 octets) Tot. Path Attr. Len Path attributes Variable Length Network Layer Reachability Information Variable Length LEN Prefix length in bits Advertised prefix (1 -4 octets)

The KEEPALIVE and NOTIFICATION messages ● The KEEPALIVE message – – ● BGP Message containing only the default header Every Hold. Time/3 seconds, send a KEEPALIVE message if no recent BGP message was sent The NOTIFICATION message – indicates problem in processing of BGP message ● BGP session is released upon transmission/reception of NOTIFICATION Example errors: ● 2: OPEN Message Error ● Unsupported Version, Unsupported Optional Parameter, . . . ● 3: UPDATE Message Error ● Malformed Attribute List, . . . ● 4: Hold Timer Expired ● 5: Finite State Machine Error ● 6: Cease ● Err Code Sub. Code Additional data (variable length)

BGP and IP A first example – Initial updates UPDATE ●prefix: 194. 100. 0. 0/24, ●Next. Hop: R 2 ●ASPath: AS 20: AS 10 UPDATE ●prefix: 194. 100. 0. 0/24, ●Next. Hop: R 1 ●ASPath: AS 10 AS 20 R 1 BGP 194. 100. 0. 0/24 UPDATE ●prefix: 194. 100. 0. 0/24, ●Next. Hop: R 1 ●ASPath: AS 10 R 2 R 3 194. 100. 1. 0/24 BGP AS 30 UPDATE ●prefix: 194. 100. 0. 0/24, ●Next. Hop: R 4 ●ASPath: AS 40: AS 10 R 4 AS 40 – What happens if link AS 10 -AS 20 goes down ?

BGP and IP A first example (2) ● ● ● If link AS 10 -AS 20 goes down, AS 20 will not consider anymore the path learned from AS 10 AS 20 will thus remove this path from its routing table and will instead select the path learned from AS 40 This will force AS 20 to send the following UPDATE to AS 30: UPDATE ●prefix: 194. 100. 0. 0/24, ●Next. Hop: R 2 ●ASPath: AS 20: AS 40: AS 10

BGP and IP A second example AS 20 AS 10 AS 30 195. 100. 0. 0/30 R 1 195. 100. 0. 2 194. 100. 0. 0/24 R 2 195. 100. 0. 4/30 195. 100. 0. 5 195. 100. 0. 6 BGP R 3 194. 100. 1. 0/24 194. 100. 2. 0/23 UPDATE ●prefix: 194. 100. 0. 0/24, ●Next. Hop: 195. 100. 0. 1 ●ASPath: AS 10 UPDATE ●prefix: 194. 100. 2. 0/23, ●Next. Hop: 195. 100. 0. 2 ●ASPath: AS 20 – In this example, we only consider the BGP messages concerning the following IP networks: 194. 100. 0. 0/24, 194. 100. 1. 0/24 and 194. 100. 2. 0/23 Main Path attributes of UPDATE message ● ● Next. Hop : IP address of router used to reach destination ASPath : Path followed by the route advertisement

BGP and IP A second example (2) AS 20 AS 10 AS 30 195. 100. 0. 4/30 195. 100. 0. 0/30 R 1 195. 100. 0. 2 R 2 195. 100. 0. 5 195. 100. 0. 6 194. 100. 0. 0/24 BGP 194. 100. 2. 0/23 R 3 194. 100. 1. 0/24 UPDATE ●prefix: 194. 100. 0. 0/24 ●Next. Hop: 195. 100. 0. 5 ●ASPath: AS 20: AS 10 UPDATE ●prefix: 194. 100. 2. 0/23 ●Next. Hop: 195. 100. 0. 5 ●ASPath: AS 20 UPDATE ●prefix: 194. 100. 1. 0/24, ●Next. Hop: 195. 100. 0. 2 ●ASPath: AS 20; AS 30 UPDATE ●prefix: 194. 100. 1. 0/24, ●Next. Hop: 195. 100. 0. 6 ●ASPath: AS 30

BGP and IP A second example (3) AS 20 AS 10 AS 30 195. 100. 0. 4/30 195. 100. 0. 0/30 R 1 195. 100. 0. 2 R 2 195. 100. 0. 5 195. 100. 0. 6 R 3 194. 100. 0. 0/24 BGP 194. 100. 2. 0/23 WITHDRAW ●prefix: 194. 100. 1. 0/24

Outline ● Organization of the global Internet ● BGP basics – – – ● Routing policies The Border Gateway Protocol How to prefer some routes over others BGP in large networks

How to prefer some routes over others ? RA RB AS 2 Backup: 2 Mbps Primary: 34 Mbps R 1 AS 1 ● How to ensure that packets will flow on primary link ? RA AS 2 RB R 3 Expensive AS 1 R 1 ● AS 3 R 5 Cheap R 2 AS 4 How to prefer cheap link over expensive link ? AS 5

How to prefer some routes over others (2) ? BGP Msgs from Peer[N] Peer[1] Import filter BGP Msgs from Peer[1] Attribute manipulation BGP RIB All acceptable routes BGP Decision Process One best route to each destination Peer[N] BGP Msgs to Peer[N] Peer[1] Export filter Attribute manipulation BGP Msgs to Peer[1] Simplified BGP Decision Process Import filter ● Select routes with highest ● Selection of acceptable routes local-pref ● Addition of local-pref attribute ● If there are several routes, inside received BGP Msg choose routes with the ● Normal quality route: local-pref=100 shortest ASPath ● Better than normal route: local-pref=200 ● If there are still several routes ● Worse than normal route: local-pref=50 tie-breaking rule

How to prefer some routes over others (3) ? RA AS 2 Backup: 2 Mbps RB Primary: 34 Mbps R 1 AS 1 RPSL-like policy for AS 1 aut-num: AS 1 import: from AS 2 RA at R 1 set localpref=100; from AS 2 RB at R 1 set localpref=200; accept ANY export: to AS 2 RA at R 1 announce AS 1 to AS 2 RB at R 1 announce AS 1 RPSL-like policy for AS 2 aut-num: AS 2 import: from AS 1 R 1 at RA set localpref=100; from AS 1 R 1 at RB set localpref=200; accept AS 1 export: to AS 1 R 1 at RA announce ANY to AS 2 R 1 at RB announce ANY

How to prefer some routes over others (4) ? RA AS 2 RB R 3 AS 3 Expensive R 5 AS 1 R 1 Cheap R 2 AS 5 AS 4 RPSL policy for AS 1 aut-num: AS 1 import: from AS 2 RA at R 1 set localpref=100; from AS 4 R 2 at R 1 set localpref=200; accept ANY export: to AS 2 RA at R 1 announce AS 1 to AS 4 R 2 at R 1 announce AS 1 ● ● AS 1 will prefer to send packets over the cheap link But the flow of the packets destined to AS 1 will depend on the routing policy of the other domains

Limitations of local-pref – In theory ● Each domain is free to define its order of preference for the routes learned from external peers 1. 0. 0. 0/8 Preferred paths for AS 3 1. AS 4: AS 1 2. AS 1 AS 3 ● AS 1 Preferred paths for AS 4 1. AS 3: AS 1 2. AS 1 AS 4 How to reach 1. 0. 0. 0/8 from AS 3 and AS 4 ?

Limitations of local-pref (2) ● AS 1 sends its UPDATE messages. . . 1. 0. 0. 0/8 UPDATE ●Prefix: 1. 0. 0. 0/8 ●ASPath: AS 1 AS 3 AS 1 UPDATE ●Prefix: 1. 0. 0. 0/8 ●ASPath: AS 1 AS 4 Preferred paths for AS 3 1. AS 4: AS 1 2. AS 1 Preferred paths for AS 4 1. AS 3: AS 1 2. AS 1 Routing table for AS 3 1. 0. 0. 0/8 ASPath: AS 1 (best) Routing table for AS 4 1. 0. 0. 0/8 ASPath: AS 1 (best)

Limitations of local-pref (3) ● First possibility – AS 3 sends its UPDATE first. . . 1. 0. 0. 0/8 AS 1 Preferred paths for AS 3 1. AS 4: AS 1 2. AS 1 AS 3 Routing table for AS 3 1. 0. 0. 0/8 ASPath: AS 1 (best) ● Preferred paths for AS 4 1. AS 3: AS 1 2. AS 1 AS 4 UPDATE ●Prefix: 1. 0. 0. 0/8 ●ASPath: AS 3: AS 1 Stable route assignment Routing table for AS 4 1. 0. 0. 0/8 ASPath: AS 1 1. 0. 0. 0/8 ASPath: AS 3: AS 1 (best)

Limitations of local-pref (4) ● Second possibility – AS 4 sends its UPDATE first. . . 1. 0. 0. 0/8 Preferred paths for AS 3 1. AS 4: AS 1 2. AS 1 AS 3 Routing table for AS 3 1. 0. 0. 0/8 ASPath: AS 1 1. 0. 0. 0/8 ASPath: AS 4: AS 1 (best) ● Preferred paths for AS 4 1. AS 3: AS 1 2. AS 1 AS 4 UPDATE ●Prefix: 1. 0. 0. 0/8 ●ASPath: AS 4: AS 1 Routing table for AS 4 1. 0. 0. 0/8 ASPath: AS 1 (best) Another (but different) stable route assignment

Limitations of local-pref (5) ● Third possibility – AS 3 and AS 4 send their UPDATE together. . . 1. 0. 0. 0/8 Preferred paths for AS 3 1. AS 4: AS 1 2. AS 1 AS 3 UPDATE ●Prefix: 1. 0. 0. 0/8 ●ASPath: AS 3: AS 1 ● ● AS 1 Preferred paths for AS 4 1. AS 3: AS 1 2. AS 1 AS 4 UPDATE ●Prefix: 1. 0. 0. 0/8 ●ASPath: AS 4: AS 1 AS 3 prefers the indirect path and will thus send withdraw since the chosen best path is via AS 4 prefers the indirect path and will thus send withdraw since the chosen best path is via AS 3

Limitations of local-pref (6) ● Third possibility (cont. ) – AS 3 and AS 4 send their UPDATE together. . . Preferred paths for AS 3 1. AS 4: AS 1 2. AS 1 1. 0. 0. 0/8 Preferred paths for AS 4 1. AS 3: AS 1 2. AS 1 AS 3 WITHDRAW ●Prefix: 1. 0. 0. 0/8 ● WITHDRAW ●Prefix: 1. 0. 0. 0/8 AS 3 learns that the indirect route is not available anymore – ● AS 4 AS 3 will reannounce its direct route. . . AS 4 learns that the indirect route is not available anymore – AS 4 will reannounce its direct route. . .

More limitations of local-pref ● Unfortunately, inter-domain routing may not converge at all in some cases. . . Preferred paths for AS 3 1. AS 4: AS 0 2. AS 0 AS 3 ● AS 1 Preferred paths for AS 1 1. AS 3: AS 0 2. AS 0 Preferred paths for AS 4 1. AS 1: AS 0 2. AS 0 AS 4 How to reach a destination inside AS 0 in this case ?

local-pref and economical relationships ● In practice, local-pref is often used to enforce economical relationships Prov 1 Prov 2 $ $ Peer 1 AS 1 Peer 2 $ Cust 1 Local-pref values used by AS 1 > 1000 for the routes received from a Customer 500 – 999 for the routes learned from a Peer < 500 for the routes learned from a Provider Peer 3 Peer 4 $ Cust 2 Shared-cost $ Customer-provider Since AS 1 is paid to carry packets towards Cust 1 and Cust 2, it will select a route towards those networks whenever possible ● Since AS 1 does not pay to carry packets towards Peer 1 -4, AS 1 will select a route towards those networks whenever possible ●

Consequence of this utilization of local-pref Which route will be used by AS 1 to reach AS 5 ? AS 2 $ AS 1 AS 3 $ $ AS 4 AS 8 $ $ AS 5 $ AS 7 – Internet and how willare AS 5 reach AS 1 ? paths often asymmetrical $ AS 6 Shared-cost $ Customer-provider $ ●

Guidelines for a safe utilization of local-pref ● The directed graph composed of the customer-> provider links is loop-free – An AS cannot be a customer of a provider of its providers AS 1 – $ AS 2 $ AS 3 $ An AS always prefer a route via a customer over a route via a provider or a peer ● With some restrictions on the graph composed of peer-topeer relationships, it is also possible to allow an AS to give the same preference to a route via a customer or via a peer

The Organization of the Internet – Tier-1 ISPs ● ● Dozen of large ISPs interconnected by shared-cost Provide transit service – – Tier-2 ISPs ● ● Regional or National ISPs Customer of T 1 ISP(s) Provider of T 2 ISP(s) shared-cost with other T 2 ISPs – – Uunet, Level 3, Open. Transit, . . . France Telecom, BT, Belgacom Tier-3 ISPs ● ● ● Smaller ISPs, Corporate Networks, Content providers Customers of T 2 or T 1 ISPs shared-cost with other T 3 ISPs

Composition of Internet paths ● Most Internet paths contain a sequence of – – – 0 or more Customer->Provider relationships 0 or 1 Peer-to-Peer relationships 0 or more Provider->Customer relationships AS 1 AS 2 $ $ $ AS 4 AS 3 $ AS 9 $ AS 8 $ $ $ AS 7 Shared-cost Customer-provider

Outline ● Organization of the global Internet ● BGP basics ● BGP in large networks – – – ● The needs for i. BGP Confederations and Route Reflectors The dynamics of BGP Inter-domain traffic engineering with BGP

BGP and IP Second example 194. 100. 2. 0/23 AS 10 195. 100. 0. 2 195. 100. 0. 0/30 R 1 195. 100. 0. 1 194. 100. 0. 0/23 AS 30 R 2 195. 100. 0. 10 BGP AS 20 195. 100. 0. 8/30 195. 100. 0. 9 194. 100. 4. 0/23 ● 195. 100. 0. 6 R 3 BGP 195. 100. 0. 4/30 R 4 195. 100. 0. 5 Problem – How can R 2 (resp. R 4) advertise to R 4 (resp. R 2) the routes learned from AS 10 (resp. AS 30) ?

BGP and IP Second example (2) 194. 100. 2. 0/23 AS 10 195. 100. 0. 2 195. 100. 0. 0/30 R 1 195. 100. 0. 1 194. 100. 0. 0/23 AS 30 R 2 195. 100. 0. 10 BGP AS 20 IGP ● 195. 100. 0. 4/30 R 4 195. 100. 0. 5 First solution – ● BGP 195. 100. 0. 8/30 195. 100. 0. 9 194. 100. 4. 0/23 Use IGP (OSPF/ISIS, RIP) to carry BGP routes Drawbacks – – 195. 100. 0. 6 IGP may not be able to support so many routes IGP does not carry BGP attributes like ASPath ! R 3

The AS 7007 incident ● The AS 7007 incident AS 7007 AS x RX R 1 4. 0. 0. 0/8 : AS x: AS 3: AS 6 ● R 2 RY 4. 0. 0. 0/8 : AS 7007 !!!!!! A single configuration error in two routers – – All routes learned from ASX on R 1 were redistributed to R 2 via IGP and R 2 announced them to ASY Consequence ● ● – ● AS Y AS 7007 advertised routes that almost all IP addresses were belonging to AS 7007 These routes were shorter than the real routes. . . Two hours of disruption for large parts of the Internet ! http: //answerpointe. cctec. com/maillists/nanog/historical/9704/msg 00342. html

i. BGP and e. BGP 194. 100. 2. 0/23 AS 30 AS 10 195. 100. 0. 2 R 2 195. 100. 0. 0/30 195. 100. 0. 10 R 1 195. 100. 0. 1 e. BGP 195. 100. 0. 8/30 194. 100. 0. 0/23 AS 20 i. BGP 195. 100. 0. 9 194. 100. 4. 0/23 ● 195. 100. 0. 6 R 3 e. BGP 195. 100. 0. 4/30 R 4 195. 100. 0. 5 Solution – Use BGP to carry routes between all routers of domain ● ● ● Two different types of BGP sessions e. BGP between routers belonging to different ASes i. BGP between each pair of routers belonging to the same AS – – Each BGP router inside ASx maintains an i. BGP session with all other BGP routers of ASx (full i. BGP mesh) Note that the i. BGP sessions do not necessarily follow physical topology

i. BGP versus e. BGP ● Differences between i. BGP and e. BGP – local-pref attribute is only carried inside messages sent over i. BGP session – Over an e. BGP session, a router only advertises its best route towards each destination ● – Usually, import and export filters are defined for each e. BGP session Over an i. BGP session, a router advertises only its best routes learned over e. BGP sessions ● ● A route learned over an i. BGP session is never advertised over another i. BGP session Usually, no filter is applied on i. BGP sessions

i. BGP and e. BGP : Example UPDATE (via e. BGP) ●Prefix: 194. 100. 0. 0/23 , ●Next. Hop: 195. 100. 0. 1 ●ASPath: AS 10 194. 100. 0. 0/23 194. 100. 2. 0/23 195. 100. 0. 2 195. 100. 0. 0/30 R 1 195. 100. 0. 1 e. BGP AS 20 AS 30 R 2 195. 100. 0. 6 195. 100. 0. 10 R 3 195. 100. 0. 8/30 i. BGP e. BGP UPDATE (via i. BGP) 195. 100. 0. 9 195. 100. 0. 4/30 ●Prefix: 194. 100. 0. 0/23 UPDATE (via e. BGP) , R 4 195. 100. 0. 5 ●Prefix: 194. 100. 0. 0/23 ●Next. Hop: 195. 100. 0. 1 , ●ASPath: AS 10 194. 100. 4. 0/23 ●Next. Hop: 195. 100. 0. 5 ●Local-pref: 1000 ●ASPath: AS 20: AS 10 ● Note that the next-hop and the AS-Path of BGP update messages are only updated when sent over an e. BGP session

i. BGP and e. BGP Packet Forwarding 194. 100. 2. 0/23 AS 30 AS 10 195. 100. 0. 2 R 2 195. 100. 0. 0/30 195. 100. 0. 10 R 1 195. 100. 0. 1 e. BGP 195. 100. 0. 8/30 194. 100. 0. 0/23 AS 20 i. BGP 195. 100. 0. 9 194. 100. 4. 0/23 195. 100. 0. 6 R 3 e. BGP 195. 100. 0. 4/30 R 4 195. 100. 0. 5 BGP routing table of R 2 194. 100. 0. 0/23 via 195. 100. 0. 1 BGP routing table of R 4 194. 100. 0. 0/23 via 195. 100. 0. 1 IGP routing table of R 2 195. 100. 0. 0/30 West 195. 100. 0. 4/30 via 195. 100. 0. 9 195. 100. 0. 8/30 South 194. 100. 0. 4/23 via 195. 100. 0. 9 194. 100. 2. 0/23 North IGP routing table of R 4 195. 100. 0. 0/30 via 195. 100. 0. 10 195. 100. 0. 4/30 East 195. 100. 0. 8/30 North 194. 100. 2. 0/23 via 195. 100. 0. 10 194. 100. 0. 4/23 West

i. BGP and e. BGP Packet Forwarding (2) 194. 100. 2. 0/23 AS 30 AS 10 195. 100. 0. 2 R 2 195. 100. 0. 0/30 195. 100. 0. 10 R 1 195. 100. 0. 1 e. BGP 195. 100. 0. 8/30 194. 100. 0. 0/23 AS 20 i. BGP 195. 100. 0. 9 BGP routing table of R 4 194. 100. 0. 0/23 via 195. 100. 0. 1 194. 100. 4. 0/23 IGP routing table of R 4 195. 100. 0. 0/30 via 195. 100. 0. 10 195. 100. 0. 4/30 East 195. 100. 0. 8/30 North 194. 100. 2. 0/23 via 195. 100. 0. 10 194. 100. 4. 0/23 West 195. 100. 0. 6 R 3 e. BGP 195. 100. 0. 4/30 R 4 195. 100. 0. 5 Forwarding of R 4 194. 100. 0. 0/23 via 195. 100. 0. 10 195. 100. 0. 0/30 via 195. 100. 0. 10 195. 100. 0. 4/30 East 195. 100. 0. 8/30 North 194. 100. 2. 0/23 via 195. 100. 0. 10 194. 100. 4. 0/23 West The forwarding table of a router is thus built based on both the IGP and the BGP tables

Using non-BGP routers 194. 100. 2. 0/23 AS 30 AS 10 194. 100. 0. 0/23 195. 100. 0. 2 R 2 195. 100. 0. 0/30 R 1 195. 100. 0. 1 e. BGP AS 20 i. BGP 195. 100. 0. 6 R 5 e. BGP R 3 12. 0. 0. 0/8 195. 100. 0. 4/30 194. 100. 4. 0/23 ● R 4 195. 100. 0. 5 Problem – What happens when there are internal backbone routers between BGP routers inside an AS ? ● ● i. BGP session between BGP routers is easily established when IGP is running since i. BGP runs over TCP connection How to populate the routing table of the backbone routers to ensure that they will be able to route any IP packet ?

Using non-BGP routers (2) 194. 100. 2. 0/23 AS 30 AS 10 194. 100. 0. 0/23 195. 100. 0. 2 R 2 195. 100. 0. 0/30 R 1 195. 100. 0. 1 e. BGP AS 20 i. BGP 195. 100. 0. 6 R 5 R 3 e. BGP 195. 100. 0. 4/30 194. 100. 4. 0/23 ● R 4 195. 100. 0. 5 First solution – Use tunnels between BGP routers to encapsulate interdomain packets ● GRE tunnel – ● Needs static configuration and be careful with MTU issues MPLS tunnel – Can be dynamically established in MPLS enabled backbone

MPLS in large ISP networks ● Only one BGP table lookup inside the AS – Use a hierarchy of labels ● ● top label is used to reach egress router second label is used to reach e. BGP peer RG RH RA B 4 R 1 B 3 RB R 2 RC RD R 5 Egress Border router – packets are label switched B 6 AS 1 Ingress Border router – Maintains full BGP routing table – Attach two labels based on routing table R 7 RE RF

Using non-BGP routers (3) 194. 100. 2. 0/23 AS 30 AS 10 194. 100. 0. 0/23 195. 100. 0. 2 R 2 195. 100. 0. 0/30 R 1 195. 100. 0. 1 e. BGP AS 20 i. BGP 195. 100. 0. 6 R 5 e. BGP R 3 12. 0. 0. 0/8 195. 100. 0. 4/30 194. 100. 4. 0/23 ● R 4 195. 100. 0. 5 Second solution – – Use IGP (OSPF/IS-IS - RIP) to redistribute inter-domain routes to internal backbone routers Drawbacks ● ● Size of BGP tables may completely overload the IGP Make sure that BGP routes learned by R 2 and injected inside IGP will not be re-injected inside BGP by R 4 !

Using non-BGP routers (4) 194. 100. 2. 0/23 AS 30 AS 10 194. 100. 0. 0/23 195. 100. 0. 2 R 2 195. 100. 0. 0/30 R 1 195. 100. 0. 1 e. BGP AS 20 i. BGP 194. 100. 4. 0/23 ● 195. 100. 0. 6 i. BGP R 5 e. BGP R 3 12. 0. 0. 0/8 195. 100. 0. 4/30 R 4 195. 100. 0. 5 Third solution – – Run BGP on internal backbone routers Internal backbone routers need to participate in i. BGP full mesh ● Internal backbone routers receive BGP routes via i. BGP but never advertise any routes – Remember: a route learned over an i. BGP session is never advertised over another i. BGP session

The roles of IGP and BGP 194. 100. 2. 0/23 AS 10 195. 100. 0. 2 R 2 195. 100. 0. 0/30 R 1 195. 100. 0. 1 194. 100. 0. 0/23 e. BGP AS 20 i. BGP 194. 100. 4. 0/23 – R 5 i. BGP R 4 AS 30 195. 100. 0. 4/30 195. 100. 0. 6 e. BGP Role of the IGP inside AS 20 ● – i. BGP 12. 0. 0. 0/8 Distribute internal topology and internal addresses R 2 -R 4 -R 5) Role of BGP inside AS 20 ● ● R 3 Distribute the routes towards external destinations IGP must run to allow BGP routers to establish i. BGP sessions

The i. BGP full mesh ● Drawback – N*(N-1)/2 i. BGP sessions for N routers R R R R i. BGP session R

Outline ● Organization of the global Internet ● BGP basics ● BGP in large networks – – – ● The needs for i. BGP Confederations and Route Reflectors The dynamics of BGP Inter-domain traffic engineering with BGP

How to scale i. BGP in large domains ? ● Confederations – Divide the large domain in smaller sub-domains ● ● Use i. BGP full mesh inside each sub-domain Use e. BGP between sub-domains Confederation : AS 20 R R Member-AS AS 65001 – R R R i. BGP session e. BGP session Each router is configured with two AS numbers ● ● – R Member-AS AS 65002 Its confederation AS number Its Member-AS AS number Usually, a single IGP covers the whole domain

Confederations: example UPDATE (via e. BGP) ●Prefix: 1. 0. 0. 0/8, ●ASPath: AS 10 RX AS 20 e. BGP R 2 AS 10 R 6 i. BGP e. BGP i. BGP R 1 AS 65021 i. BGP R 3 ● ● ● AS 65020 R 5 e. BGP RY AS 30 On the e. BGP session between R 2 and RX, R 2 belongs to AS 20 On the e. BGP session between R 5 and RY, R 5 belongs to AS 20 On the e. BGP session between R 1 and R 6, R 1 belongs to AS 65020 and R 6 belongs to AS 65021

Confederations : example (2) UPDATE (via i. BGP) ●Prefix: 1. 0. 0. 0/8, ●ASPath: AS 10 e. BGP RX AS 20 R 2 AS 10 i. BGP e. BGP i. BGP R 1 UPDATE (via e. BGP) ●Prefix: 1. 0. 0. 0/8, ●ASPath: [AS 65020]: AS 10 R 6 AS 65021 i. BGP R 3 ● AS 65020 R 5 e. BGP RY AS 30 When propagating an UPDATE via e. BGP to another router of the same confederation, R 1 inserts its Member-AS number in the AS_PATH

Confederations : example (3) ● When propagating an UPDATE via e. BGP to a router outside its confederation, R 5 removes the internal path from the AS_Path and inserts its Confederation AS number in the AS_PATH RX e. BGP AS 20 R 2 AS 10 i. BGP e. BGP i. BGP R 1 i. BGP R 3 ● UPDATE (via i. BGP) ●Prefix: 1. 0. 0. 0/8, ●ASPath: [AS 65020]: AS 10 AS 65020 R 6 AS 65021 UPDATE (via e. BGP) Prefix: 1. 0. 0. 0/8, i. BGP ASPath: ● ● AS 20: AS 10 R 5 e. BGP RY AS 30 In practice, BGP confederations are particularly useful when two companies or two distinct ASes from the same company must be merged in a single AS

Route reflectors An alternative to confederations ● Route reflectors (RFC 2796) – A route reflector is a special router that is allowed to propagate the routes learned over i. BGP sessions on other i. BGP sessions Normal i. BGP full mesh e. BGP R 2 i. BGP with one route reflector e. BGP i. BGP R 2 i. BGP R 1 i. BGP e. BGP R 3 RR i. BGP e. BGP R 3 Route Reflector

Behavior of a Route Reflector ● Two types of i. BGP peers of a route reflector R 1 R 2 i. BGP . . RN i. BGP RR clients peers ( do not participate in i. BGP full mesh) RR i. BGP RX i. BGP RZ i. BGP RY i. BGP Non-clients peers (participate in i. BGP full mesh)

Behavior of a Route Reflector ● Route received from an e. BGP session or a client peer – Select best path – Advertise to ● ● RR clients peers. . R 2 R 1 All client peers All non-client peers i. BGP RN i. BGP RR ● i. BGP Route received from non-client peer – Select best path – Advertise to ● All client peers i. BGP RX i. BGP RZ i. BGP RY i. BGP Non-clients peers

Fault tolerance of route reflectors ● How to avoid having the RR as a single point of failure ? – Solution ● Allow each client peer to be connected at 2 RRs R 1 RR clients peers. . R 2 i. BGP RR 1 – i. BGP RN i. BGP RR 2 Issue ● Configuration errors may cause redistribution loops – – ORIGINATOR_ID used to carry router ID of originator of route CLUSTER_LIST contains the list of RR that sent the UPDATE message inside the current AS

Route reflectors : an example UPDATE (via e. BGP) ●Prefix: 1. 0. 0. 0/8, ●ASPath: AS 10 AS 20 e. BGP RX R 2 AS 10 i. BGP UPDATE (via e. BGP) ●Prefix: 1. 0. 0. 0/8, ●ASPath: AS 10 e. BGP RZ ● ● ● RR 1 RR 6 i. BGP R 3 R 5 R 2 and R 3 are clients of Route Reflector RR 1 and RR 6 are in i. BGP full mesh R 5 is client of Route Reflector RR 6 e. BGP RY AS 30

Route reflectors : an example (2) UPDATE (via i. BGP) ●Prefix: 1. 0. 0. 0/8, ●ASPath: AS 10 AS 20 ●Nexthop: RX e. BGP RX R 2 AS 10 i. BGP RR 1 RR 6 i. BGP R 3 e. BGP RZ ● UPDATE (via i. BGP) ●Prefix: 1. 0. 0. 0/8, ●ASPath: AS 10 ●Nexthop: RZ R 5 e. BGP RY AS 30 RR 1 will select its best path towards 1. 0. 0. 0/8 and will re-advertise it by adding the ORIGINATOR_ID and the CLUSTERID

Route reflectors : an example (3) e. BGP RX UPDATE (via i. BGP) ●Prefix: 1. 0. 0. 0/8, ●ASPath: AS 10 ●Nexthop: RX ●ORIGINATOR_ID: R 2 AS 20 ●CLUSTER_ID: RR 1 UPDATE (via i. BGP) ●Prefix: 1. 0. 0. 0/8, ●ASPath: AS 10 ●Nexthop: RX ●ORIGINATOR_ID: R 2 ●CLUSTER_ID: RR 1 AS 10 R 2 i. BGP RR 1 e. BGP ● i. BGP R 3 RZ RR 6 R 5 e. BGP RY AS 30 RR 1 prefers the path to 1. 0. 0. 0/8 via RX-R 2 – – RR 1 advertises this path to its client peer (R 3) ● the path is not advertised to R 2 since R 2 already received it RR 1 advertises this path to its non-client peer (RR 6)

Route reflectors : an example (4) e. BGP RX UPDATE (via i. BGP) ●Prefix: 1. 0. 0. 0/8, ●ASPath: AS 10 ●Nexthop: RX ●ORIGINATOR_ID: R 2 ●CLUSTER_ID: RR 1: RR 6 AS 20 R 2 AS 10 i. BGP RR 1 RR 6 i. BGP R 3 e. BGP RZ ● RY AS 30 RR 6 advertises the path to 1. 0. 0. 0/8 via RX-R 2 – ● R 5 e. BGP to its client peer R 5 will remove ORIGINATOR_ID and CLUSTER_ID before advertising the path to RY via e. BGP

Hierarchy of route reflectors ● In large domains, a hierarchy of route reflectors can be built R 1, R 2 and R 3 are clients of route reflectors RR 1 and RR 2 ● R 5 R 4 R 1 RR 4 RR 1 R 2 RRA RR 1 and RR 2 are clients of route reflectors RRA and RRB ● R 6 R 4 and R 5 are clients of route reflector RRA ● R 3 RR 5 RR 2 RRC RRB R 6 is client of route reflectors RR 4 and RR 5 ● RRA, RRB and RRC are in full i. BGP mesh ● i. BGP session

Confederations versus Route reflectors ● Confederations – – – Solves i. BGP scaling Redundancy with i. BGP full-mesh inside each Member. AS Possible to run one IGP per Member AS Requires manual router configuration Can be used when merging domains Can lead to some routing oscillations ● Route reflectors – – – Solves i. BGP scaling Redundancy by using Redundant RRs Usually a single IGP for the whole AS Requires manual router configuration Can lead to some routing oscillations