External Routing BGP JeanYves Le Boudec Fall 2009

External Routing BGP Jean-Yves Le Boudec Fall 2009 Self-Organization 1

Contents A. What Inter-Domain Routing does 1. Inter-Domain Routing 2. Policy Routing 3. Route Aggregation B. How BGP works 1. How it works 2. Interaction BGP—IGP—Packet Forwarding 3. Other Bells and Whistles C. Examples D. Illustrations and Statistics 2

A. What BGP Does 1. Inter-Domain Routing Why invented ? The Internet is too large to be run by one routing protocol Hierarchical routing is used the Internet is split into Domains, or Autonomous Systems with OSPF: large domains are split into Areas Routing protocols are said interior: (Internal Gateway Protocols, IGPs): inside ASs: RIP, OSPF (standard), IGRP (Cisco) exterior: between ASs: EGP (old) and BGP-1 to BGP-4 (today), IDRP (tomorrow, maybe) 3

What is an ARD ? An AS ? ARD = = Autonomous Routing Domain routing domain under one single administration one or more border routers all subnetworks inside an ARD should be connected should learn about other subnetwork prefixes - the routing tables of internal routers should contain entries of all destination of the Internet AS = = Autonomous System ARD with a number (“AS number”) AS number is 16 bits public: 1 - 64511 private: 64512 – 65535 ARDs that are do not need a number are typically those with default route to the rest of the world Examples AS 1942 - CICG-GRENOBLE, AS 2200 - Renater AS 559 - SWITCH Teleinformatics Services AS 5511 – OPENTRANSIT EPFL: one ARD, no number 4

the figure shows three domains, or ARDs can be transit (B and D), stub (A) or multihomed (C). Only non stub domains need an AS number. ARD D D 1 D 4 area 1 BGP-4 D 6 A 1 A 4 area 0 D 3 area 2 D 2 BGP-4 D 5 OSPF C 1 BGP-4 OSPF A 2 B 4 BGP-4 B 1 B 2 A 3 Autonomous Routing Domain A IGRP B 3 C 2 C 4 C 3 ARD C ARD B 5

What does BGP do ? BGP is a routing protocol between ARDs. It is used to compute paths from one router in one ARD to any network prefix anywhere in the world There are only two levels for BGP : Inter-domain within domain The method of routing is Path vector With policy 6

Path Vector Routing What is the requirement ? Find best routes, in a sense that can be decided by every ARD using their own criteria How does it work ? a route between neighbours is (path: dest) where path is a sequence of AS numbers and dest is an IP prefix example: B A: n 1 every AS uses its own rules for deciding which path is better BGP table keeps a record of best paths to all destinations AS announces only the best paths it knows Q. Explain how E can choose the best paths to n 1 and n 2 ? Q. How can loops be avoided ? solution n 5 A: n 1, n 2 B BGP table in E B A: n 1, n 2 B: n 5 n 1, n 2 E n 4 A dest D C A: n 1, n 2 n 3 C A: n 1, n 2 C: n 3 n 1 n 2 n 3 n 4 n 5 AS path B A D C D B D C A: n 1, n 2 D C: n 3 D: n 4 7

Border Gateways, E- and I-BGP A router that runs BGP is called a BGP speaker At the boundary between 2 ARDs there are 2 BGP speakers, one in each domain Q: compare to OSPF Inside one ARD there are usually several BGP speakers They all talk to each other, to exchange what they have learnt Using “Internal BGP” (I-BGP) Over TCP connections, full mesh called the “BGP mesh” I-BGP is the same as E-BGP except for one rule: routes learned from a node in the mesh are not repeated inside the mesh X: n 1 D 1 A E-BGP C I-BGP B D 2 D A->C: D 1 X: n 1 C->E: D 1 X: n 1 E->G: D 3 D 1, X: n 1 D 3 E D 4 G F E-BGP H D 5 8

Q 1: Which of the following announcements do you think router D will send to router E ? 1. D->E: D 2 X: n 1 2. D->E: D 1 X: n 1 A: 1 only; 2 cannot be sent because over I-BGP we do not repeat a route learnt from another BGP speaker in the same ARD Q 2: Assume that in the figure below the router R does not run BGP. Is this possible ? A: yes. Q 3: Do BGP speakers in the same ARD need to be on-link ? A: no; C and F are not directly connected (are not on-link) X: n 1 D 1 A solution E-BGP D 3 R E D 4 G D 2 D C I-BGP B F E-BGP H D 5 9

2. Policy Routing Why invented ? Interconnection of ASs is self-organized point to point links between networks: ex: EPFL to Switch, Switch to Telianet interconnection points: NAP (Network Access Point), MAE (Metropolitan Area Ethernet), CIX (Commercial Internet e. Xchange), GIX (Global Internet e. Xchange), IXP, SFINX, LINX Mainly 3 types of relations, depending on money flows customer: EPFL is customer of Switch. EPFL pays Switch provider: Switch is provider for EPFL; Switch is paid by EPFL peer: EPFL and CERN are peers: costs of interconnection is shared the type of relation is negotiated in bilateral agreements there is no architecture rule, just business 10

What is the Goal of Policy Routing ? Example: ISP 3 ISP 2 ISP 1 C 2 C 3 n 2 C 1 peers provider customer ISP 3 -ISP 2 is transatlantic link, cost shared between ISP 2 and ISP 3 - ISP 1 is a local, inexpensive link Ci is customer of ISPi, ISPs are peers It is advantageous for ISP 3 to send traffic to n 2 via ISP 1 does not agree to carry traffic from C 3 to C 2 ISP 1 offers a “transit service” to C 1 and a “nontransit” service to ISP 2 and ISP 3 The goal of “policy routing” is to support this and other similar requirements 11

How does Policy Routing Work ? Policy routing is implemented by rules imposed to BGP routers inside an AS, who may ISP 3 ISP 2 Example ISP 1 C 2 C 3 n 2 n 3 C 1 refuse to import or announce some paths modify the attributes that control which path is preferred (see later) ISP 1 announces to ISP 3 all networks of C 1 – so that C 1 can be reached by all sources in the world ISP 1 announces to C 1 all routes it has learnt from ISP 3 and ISP 2 – so that C 1 can send traffic to all destinations in the world ISP 2 announces “ISP 2 n 2” to ISP 3 and ISP 1 ; assume that ISP 1 annouces “ISP 1 ISP 2 n 2” to ISP 3 has two routes to n 2: “ISP 2 n 2” and “ISP 1 ISP 2 n 2”; assume that ISP 3 gives preference to the latter packets from n 3 to n 2 are routed via ISP 1 – undesired solution: ISP 1 announces to ISP 3 only routes to ISP 3’s customers 12

Typical Policy Routing Rules Provider (ISP 1) to customer (C 1) ISP 3 ISP 2 Customer (C 1) to Provider (ISP 1) ISP 1 C 2 C 3 announce all routes learnt from other ISs import only routes that belong to domain C 1 example: import from EPFL only one route 128. 178/15 announce all routes that belong to domain C 1 import all routes Peers (ISP 1 to ISP 3) C 1 announce only routes to all customers of ISP 1 import only routes to ISP 3’s customer these routes are defined as part of peering agreement The rules are defined by every AS (selforganization) and implemented in all BGP speakers in one AS 13

3. Aggregation Domains that do not have a default route (i. e. all transit ISPs) must know all routes in the world (> 170 000 prefixes) in IP routing tables unless default routes are used in BGP announcements Aggregation is a way to reduce the number of routes 14

Aggregation Example 1 AS 1 193. 212. 0. 0/24 193. 212. 0. 0/23 AS 2 193. 212. 0. 0/23 AS 4 193. 212. 1. 0/24 Assume AS 3 aggregates the routes received from AS 1 and AS 2 AS 1: 193. 212. 0. 0/24 AS 2: 193. 212. 1. 0/24 AS 3: 193. 212. 0. 0/23 AS 4: 193. 212. 0. 0/23 AS_PATH: 1 AS_PATH: 2 AS_PATH: 3 {1 2} AS_PATH: 4 3 {1 2} 15

Aggregation Example 2 AS 1 197. 8. 2. 0/24 AS 3 AS 2 197. 8. 2. 0/23 AS 4 197. 8. 3. 0/24 n 2= 197. 8. 3. 0/24 197. 8. 2. 0/23 n 4 197. 8. 3. 0/24 AS 4 receives 197. 8. 2. 0/23 197. 8. 3. 0/24 AS_PATH: 3 {1 2} AS_PATH: 2 and assume that AS 4 exports only the first route (aggregation is done) But assume both routes are injected into AS 4’s local routing tables; Q: what happens to packets from n 4 to n 2 ? If AS 4 injects only the aggregate route into its local routing tables, what is the answer now ? solution 16

Example Without Aggregation AS 1 197. 8. 2. 0/24 AS 3 AS 2 197. 8. 2. 0/23 197. 8. 2. 0/24 197. 8. 3. 0/24 AS 4 197. 8. 3. 0/24 Q: If AS 3 does not aggregate, what are the routes announced by AS 4 ? Is there any benefit ? solution 17

Morale Aggregation should be performed whenever possible when all aggregated prefixes have the same path (example 1) when all aggregated prefixes have the same path before the aggregation point (example 2 ) An AS can decide to Aggregate several routes when exporting them But still maintain different routing entries inside its domain (example 2) 18

B. BGP (Border Gateway Protocol) 1. How it works, Fundamental Examples BGP-4, RFC 1771 BGP routers talk to each other over TCP connections Maintain adjacency to peers (over TCP connection): open, keep-alive Transmit only modifications How a BGP router works (see also next slide) Receives routes from neighbours Accepts / rejects them (import policy) Processes attributes Stores in Adj-RIB-in (one per BGP peer, internal or external) Applies decision process and stores results in Loc-RIB (global to BGP speaker) (= decides which routes to accept among all present in the different Adj-RIB-ins) For every subnetwork prefix, at most 1 route is selected Sends to neighbours Decides whether to send or not (export policy) Aggregate multiple routes into one, if applicable Store result in Adj-RIB-out (one per BGP peer) and send to neighbour Only routes learnt from E-BGP are sent to an I-BGP neighbour Sends updates when Adj-RIB-out changes Write forwarding entries in its routing table, either directly or by re-distribution into IGP ; this is explained in detail in Section 2. 19

Inside BGP One BGP Router updates in updates out Import Policy Adj-RIB-in Decision Process: one best route / destination Loc-RIB Write forwarding entries Export Policy Routing Table Adj-RIB-out Internal routes IGP Static Configuration 20

Routes, RIBs, Routing Table The records sent in BGP messages are called “Routes”. Routes + their attributes are stored in the Adj-RIB-in, Loc-RIB, Adj-RIB-out. A route is made of: destination (subnetwork prefix) path to the destination (AS-PATH) Attributes Well-known Mandatory ORIGIN (route learnt from IGP, BGP or static) AS-PATH NEXT-HOP Well-known Discretionary LOCAL-PREF (see later) ATOMIC-AGGREGATE (= route cannot be dis-aggregated) Optional Transitive MULTI-EXIT-DISC (MED)(see later) AGGREGATOR (who aggregated this route) Optional Nontransitive WEIGHT (see later) In addition, like any IP host or router, a BGP router also has a Routing Table = IP forwarding table Used for packet forwarding, in real time This is not the same as a RIB, we will see the interaction later 21

The Decision Process The decision process decides which route is selected; its output is written into Loc-RIB At most one best route to exactly the same prefix is chosen Only one route to 2. 2/16 can be chosen But there can be different routes to 2. 2. 2/24 and 2. 2/16 Routes are compared against each other using the following sequence of criteria, until only one route remains: 0. Highest weight (Cisco proprietary) 1. Highest LOCAL-PREF 2. Shortest AS-PATH 3. Lowest MED, if taken seriously by this network 4. E-BGP > I-BGP 5. Shortest path to NEXT-HOP, according to IGP 6. Lowest BGP identifier 22

Fundamental Examples AS x I-BGP R 1 10. 1/16 AS=(y) 10. 2/16 AS=(y) 1. 1 E-BGP 1. 1. 1. 2 R 2 2. 2 E-BGP 10. 1/16 AS=(y) 10. 2/16 AS=(y) 2. 2. 2. 1 R 4 R 3 10. 2/16 Skrivañ war an daol 10. 1/16 AS y In this simple example there are 4 BGP routers. They communicate directly or indirectly via E-BGP or I-BGP, as shown on the figure. There are 2 ASs, x and y. We do not show the details of the internals of y. R 3 and R 4 send the BGP messages shown. In the next slide, we show only the following attributes in the RIBs: destination, path, NEXT-HOP 23

R 1 has the following RIBs, initially empty: Adj-RIB-In from R 3 Adj-RIB-In from R 2 Loc-RIB Adj-RIB-Out to R 3 Adj-RIB-Out to R 2 R 1 receives from R 3 two announcements, made of the routes 10. 1/16 AS=(y) and 10. 2/16 AS=(y). R 1 associates the NEXT-HOP attribute 1. 1. 1. 2 to these routes and writes them in its Adj. RIB-In: Adj-RIB-In from R 3 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 10. 2/16 AS=y NEXT-HOP=1. 1. 1. 2 Adj-RIB-In from R 2 Loc-RIB Adj-RIB-Out to R 3 Adj-RIB-Out to R 2 R 1 applies its decision process, and selects both routes: Adj-RIB-In from R 3 10. 1/16 AS=y 10. 2/16 AS=y Adj-RIB-In from R 2 Loc-RIB 10. 1/16 AS=y 10. 2/16 AS=y Adj-RIB-Out to R 3 Adj-RIB-Out to R 2 NEXT-HOP=1. 1. 1. 2 R 1 prepares to send these new routes to all its EBGP peers by writing them into Adj-RIB-Out. However, the message is not sent to R 3 as it would create a loop in the AS-path. R 1 prepares to send these new routes to all its IBGP peers by writing them into Adj-RIB-Out to R 2: Adj-RIB-In from R 3 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 10. 2/16 AS=y NEXT-HOP=1. 1. 1. 2 Adj-RIB-In from R 2 Loc-RIB 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 10. 2/16 AS=y NEXT-HOP=1. 1. 1. 2 Adj-RIB-Out to R 3 Adj-RIB-Out to R 2 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 10. 2/16 AS=y NEXT-HOP=1. 1. 1. 2 Since Adj-RIB-Out to R 2 changed, a BGP message is sent to R 2 Similarly, at some time R 1 will receive from R 2 a BGP message with the routes 10. 1/16 AS=y NEXT -HOP=2. 2. 2. 1 and 10. 2/16 AS=y NEXTHOP=2. 2. 2. 1: Adj-RIB-In from R 3 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 10. 2/16 AS=y NEXT-HOP=1. 1. 1. 2 Adj-RIB-In from R 2 10. 1/16 AS=y NEXT-HOP=2. 2. 2. 1 10. 2/16 AS=y NEXT-HOP=2. 2. 2. 1 Loc-RIB 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 10. 2/16 AS=y NEXT-HOP=1. 1. 1. 2 Adj-RIB-Out to R 3 Adj-RIB-Out to R 2 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 10. 2/16 AS=y NEXT-HOP=1. 1. 1. 2 24

R 1 applies again its decision process. Now it has several possible routes to each prefix. The first applicable rule in the decision process (slide “The Decision Process”) says that if a route is learnt from E-BGP it has precedence over a route learnt from I-BGP. Since all routes in Adj-RIB-In from R 2 are learnt from I-BGP, and all routes in Adj-RIB -In from R 3 are learnt from E-BGP, the winners are the latter, so there is no change in Loc-RIB: Adj-RIB-In from R 3 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 10. 2/16 AS=y NEXT-HOP=1. 1. 1. 2 Adj-RIB-In from R 2 10. 1/16 AS=y NEXT-HOP=2. 2. 2. 1 10. 2/16 AS=y NEXT-HOP=2. 2. 2. 1 Loc-RIB 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 10. 2/16 AS=y NEXT-HOP=1. 1. 1. 2 Adj-RIB-Out to R 3 Adj-RIB-Out to R 2 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 10. 2/16 AS=y NEXT-HOP=1. 1. 1. 2 Since there is no change in Loc-RIB there is no change in Adj-RIB-Out and therefore no message is sent by R 1. 25

Fundamental Examples, continued AS x I-BGP R 1 10. 1/16 AS=(y) E-BGP 1. 1. 1. 2 R 3 10. 1/16 R 21 I-BGP R 22 2. 2 3. 3. 3. 2 10. 2/16 AS=(y) E-BGP 10. 2/16 AS=(y) 2. 2. 2. 1 3. 3. 3. 1 R 4 10. 2/16 AS y There are now 3 BGP routers in AS x. Note that the 3 BGP in AS x routers must have TCP connections with each other (same in AS y, but not shown on figure). An IGP (for example OSPF) also runs on R 1, R 21 and R 22. All link costs are equal to 1. The announcements made by R 3 and R 4 are different, as shown on the figure. 26

Here is a possible sequence of messages and the resulting states of RIBs, at R 1, after message is processed: R 1 receives message 10. 1/16 AS=(y) from R 3 over E_BGP: Adj-RIB-In from R 3 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 Adj-RIB-In from R 21 Loc-RIB 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 Adj-RIB-Out to R 3 Adj-RIB-Out to R 22 Adj-RIB-Out to R 21 R 1 applies its decision process, accepts the route and sends it to R 22 and R 21 Adj-RIB-In from R 3 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 Adj-RIB-In from R 21 Loc-RIB 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 Adj-RIB-Out to R 3 Adj-RIB-Out to R 22 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 Adj-RIB-Out to R 21 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 Assume R 22 also received the E-BGP messages shown on the figure and did its job. R 1 receives message 10. 2/16 AS=y, NEXT-HOP=2. 2. 2. 1 from R 22 over I-BGP, and the decision process accepts it: Adj-RIB-In from R 3 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 Adj-RIB-In from R 22 10. 2/16 AS=y, NEXT-HOP=2. 2. 2. 1 Adj-RIB-In from R 21 Loc-RIB 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 10. 2/16 AS=y NEXT-HOP=2. 2. 2. 1 Adj-RIB-Out to R 3 Adj-RIB-Out to R 22 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 Adj-RIB-Out to R 21 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 Assume R 21 also received the E-BGP messages shown on the figure and did its job. R 1 receives message 10. 2/16 AS=y, NEXT-HOP=3. 3. 3. 1 from R 21 over I-BGP: Adj-RIB-In from R 3 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 Adj-RIB-In from R 22 10. 2/16 AS=y, NEXT-HOP=2. 2. 2. 1 Adj-RIB-In from R 21 10. 2/16 AS=y, NEXT-HOP=3. 3. 3. 1 Loc-RIB 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 10. 2/16 AS=y NEXT-HOP=2. 2. 2. 1 Adj-RIB-Out to R 3 Adj-RIB-Out to R 22 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 Adj-RIB-Out to R 21 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 The decision process now has to choose between two routes with same destination prefix 10. 2/16. Both were learnt from I-BGP, so we apply criterion 5 in slide “The Decision Process”. The distance, computed by the IGP, to 2. 2. 2. 1 is 3 and the distance to 3. 3. 3. 1 is 2. Thus the route that has NEXTHOP=3. 3. 3. 1 is preferred by the decision process: Adj-RIB-In from R 3 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 Adj-RIB-In from R 22 10. 2/16 AS=y, NEXT-HOP=2. 2. 2. 1 Adj-RIB-In from R 21 10. 2/16 AS=y, NEXT-HOP=3. 3. 3. 1 Loc-RIB 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 10. 2/16 AS=y NEXT-HOP=3. 3. 3. 1 Adj-RIB-Out to R 3 Adj-RIB-Out to R 22 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 Adj-RIB-Out to R 21 10. 1/16 AS=y NEXT-HOP=1. 1. 1. 2 Since the new message is learnt from I-BGP it is not put into any Adj-RIB-Out 27

Questions What is the difference between I-BGP and IGP ? What is the difference between I-BGP and E-BGP ? What is the difference between a RIB and a Routing Table ? 28

How are routes injected into BGP propagates route information, but how is this bootstrapped ? Two methods Static configuration: tell this BGP router which are the prefixes owned by this domain Import from IGP: tell this router to copy all the prefixes that the IGP has learnt Assumes the IGP either does not propagate external prefixes, or has a way to differentiate them Such routes are sent to E-BGP neighbours only, with ORIGIN=IGP 29

2. How Forwarding Entries learnt by BGP are written into Routing Tables So far, we have seen how BGP routers learn about all the prefixes in the world. It remains to see how they inform all the routers in their AS about these prefixes ( « write forwarding entries » in slide « Inside BGP » ). This is the topic of this section. There are two possible ways for this, usually mutually exclusive: Redistribution : routes learnt by BGP are passed to IGP (ex: OSPF) Called “redistribution of BGP into OSPF” OSPF propagates the routes using type 4 LSAs to all routers in OSPF cloud Not found much in practice Injection : routes learnt by BGP are written into the forwarding table of this router Routes do not propagate; this helps only this router 30

Redistribution Example R 6 R 5 E-BGP 2. 2 18. 1/16 IGP (OSPF) AS z R 5 advertises 18. 1/16 to R 6 via E-BGP R 6 transmits it to R 2 via I-BGP AS x I-BGP R 1 R 2 IGP (OSPF) E-BGP TCP connection between R 6 and R 2 R 4 (redistribute BGP into IGP ) R 6 injects 18. 1/16 into IGP (OSPF) OSPF propagates 18. 1/16 (type 4 LSA) and updates forwarding tables After OSPF converges, R 1, R 2 now have a route to 18. 1/6 R 2 advertises route to R 4 via E-BGP AS y (synchronize with IGP) R 2 must wait for the OSPF entry to 18. 1/6 before advertising via E-BGP Packet to 18. 1/16 from AS y finds forwarding table entries in R 2, R 1 and R 6 31

Example with Re-Distribution by ___, F learns from G the route D 2 -D 1 -n 1 C redistributes the external route D 2: n 2 into OSPF by ____, D learns the route D 2: n 2; by ____ D learns the route DYD 2: n 2; D selects D 2: n 2 and does not redistribute it to OSPF by ___, B learns the route D 2: n 2 by ___, A learns the route DX: D 2: n 2 by ___, L learns the route to n 2 via C D 1: n 1 D 1 = EPFL n 1 D 2 : n 2 B L DX solution F D 2 = UCB n 2 C A D 1: n 1 G D E DY D 2: n 2 DY D 2 : n 2 physical link TCP connection 32

Re-Distribution is Considered Harmful In practice, operators avoid re-distribution of BGP into IGP Large number of routing entries in IGP Convergence time after failures is large if IGP has many routing table entries Therefore, injection is used instead Injection assumes that routers in the AS employ recursive table lookup When IP packet is submitted to router, the forwarding table may indicate a “NEXT-HOP” which is not on-link with router A second table lookup needs to be done to resolve the next-hop into an onlink neighbour in practice, second lookup may be done in advance – not in real time– by preprocessing the routing table 33

Example: Recursive Table Lookup At R 1, data packet to 10. 1. x. y is received The forwarding table at R 1 is looked up Q: what are the next events ? Forwarding Table at R 1 To NEXT-HOP layer-2 addr 10. 1/16 2. 2. 2. 63 N/A 2. 2. 2. 63 2. 2. 2. 33 x 09: F 1: 6 A: 33: 76: 21 2. 2. 2. 33 R 1 R 2 2. 2. 2. 63 10. 1/16 R 3 2. 2. 2. 93 solution 34

Injection Example R 6 R 5 E-BGP 2. 2 18. 1/16 IGP (OSPF) AS z AS x I-BGP R 1 R 2 IGP R 5 advertises 18. 1/16, NEXT-HOP = 2. 2 to R 6 via E-BGP R 4 R 6 injects 18. 1/16, NEXT-HOP = 2. 2 into its local forwarding table (does not re-distribute into OSPF) R 2 learns route from R 6 via I-BGP AS y R 2 injects 18. 1/16, NEXT-HOP = 2. 2 into its local forwarding table Data packet to 18. 1. 2. 3 is received by R 2 Recursive table lookup at R 2 can be used Q: there is a problem at R 1: how can we solve it ? solution 35

Injection in Practice Requires all Routers to Run BGP AS x R 6 2. 2. 20. 1 R 5 E-BGP 2. 2 18. 1/16 IGP (OSPF) AS z I-BGP R 1 R 2 IGP E-BGP Practical solution often deployed All core routers also run I-BGP (in addition to IGP) Routes learnt from BGP are injected into local forwarding table Recursive table lookup is done at all routers Q: repeat the sequence of previous slide with this new assumption R 4 AS y Potential problem: I-BGP mesh -> use reflectors IGP handles only internal networks – very few solution 36

Questions What is the difference between injection and re-distribution ? Can BGP routes be learnt by a non BGP router ? What is recursive table lookup ? Where is it needed ? 37

3. Other Bells and Whistles MULTI-EXIT-DISC (MED) packet to 10. 1. 2. 3 packet to 10. 2. 3. 4 AS x R 2 R 1 10. 1/16 MED=10 10. 2/16 MED=50 2. 2 E-BGP 1. 1 E-BGP 10. 1/16 MED=50 10. 2/16 MED=10 R 4 R 3 10. 2/16 10. 1/16 AS y One AS connected to another over several links ex: multinational company connected to worldwide ISP AS y advertises its prefixes with different MEDs (low = preferred) If AS x accepts to use MEDs put by ASy: traffic goes on preferred link 38

MED Example Q 1: by which mechanisms will R 1 and R 2 make sure that packets to ASy use the preferred links ? Q 2: router R 3 crashes; can 10. 1/16 still be reached ? explain the sequence of actions. solution 39

MED Question Q 1: Assume now ASx and ASy are peers (ex: both are ISPs). Explain why ASx is not interested in taking MED into account. Q 2: By which mechanisms can ASx pick the nearest route to ASy ? solution 40

LOCAL-PREF pref=100 R 6 E-BGP I-BGP E-BGP R 1 AS x I-BGP R 2 pref=10 E-BGP Used inside an AS to select a best AS path Assigned by border router when receiving route over EBGP Propagated without change over I-BGP Example R 6 associates pref=100, R 2 pref=10 R 1 chooses the largest preference bgp default local-preference pref-value 41

LOCAL-PREF Example Q 1: The link AS 2 -AS 4 is expensive. How should AS 4 set local-prefs on routes received from AS 3 and AS 2 in order to route traffic preferably through AS 3 ? Q 2: Explain the sequence of events for R 1, R 2 and R 3. 10. 1/16 AS 1: 10. 1/16 AS 2 AS 1 R 1 AS 4 R 2 AS 1: 10. 1/16 AS 3 R 3 AS 5 solution 42

LOCAL-PREF Question Q: Compare MED to LOCAL-PREF solution 43

Weight This is a route attribute given by Cisco or Zebra router It remains local to this router Never propagated to other routers, even in the same cloud Therefore there is no weight attribute in route announcements 44

Route Flap Dampening Route modification propagates everywhere Sometimes routes are flapping successive UPDATE and WITHDRAW caused for example by BGP speaker that often crashes and reboots Solution: decision process eliminates flapping routes How withdrawn routes are kept in Adj-RIN-in if comes up again soon (ie : flap), route receives a penalty if penalty ¸ suppress-limit route is not selected penalty fades out exponentially, see next slide 45

Route Flap Dampening penalty suppress-limit reuse-limit time t 1 t 2 Route suppressed at t 1, restored at t 2 46

Avoid I-BGP Mesh: Confederations AS P 1 AS P 2 AS P 3 E-BGP I-BGP AS z E-BGP AS decomposed into sub-AS private AS number similar to OSPF areas I-BGP inside sub-AS (full interconnection) E-BGP between sub-AS 47

Avoid I-BGP Mesh : Route reflectors I-BGP RR RR RR I-BGP cluster 1 cluster 2 I-BGP cluster 3 AS z E-BGP Cluster of routers one I-BGP session between one client and RR CLUSTER_ID Route reflector re-advertises a route learnt via I-BGP to avoid loops ORIGINATOR_ID attribute associated with the advertisement 48

I-BGP configuration lo 0 I-BGP AS z I-BGP configured on loopback interface (lo 0) interface always up IP address associated with the interface IGP routing guarantees packet forwarding to the interface 49

Avoid E-BGP mesh: Route server E-BGP At interconnection point Instead of n(n-1)/2 peer to peer E-BGP connections n connections to Route Server To avoid loops ADVERTISER attribute indicates which router in the AS generated the route 50

Communities Other attributes can be associated with routes in order to simplify rules. They are called « communities » Pre-defined: Example: NO-EXPORT ( a well known, pre-defined attribute) – see later for an example Defined by one AS (a label of the form ASN: x where AS= AS number, x = a 2 byte—number) 51

NO—EXPORT Written on E-BGP by one AS, transmitted on I-BGP by accepting AS, not forwarded Example: AS 2 has different routes to AS 1 but AS 2 sends only one aggregate route to AS 3 simplifies the aggregation rules at AS 2 What is the route followed by a packet sent to 2. 2. 48 received by R 4 ? 2. 2. 0/17 NO-EXPORT 2. 2/16 R 1 2. 2. 0/17 R 3 R 4 2. 2. 128/17 R 2 solution 2. 2/16 R 5 2. 2. 128/17 NO-EXPORT 2. 2/16 52

C. Examples A network operator implements their policy by selecting the rules for Import policy (accept route or not, if required modify/add/delete attributes) Export policy (export or not a route, decide which attribute value) We illustrate two scenarios: Dual Homing Hot potato routing 53

Ex 1: Stub Area Operator AS R 2 R 1 Client AS BGP not needed between Client and Operator No AS number for client R 2 learns all prefixes in Client by static configuration or RIP on link R 1—R 2 Example: EPFL and Switch what if R 1 fails ? 54

Ex 2: Stub Area, Dual Homing to Single Provider AS x Provider R 1 10. 1/16 MED=10 10. 2/16 MED=50 R 2 E-BGP 10. 1/16 MED=50 10. 2/16 MED=10 R 4 R 3 10. 1/16 Client 10. 2/16 AS y With numbered Client AS Use MED to share traffic from ISP to Client on two links Use Client IGP configuration to share traffic from Client to two links Q 1: is it possible to avoid distributing BGP routes into Client IGP ? Q 2: is it possible to avoid assigning an AS number to Client ? Q 3: is it possible to avoid BGP between Client and Provider ? solution 55

Ex 3: Stub Area, Dual Homing to Several Providers AS z AS x AS y R 2 Providers R 1 E-BGP R 4 R 3 2. 0/17 Client 2. 1/17 AS 100 Client has own address space and AS number Q: how can routes be announced between AS 100 and AS x ? AS x and AS z ? Q: assume Client wants most traffic to favour AS x. How can that be done ? solution 56

Ex 4: Hot Potato Routing Customer 1 R 11 Customer 2 R 21 ISP 2 R 12 R 22 Packets from Customer 2 to Customer 1 Both R 21 and R 22 have a route to Customer 1 Shortest path routing favours R 21 Q 1: by which mechanism is that done ? Q 2: what is the path followed in the reverse direction ? solution 57

D. Illustrations: The Switch Network www. switch. ch 58

An Interconnection Point 59

60

from www. ris. ripe. net: all routes to 128. 178. 0. 0/15 on RIPE Route Collectors Type A Next HOP 128. 178. 0. 0/ 15 2003 -10 -02 05: 49 Z 129. 250. 0. 23 2 MED 9 Origin Community RRC ID 2914 1299 559 2914: 420 2914: 2000 2914: 3000 RIPE NCC IGP 2603 3356 1299 559 2603: 666 3356: 2 3356: 86 3356: 507 3356: 666 3356: 2076 Netnod 12381: 1653 Netnod Not defined A 128. 178. 0. 0/ 15 2003 -10 -02 06: 16: 00 Z A 128. 178. 0. 0/ 15 2003 -10 -02 06: 17 Z 194. 68. 48. 1 0 IGP 12381 1653 2603 20965 559 A 128. 178. 0. 0/ 15 2003 -10 -02 06: 16: 37 Z 194. 68. 48. 1 0 IGP 12381 1653 2603 3356 1299 559 12381: 1653 Netnod A 128. 178. 0. 0/ 15 2003 -10 -02 06: 21: 08 Z 193. 10. 252. 5 0 IGP 2603 20965 559 2603: 222 2603: 666 20965: 155 Netnod A 128. 178. 0. 0/ 15 2003 -10 -02 06: 21: 17 Z 194. 68. 48. 1 0 IGP 12381 1653 2603 20965 559 12381: 1653 Netnod A 128. 178. 0. 0/ 15 2003 -10 -02 07: 24: 06 Z 129. 250. 0. 23 2 9 Not defined 2914 3549 559 2914: 420 193. 10. 252. 5 0 61

The World seen from EPFL http: //www. ris. ripe. net/bgpviz/ 62

Some statistics Source: http: //www. cidr-report. org Address prefixes ASs 63

Number of hosts 64

65

Exercise What ASs does EPFL receive service from ? What ASs does Switch receive service from ? Find the names of the networks that have these AS numbers 66

Exercise Lookup http: //rpsl. info. ucl. ac. be. to find out the relationships between Switch and other providers How does the software on this site decide whether a relationship is client, provider or peer ? 67

Further Reading Slow convergence after route suppression – “BGP path exploration”, similar to (but worse than) distance vector slow convergence. Is in the nature of path vector routing with explicit suppression. Craig Labovitz, Abha Ahuja, Abhijit Bose, Farnam Jahanian: Delayed Internet routing convergence. IEEE/ACM Trans. Netw. 9(3): 293 -306 (2001) Route flap dampening slows down convergence Zhuoqing Morley Mao, Ramesh Govindan, George Varghese, Randy H. Katz: Route flap damping exacerbates internet routing convergence. SIGCOMM 2002: 221 -233 Path vector + policy may suffer from incompatibilities (loops) The stable paths problem and interdomain routing Griffin, T. G. ; Shepherd, F. B. ; Wilfong, G. ACM/IEEE To. N April 2002, Page(s): 232 -243 68

References Timothy Griffin’s home page at Intel RFC 1771 (BGP-4) C. Huitema, “Le Routage dans l’Internet” John W. Stewart III “ BGP 4” www. ris. ripe. net : AS paths www. cidr-report. org aggregation statistics www. caida. org map of Internet rpsl. info. ucl. ac. be relations between ASs 69

Solutions 70

Path Vector Routing Q. Explain how E can choose the best paths to n 1 and n 2 A. E receives the routes “B A n 1” and “D C A n 1”. E selects as best routes the ones with shorter AS path. Q. How can loops be avoided ? A: BGP routers recognize looping announcements by the repetition of the same AS in the path. Such announcements are discarded back n 5 A: n 1, n 2 B BGP table in E B A: n 1, n 2 B: n 5 n 1, n 2 E n 4 A dest D C A: n 1, n 2 n 3 C A: n 1, n 2 C: n 3 n 1 n 2 n 3 n 4 n 5 AS path B A D C D B D C A: n 1, n 2 D C: n 3 D: n 4 71

Q 1: Which of the following announcements do you think router D will send to router E ? 1. D->E: D 2 X: n 1 2. D->E: D 1 X: n 1 A: 1 only; 2 cannot be sent because over I-BGP we do not repeat a route learnt from another BGP speaker in the same ARD Q 2: Assume that in the figure below the router R does not run BGP. Is this possible ? A: yes. Q 3: Do BGP speakers in the same ARD need to be on-link ? A: no; C and F are not directly connected (are not on-link) X: n 1 D 1 A back E-BGP D 3 R E D 4 G D 2 D C I-BGP B F E-BGP H D 5 72

Aggregation Example 2 AS 1 197. 8. 2. 0/24 AS 3 AS 2 n 2 197. 8. 2. 0/23 197. 8. 3. 0/24 AS 4 197. 8. 2. 0/23 n 4 197. 8. 3. 0/24 Q: what happens to packets from n 4 to n 2 ? A: There are two routing entries in AS 4 routers: one for 197. 8. 2. 0/23 and one for 197. 8. 3. 0/24. Longest prefix match in the packet forwarding algorithm ensures that packets to n 2 go on the direct route (sent to AS 2). If AS 4 injects only the aggregate route into its local routing tables, what is the answer now ? There is now only one entry (197. 8. 2. 0/23 ) and all packets to n 2 are sent to AS 3 back 73

Aggregation Example 3 AS 1 197. 8. 2. 0/24 AS 3 AS 2 197. 8. 2. 0/23 197. 8. 3. 0/24 n 2 197. 8. 3. 0/24 AS 4 197. 8. 2. 0/23 n 4 197. 8. 3. 0/24 AS 6 AS 5 197. 8. 3. 0/24 AS 4 receives 197. 8. 2. 0/23 197. 8. 3. 0/24 AS_PATH: 3 {1 2} AS_PATH: 6 5 2 Both routes are received by AS 4; Q. what happens for packets from n 4 to n 2? A. they go the long way Assume now that only shortest AS paths routes are injected into routing tables Q: what happens to packets from n 4 to n 2? A: they go via AS 3 back 74

Example Without Aggregation AS 1 197. 8. 2. 0/24 AS 3 AS 2 197. 8. 0. 0/23 197. 8. 2. 0/24 197. 8. 3. 0/24 AS 4 197. 8. 3. 0/24 Q: If AS 3 does not aggregate, what are the routes announced by AS 4 ? Is there any benefit ? A: 197. 8. 0. 0/23 197. 8. 2. 0/24 197. 8. 3. 0/24 AS_PATH: 4 3 1 AS_PATH: 4 2 A: there is no benefit since all routes go via AS 4 anyhow. AS 4 should aggregate. back 75

NEXT-HOP E-BGP R 5 AS x R 6 I-BGP R 1 I-BGP R 2 AS z E-BGP R 4 R 3 10. 1/16 AS y 10. 2/16 R 3 advertises 10. 2/16 to R 1, NEXT-HOP = R 4 IP address R 6 advertises 10. 2/16 to R 5, NEXT-HOP = R 6 IP address Q. where is such a scenario likely to happen ? A: in interconnection points with many providers interconnected on one LAN back 76

MED Example Q 1: by which mechanisms will R 1 and R 2 make sure that packets to ASy use the preferred links ? A: R 1 and R 2 exchange their routes to AX y via I-BGP R 1 has 2 routes to 10. 1/16, one learnt by E-BGP (MED=10), one learnt by IBGP (MED=5=); prefers the former R 1 has 2 routes to 10. 2/16, prefers the latter (route via R 2) Q 2: router R 3 crashes; can 10. 1/16 still be reached ? explain the sequence of actions. A: R 1 clears routes to ASy learnt from R 3 (keep-alive mechanism) R 2 is informed of the route suppression by I-BGP R 2 has now only 1 route to 10. 1/16 and 1 route to 10. 2/16; traffic to 10. 1/16 now goes to R 2 back 77

MED Question Q 1: Assume now ASx and ASy are peers (ex: both are ISPs). Explain why ASx is not interested in taking MED into account. A: ASx is interested in sending traffic to ASy to the nearest exit, avoiding transit inside ASx as much as possible. Thus ASx will choose the nearest route to ASy, and will ignore MEDs Q 2: By which mechanisms can ASx pick the nearest route to ASy ? A: it depends on the IGP. With OSPF: all routes to ASy are injected into OSPF by means type 5 LSAs. These LSAs say: send to router R 3 or R 4. Every OSPF router inside ASx knows the cost (determined by OSPF weights) of the path from self to R 3 and R 4. Packets to 10. 1/16 and 10. 2/16 are routed to the nearest among R 3 and R 4 (nearest = lowest OSPF cost). back 78

LOCAL-PREF Example Q 1: The link AS 2 -AS 4 is expensive. How should AS 4 set local-prefs on routes received from AS 3 and AS 2 in order to route traffic preferably through AS 3 ? A: for example: set LOCAL-PREF to 100 to all routes received from AS 3 and to 50 to all routes received from AS 2 AS 1: 10. 1/16 AS 2 R 1 10. 1/16 AS 1 AS 4 R 2 AS 1: 10. 1/16 q Q 2: Explain the sequence of events for R 1, R 2 and R 3 AS 1: 10. 1/16 AS 5 A: R 1 receives the route AS 2 AS 1 10. 1/16 over E-BGP; sets LOCAL-PREF to 50 R 2 receives the route AS 3 AS 1 10. 1/16 over E-BGP; sets LOCAL-PREF to 100 R 3 receives AS 2 AS 1 10. 1/16, LOCALPREF=50 from R 1 over I-BGP and AS 3 AS 1 10. 1/16, LOCAL-PREF=100 from R 1 over I-BGP R 3 selects AS 3 AS 1 10. 1/16, LOCALPREF=100 and installs it into local-RIB R 3 announces only AS 3 AS 1 10. 1/16 to AS 5 back 79

LOCAL-PREF Question Q: Compare MED to LOCAL-PREF A: MED is used between ASs (i. e. over E-BGP); LOCAL-PREF is used inside one AS (over I-BGP) MED is used to tell one provider AS which entry link to prefer; LOCAL-PREF is used to tell the rest of the world which AS path we want to use, by not announcing the other ones. back 80

Example with Re-Distribution by I-BGP, F learns from G the route to D 2 -D 1 -n 1 C redistributes the external route D 2: n 2 into OSPF; by I-BGP, D learns the route D 2: n 2; by E-BGP D learns the route DYD 2: n 2; D selects D 2: n 2 and does not redistribute it to OSPF I-BGP - internal BGP by I-BGP, B learns the route D 2: n 2 from C E-BGP - external BGP by E-BGP, A learns the route DX: D 2: n 2 by OSPF, L learns the route to n 2 via C D 1: n 1 D 1 = EPFL n 1 D 2 : n 2 B L DX back F D 2 = UCB n 2 C A D 1: n 1 G D E DY D 2: n 2 DY D 2 : n 2 physical link TCP connection 81

Example: Recursive Table Lookup At R 1, data packet to 10. 1. x. y is received The forwarding table at R 1 is looked up Q: what are the next events ? A: first, the nex-hop 2. 2. 2. 63 is found; a second lookup for 2. 2. 2. 63 is done; the packet is sent to MAC address x 09: F 1: 6 A: 33: 76: 21 To NEXT-HOP layer-2 addr 10. 1/16 2. 2. 2. 63 N/A 2. 2. 2. 63 2. 2. 2. 33 x 09: F 1: 6 A: 33: 76: 21 2. 2. 2. 63 2. 2. 2. 33 R 1 R 2 R 3 10. 1/16 2. 2. 2. 93 back 82

Avoid Redistribution: Combine Recursive Lookup and NEXT-HOP R 6 R 5 E-BGP 2. 2 18. 1/16 AS z IGP (OSPF) AS x I-BGP R 1 Data packet to 18. 1. 2. 3 is received by R 2 IGP Q: there is a problem at R 1: how can we solve it ? A: the problem is: Packet is sent by R 2 towards R 1, which is the neighbour towards 2. 2. At R 1, there is no forwarding entry to 18. 1/16 – blackhole ! A solution would be to use loose source routing: R 2 adds 2. 2 as loose source routing info into packet. Another solution is label switching (see MPLS module) also called tag switching. R 2 establishes a label switched path to 2. 2 and forwards the packet on this path. E-BGP R 4 AS y In practice however, another solution, as shown next. back 83

Avoid Redistribution: Practical Solution R 6 R 5 E-BGP 2. 2 18. 1/16 AS z IGP (OSPF) AS x 2. 2. 20. 1 I-BGP R 1 R 2 IGP E-BGP Q: repeat the sequence of previous slide with this new assumption R 5 advertises 18. 1/16, NEXT-HOP = 2. 2 to R 6 via E-BGP R 6 transmits 18. 1/16, NEXT-HOP = 2. 2 to R 1 and R 2 via I -BGP R 6 injects 18. 1/16, NEXT-HOP = 2. 2 into its local forwarding table R 2 injects 18. 1/16, NEXT-HOP = 2. 2 into its local forwarding table R 4 AS y Independently, IGP finds that, at R 2, packets to 2. 2. 10. 1 should be sent to R 1 Data packet to 18. 1. 2. 3 is received by R 2 At R 2, recursive table lookup determines that packet should be forwarded to R 1 At R 1, recursive table lookup determines that packet should be forwarded to R 6 At R 6, recursive table lookup determines that packet should be forwarded to 2. 2 back 84

NO—EXPORT Q: What is the route followed by a packet sent to 2. 2. 48 received by R 4 ? A: the packet is sent via R 3 and R 1 2. 2. 0/17 NO-EXPORT 2. 2/16 R 1 2. 2. 0/17 R 3 R 4 2. 2. 128/17 R 2 2. 2/16 R 5 2. 2. 128/17 NO-EXPORT 2. 2/16 back 85

Sol. Ex 2: Stub Area, Dual Homing to Single Provider AS x Provider R 1 10. 1/16 MED=10 10. 2/16 MED=50 R 2 E-BGP 10. 1/16 MED=50 10. 2/16 MED=10 R 4 R 3 10. 1/16 Client 10. 2/16 AS y Q 1: is it possible to avoid distributing BGP routes into Client IGP ? A: yes, for example: configure R 3 and R 4 as default routers in Client AS; traffic from Client AS is forwarded to nearest of R 3 and R 4. If R 3 or R 4 fails, to the remaining one Q 2: is it possible to avoid assigning an AS number to Client ? A: Yes, it is sufficient to assign to Client a private AS number: Provider translates this number to its own. Q 3: is it possible to avoid BGP between Client and Provider ? A: Yes, by running a protocol like RIP between Client and Provider and redistributing Client routes into Provider IGP. Thus Provider pretends to the rest of the world that the prefixes of Client are its own. back 86

Sol. Ex 3: Stub Area, Dual Homing to Several Providers AS z AS x AS y R 2 Providers R 1 E-BGP R 4 R 3 2. 0/17 Client 2. 1/17 AS 100 Client has own address space and AS number Q: how can routes be announced between AS 100 and AS x ? AS x and AS z ? A: R 3 announces 2. 0/17 and 2. 0/16; traffic from AS x to 2. 0/17 will flow via AS x; if R 3 fails, it will use the longer prefix and flow via Asy. ASx announces 2. 0/17 and 2. 0/16 to AS z Q: assume Client wants most traffic to prefer AS y. How can that be done ? A: R 3 announces an artificially inflated path: 100 100 : 2. 0/17. AS z will favour the path via AS y which has a shorter AS path length back 87

Sol. Ex 4: Hot Potato Routing Customer 1 R 11 Customer 2 R 21 ISP 2 R 12 R 22 Packets from Customer 2 to Customer 1 Both R 21 and R 22 have a route to Customer 1 Shortest path routing favours R 21 Q 1: by which mechanism is that done ? A: « Choice of the best route » (criteria 5), assuming alll routers in ISP 2 run BGP Q 2: what is the path followed in the reverse direction ? A: see picture. Note the asymmetric routing back 88

Exercise What ASs does EPFL receive service from ? from the previous routes, we find AS 559 (Switch) What ASs does Switch receive service from ? from the previous routes we see that there at least: AS 1299 AS 20965 AS 3549 Find the names of the networks that have these AS numbers from whois on www. ripe. net: AS 1299: Telianet AS 20965: Geant AS 3549: Global Crossing 89

Exercise Lookup http: //rpsl. info. ucl. ac. be. to find out the relationships between Switch and other providers How does the software on this site decide whether a relationship is client, provider or peer ? AS X is client of Switch if AS X accepts ANY path and announces only self (AS X) AS X is provider of Switch if AS X announces ANY path and accepts only AS Switch AS X is a peer if AS X accepts and announces only a small set of routes 90

91