Africa Interconnection Overview Terminologies By Michuki Mwangi ISOC

Africa Interconnection Overview & Terminologies By Michuki Mwangi (ISOC) Zimbabwe IXP Growth Best Practices Workshop 25 – 26 November 2015

What is the Internet Society? The Internet Society (ISOC) is a cause-based organization that works with governments, industries, and others to ensure the technologies and policies that helped develop and evolve the Internet will continue into the future. Our programs cultivate an Internet that is open to everyone, everywhere and aim to ensure that it will continue to be a tool for creativity, innovation, and economic growth. MISSION: To promote the open development, evolution, and use of the Internet for the benefit of all people throughout the world. VISION: The Internet is for everyone

Global Presence EUROPE NORTH AMERICA THE MIDDLE EAST AFRICA ASIA Chapters OCTOBER 2013 LATIN AMERICA & CARIBBEAN 100+ 65, 000+ 145+ 6 18 Chapters Worldwide Members and Supporters Regional Bureaus Countries with ISOC Offices Organization Members

Agenda q Background on Africa Interconnection q Peering and Interconnection Terminologies q Interconnection Ecosystem: The Players and Actors q Interconnection Economics: Peering Relationships

Background on Africa Interconnection:

2008 Fiber infrastructure Against IXPs in Africa 2000 – 2005: SEA-ME-WE-3 Total Capacity 480 Gbps • 17 IXPs in 15 African Countries • 12 were considered responsive based on a survey conducted in 2008 • Average number of years across all IXPs was 4. 1 yrs • 10 of 12 IXPs provided traffic stats • Highest had 200 Mbps and lowest was 300 Kbps • 1/3 of the IXPs had an open membership policy. Remaining 2/3’s were subject to regulatory or membership set criteria • 75% of the respondents had a Mandatory Multilateral peering policy • Copper and wireless connectivity was predominant over fiber • 41% of the IXPs did not charge fees and the highest fees were $9, 000 and lowest was $50 • 75% were ran by ISPA. Others by NREN, Govt. and by a not-for-profit (non-ISPA) 1993 – 2001: SAT 2 Total Capacity 560 Mbps 2001 – 2008: SAT 3 Total Capacity 340 Gbps 2001 - 2008: SAFE Total Capacity 440 Gbps

2008: Traffic distribution across 10 African IXPs

What we observed from 2008 Survey Positive Negative • There was a seemingly good linkage between IXPs and the cc. TLD • The Mandatory Multi. Lateral peering policy was a dominant obstacle to IXP growth • IXPs had relatively “good” connectivity • The role of Governments in IXPs was seemingly progressive • IXP Models were ideal “at that stage” • Membership policies were not progressive • IXP sustainability was an area of concern. • More value added services need to be implemented at the IXPs

Gaps identified • There was a lack of general knowledge on IXP best practices to enable them grow • There was no platform for sharing experiences or learning amongst the IXPs and operators • Cross-border interconnection between networks was almost non-existent • 30% of the known IXPs were unresponsive for unknown reasons • Central, North and West Africa were lagging behind in IXP deployment • Africa has a large “Internet Transit Deficit” in comparison to other regions.

Impact on Africa’s Interconnection Scene • Conducted 15 workshops in 15 Africa countries on Technical and policy issues related to IXP development • Workshops trained over 300 engineers and policy makers • Established and organized 6 Af. PIF events • Supported the establishment and activities of the Africa IXP association (Af-IX) • Provided technical assistance, equipment support to at least 10 IXPs • Created awareness with key policy stakeholders i. e UNECA, African Union, Regional Regulatory Associations, Regional Economic Communities (RECs) • Overall growth of the African peering ecosystem • Work lead to award to implement African Union AXIS project

AXIS Project • African Union awarded Internet Society two (2) contracts to implement AXIS project between 2012 – 2014 • Project focused on Capacity building and technical assistance in establishing IXPs • Overall, phase 1 of the AXIS project trained over 1, 200 people from 28 African Countries over the 2 year period • AXIS Best practice workshops covered 28 African countries and trained over 700 people in English, French and Portuguese • AXIS Technical Aspects workshops covered 28 African countries and trained over 500 engineers in interconnection technologies in English, French and Portuguese • Phase 2 of AXIS conducted 5 workshops across the 5 geographic regions of Africa (Central, East, North, South and West Africa) • Engaged over 350 experts from relevant public and private sector to discuss regional interconnection issues in in English and French

AXIS Outcomes • Phase 1 Outcomes • 8 new IXPs have been established as a result of the AXIS project • Most successful IXP project in the region and raised importance of IXPs across the board • Established new partnerships to enable deliver workshops in different languages (French & Portuguese) • In the process, we also developed regional francophone IXP expertise with support of partners • Phase 2 Outcomes • At least 2 IXPs per region will receive financial assistance to enable them evolve and grow to become regional IXPs • Project will provide technical assistance to ISPs to enable them evolve and become regional carriers • Project established and supported work of 5 regional task-forces to review policy and regulatory barriers that inhibit national and crossborder interconnection in their respective regions.

• 33 IXPs in 25 Countries or 17% increase since 2008 • Total Traffic exchanged at African IXPs >160 Gbps • More IXPs in West & Central Africa • Data shows more cross-border interconnection/peering in East and Southern Africa

Cross-border: Percentage of ASNs (origin ASNs) by country assignment in routes collected IXP: CAIX (EG) All ASNs Origin ASNs Source: Roderick Fanou Presentation at Af. PIF-2015

Cross-Border: Percentage of ASNs (origin ASNs) by country assignment in routes collected IXP: JINX (ZA) All ASNs Origin ASNs Source: Roderick Fanou Presentation at Af. PIF-2015

Cross-Border: Percentage of ASNs (origin ASNs) by country assignment in routes collected IXP: KIXP (KE) All ASNs Origin ASNs

Cross-border: Percentage of ASNs (origin ASNs) by country assignment in routes collected IXP: MIX (MZ) All ASNs Origin ASNs Source: Roderick Fanou Presentation at Af. PIF-2015

Peering and Interconnection Terminologies

Peering and Interconnection Terminologies q Peer • A network with whom you exchange traffic. q Peering • The act of exchanging traffic with a peer. Often your network and customers traffic. q Peering Coordinator • An individual within an organization that handles all peering-related matters for that network. q Private Peering a. k. a Private Network Interconnect (PNI) • Peering that does not typically involve any public exchange points, i. e. , back-to-back agreements q Public Peering • Peering typically done across a public exchange point

Peering and Interconnection Terminologies q Transit • A service where a network pays another for access to the global Internet. q Settlement-based Peering • One of the networks pays the other for the exchange of traffic q Transit-free • A situation where a network does not purchase any Transit from any other network, and yet “usually” has a full view of the global Internet. q Paid Peering • Similar to Transit where one network pays another for access to its backbone, but here, the network being paid provides connectivity only to its customers, and not the whole Internet. q Settlement-free Peering aka SFI • Neither party pays the other for the exchange of traffic.

Peering and Interconnection Terminologies q Mandatory Peering • A situation where members at an IXP are “forced” to peer with one another q Multilateral Peering • A situation where members at an IXP are required but not obliged to peer with all other networks present at the IXP. Its implementation is simplified with the presence of a Route-Server q Bilateral Peering • A situation where peering at an IXP is setup “directly” between two consenting networks q Hybrid Peering • A situation where an IXP supports both Multilateral and Bilateral peering for its members.

Peering and Interconnection Terminologies q Route-Server • A centralized router at a public peering exchange point that is able to serve all member routes via a multi- lateral peering strategy. q Looking glass: • A device/resource that permits anybody that is interested to analyze a network’s view of the Internet q Route registry: • A centralized database that contains routing information, e. g. , prefixes, AS_PATH’s, ASN’s, e. t. c.

Peering and Interconnection Terminologies q Peering Policy • A set of guidelines by which network operators will peer with external networks. q Open Peering Policy • A network implementing this general policy is happy to peer with any other network without restriction (excluding its customers) q Selective Peering Policy • A network implementing this general policy is normally happy to peer provided a minimum set of criteria are met q Restrictive Peering Policy • A network implementing this general policy is normally not interested in peering with any other networks especially in their service region. They may peer outside their service region but this is often rare q No Peering Policy • A network implementing this general policy means that there is no intention for the entity to ever peer.

Peering and Interconnection Terminologies q DFZ – Default Free Zone • A situation where networks run their routers with the full Internet BGP routing table and no default route. q Downstreams • Typically a network’s customers q Upstreams • Typically networks to whom you hand-off traffic and pay a fee, e. g. , Transit providers q Eyeballs • End-users of a network that are typically requesting content off online resources. q De-peer • A situation where a network terminates a peering relationship with another

Peering and Interconnection Terminologies q Traffic Ratio • The balance between how much traffic a network sends to its peers vs. what it receives from them. q Hot Potato Routing • The ability of a network to hand traffic off to other networks at earliest possible moment q Cold Potato Routing • A situation where a network retains traffic on its network for as long as possible q Off-net traffic • Traffic that is handed off to another network at some point in its flight. q On-net traffic • Traffic under the control of the same network, i. e. , the origination and termination of traffic remains on the same network.

Peering and Interconnection Terminologies q Po. I (Point of Interconnect): • A location, mutually agreed on by peering parties, where peering will occur. q Facility: • May be synonymous with a data centre or co-lo site where networks house their infrastructure. q Data Centre: • A purpose-built facility that provides space, power, cooling and network facilities to customers. q MMR (Meet Me Room): • A centralized passive, cable switching panel in a data centre where interconnects between networks occur. q Demarc (Demarcation): • Typically information about a co-lo customer, e. g. , rack number, patch panel and port numbers, e. t. c. q Interconnect charges (cross-connect): • Monies paid by peering parties to a data center for them to interconnect, e. g. , cost of cabling.

Peering and Interconnection Terminologies q Lit Fibre: • Fibre pairs owned by network operator who has attached equipment at either end to generate bandwidth from them q Dark Fibre: • Fibre pairs offered by the owner, without any equipment at each end of it to “activate” it q IPLC: • International Private Leased Circuit – a leased line that spans two or more countries. q IRU: • Indefeasible Rights of Use – Advance purchase of dark fiber (Terrestrial or Submarine cable) for a defined period (often long term 15 – 20 yrs ). IRU are subject to monthly payments of the shared operational and maintenance costs of the cable. q Dark Fibre Lease: • This provides option to access dark fibre for a defined (often short term <5 yrs) lease period. Thereafter access to the fibre is subject to renewal of the lease.

Interconnection Ecosystem: The Players & Actors

Overview of Interconnection Ecosystem Players & Actors • IP Transport providers: They own or resell access to physical infrastructure (terrestrial or submarine fiber cables). Services provided are such as Dark Fiber, IPLC, IRU & Lease. • IP Transit/Carriers: Provides layer 3 IP services (access to the global Internet) to the smaller ISP networks/Operators. Some IP transit providers own terrestrial or submarine fiber cable infrastructure or provide IP transit/carrier service as an alternative service • Carrier Neutral Data Center Operators: Also known as carrier hotels. Provide carrier neutral facilities to safely and reliably host computing, data storage and networking equipment. Facilities have built in redundancies on power, connectivity and cooling. They charge based on rack space taken and power consumed per month. They also charge for cross-connects amongst other services.

Overview of Interconnection Ecosystem Players & Actors • Content Distribution Networks (CDNs): Carry and deliver content to end-users on behalf of the content producer/owner. • Internet Access Providers: Also referred to as an “ISP” provides last mile infrastructure (owned or leased) to endusers for connection to the Internet. • IXP Operators: Provide neutral facilities for ISPs, CDNs, and Carriers to interconnect their respective networks for traffic exchange. IXPs are ideally collocated at carrier neutral data center facilities •

Interconnection Economics: Peering Relationships

The Virtuous Cycle

Relationships between IP Transport Provider and; i. IP Transport Provider • • • In some instances buy, lease or swap transport to extend their network beyond their infrastructure boundaries Often compete with other IP transport providers operating in their service region In some instances they partner to over restoration on different fiber paths ii. Transit/Carrier • Lease or buy capacity for national and international long-haul, backhaul from IP transport providers iii. Carrier Neutral Data Center Operator • • • Carrier Neutral DC’s will facilitate open access terms to IP transport providers The DC provides a “Meet-Me room” for IP transport providers to interconnect with its customers. The interconnections made at the MMR between IP transport provider and customer are referred to “cross-connects” and are subject to an installation and monthly recurrent fee.

… (Cont’d) iv. Content Distribution Network • CDNs lease dark fibre from IP Transport providers to deliver content at IXP locations v. Access Provider • IP Transport provider capacity to extend the Access providers network to the IXP and its POPs vi. IXP • • • In a distributed exchange (more than one location in a city), transport providers link the different IXP switch locations together IXPs partner/collaborate with transport providers to extend the reach of their IXP beyond their geographical location using remote peering services. Some Transport providers don’t actually own any infrastructure. Instead they lease from many transport providers and offer a managed IP transport service (one invoice as opposed to having to deal with many transport providers across many countries/cities/etc)

Relationships between IP Transit/Carrier and; i. Other IP Transit/Carriers • • • ii. • • They will often peer bilaterally with other IP carriers at the closest peering point and use hot-potato routing They will often exchange their downstream routes only They are very conscious of traffic ratios between IP carriers They tend to play lots of peering games based on traffic ratios Settlement free Transit to peer networks can be made in return for similar access Data Centers Will be present in all carrier neutral DC’s where they have customers or potential customers. They pay the DC cross-connect fees to connect to their peers demarcs or IXP iii. Content Distribution Networks (CDNs) • • • They peer with CDNs at any viable location local, regional or International. Due to large traffic volume they often peer over PNIs than public peering To keep traffic On-net, they setup as many CDN caches as possible on their network

… (Cont’d) iv. Access Providers (ISPs) • • They provide regional and international transit to ISPs Traditionally, they will not peer with an ISP that is their customers at an IXP Some Carriers will only peer with non customer ISPs who meet their traffic ratios requirements. Those non customer ISPs who do not meet their traffic ratio requirements can be offered paid peering or buy transit v. Internet Exchange Points (IXPs) • • • Carriers will be present at as many IXPs they can. Carriers prefer bilateral Peering over multilateral peering to avoid having to peer with customers Carriers are an IXPs’ largest customer by traffic volume and thus revenue

Relationships between Data Centres and; i. Other Data Centres • ii. • • DC’s are natural competitors amongst each other. Content Distribution Networks (CDNs) Data Centers will often go after CDNs due to their progressive uptake of collocation space. CDNs prefer carrier neutral facilities with low cross-connect charges and where an IXP is within close reach. iii. Access Providers (ISPs) • • Some ISPs prefer to build their network and collocate their infrastructure in DC's Data Centre offer ISPs access to enterprise networks, transport providers CDNs and IXPs collocated at the facility. iv. Internet Exchange Points (IXPs) • • • DC’s gain value by having an IXP collocated at their facility DC’s will either attract an existing IXP into its facility or operator an independent IXP Since the benefit is mutual most data centers will offer “special” collocation terms and rates to IXPs

Relationships between CDNs and; i. Other CDNs • • While they are competitive from a business perspective, they are mostly collaborative from a peering point of view. They tend to learn from each other when setting up peering especially in new/emerging markets ii. Internet Exchange Points (IXPs) • • They prefer to have both bilateral and multilateral peering over Route-Server They bring significant value to an IXP’s ability to attract ISPs and IP transit/Carriers. They are among the largest traffic volume peers They influence the IXPs decisions on switch port capacity and upgrade paths

Relationships between IXPs and; q § § q Other IXPs do not necessary compete with other IXPs outside of their service region IXPs in the same service region/city may compete but it is often a healthy competition In France, IXPs have made an effort to interconnect with well defined rules Cities in South Africa (Johannesburg, Durban, Cape Town) and UK (London) have more than one independently operated IXPs that are not interconnected Case Examples § France-IX with Lyon. IX (Lyon), LU-CIX (Luxemburg), and Top-IX (Turin), Tou. IX (Toulouse) § IXPs expansion of peering services beyond service city i. e France-IX in Marseille § JINX and NAP-Africa in Johannesburg, Durban and Cape Town, or LINX and LONAP in London

Peering relationships summary q All players have mutual dependencies in the peering ecosystem q Most peering relationships have differing terms, largely due to relational economics q Competing IP Transport providers (national and cross-border) are key to lower costs and higher peering/transit capacity in a region q The existence of competing Transit/Carriers help advance peering and interconnection strategies q Carrier Neutral Data Centers are vital to the peering ecosystem and its growth q IXPs should develop partnerships/collaboration with IP Transport providers for remote peering services to IP Carriers, CDNs and ISPs q IXPs should leverage on collocating at Carrier Neutral Data Centers due to potential benefits

END