New Directions in Peering for Tier2 and Content

New Directions in Peering for Tier-2 and Content Providers Jeb R. Linton Staff Network Engineer, Earth. Link jrlinton@ieee. org

Introduction and Acknowledgements n Who am I and what am I doing here? n Acknowledgements

Topics to be Covered n n n Peering Motivations and Economics The Cost of Peering - History Disruptions - What do they mean? q q q n n New Low Transit Costs New Tier-1 Peering Initiative Metropolitan Multipoint Private Peering FUD (Teaser: What Happens Next? )

Peering Motivations and Economics

The Role of Peering n To Large Tier-1 s q n Mission-Critical To the Rest of Us q q q Used to Decrease Cost of Transit Used to Increase Quality of Routes Usually not mission-critical * * Your Mileage May Vary

The Cost of Peering - History

Peering Cost – the Legacy NAPs n 2 -3 years ago, cost justification for NAP peering was easy based on the high cost of Transit See Bill Norton’s whitepaper: A Business Case for ISP Peering for the origin and explanation of this method of cost justification. Bill’s graphing method will be used throughout.

Peering Cost – the Legacy NAPs, cont’d Disruption #1: Rapid drop in Transit cost makes legacy NAPs much less desirable

Peering Cost – Private Lines as Supplement n Evolution: NAP Peering is supplemented with Private Peering.

Peering Cost –Private Lines for Many Peers n Problem: Private Peering to many Peers can be difficult because of the repeated expenditure and low circuit utilization efficiency

Peering Cost – Tier-1 Peering Initiative Disruption #2: Large Tier-1 Providers create the Next-Generation Peering Initiative n n n Common facilities are chosen, to which each major Tier-1 will acquire fiber rights Allows excellent scalability while minimizing repeated cost Independence, Fault Tolerance, and Provisioning time improve

Next-Generation Tier-1 Peering: Before n n n N*(N-1)/2 Links in each of 7 Cities Dependence on RBOCs Slow Provisioning Mostly OC-3 and OC-12 Poorly adapted to use of Dark Fiber Spr ATT UU L 3 Gen Q CW

Next-Generation Tier-1 Peering: After n n n One-time Dark Fiber provisioning to a single facility Each Tier-1 maintains routers in the same room Cheap, fast cross -connects for OC 48 or higher peering Spr ATT UU L 3 Nex. Gen Peering Center Q CW Gen

Peering Cost – Appearance of Metro Ethernet Disruption #3: Cheap Metro Ethernet services connect to multiple peers over the same port using Point-to-Point and Multipoint VLANs

Disruptions – What do they mean?

Disruption: Low-Cost Transit n Legacy Peering methods lose popularity – NAP attachments and Private Line Peering links are no longer saving money. n Some companies retain NAP presence just to meet contractual requirements, or for marketing reasons. n Increased pressure to find cheaper Peering methods.

Next-Gen Tier-1 Peering: Implications n Large Tier-1 s draw smaller Tier-1 s to new Centers for Peering n Largest Tier-2 and Content providers drawn to the Centers hoping to peer with Tier-1 s n Smaller Content, Tier-2 s, Enterprises and others are drawn to new Centers for competitive Transit and some Peering What happens to all the other collocation?

Price Disruption: Metro-Ethernet n Most major Collocation spaces in “NFL Cities” are already Metro-Ethernet enabled n Pressure to replace individual Private Lines (DS 3, OC-12) with Metro-Ether where possible. n Legacy NAP viability may depend on ability to accommodate Ethernet links q Many are ATM-based and charge heavily for collocation q Some have adapted or may yet adapt

Metro Multipoint Private Peering: Before (The situation for Smaller Tier-1, Tier-2, and Content) n n n Private Peering NAP Peering In-Building Peering if you’re lucky. Low efficiency High Cost Slow Provisioning Those Guys ATM OC 3 Old NAP ME Private DS 3 s YOU HIM

Metro Multipoint Private Peering: After (The situation for Smaller Tier-1, Tier-2, and Content) n n Cross-Connect to an Ethernet port in your current facility Multiple VLANs for Private and Public Peering, and Transit Link to Nex. Gen Peering Centers? Those Guys HIM ME Public Peering VLAN YOU Efficient, Cheap, and Fast Transit Provider ? Nex. Gen Peering Center

Metro Multipoint Private Peering n n Addresses some limitations of Private Peering q Multiple Peer links on the same physical port q Private, Public, and Transit share same ports Many configurations and payment options q Pay by port, by utilization, or by allocation q P-to-P VLANs per peer, or semi-public Multipoint VLANs q Bandwidth allocated per port or per VLAN q Technical options: Broadcasts, Switch Config, Failover, etc.

Metro Multipoint Private Peering n Probably won’t scale to tens of Gbps q May scale further once 10 Gig-E is available q Scaling is better addressed by Dark Fiber/Lambda connections to the Nex. Gen Peering Centers q Optimum scalability only reached if all local peers use the same Metro-Ether Provider (!)

FUD* * Fear, Uncertainty, and Doubt

FUD – The Tier-1 Peering Initiative n n Q. Is this Unfair? Non-Inclusive? Anti. Competitive? A. No… it’s just common sense, and anyone can join in. Q. Will this increase Transit prices? A. Competition should improve Transit pricing in the Peering Centers, and have no effect outside them.

FUD – The Tier-1 Peering Initiative n n Q. What about my old Co. Lo? A. Thanks to cheap metro, most large Collocation centers are still viable, but you may want to get a foot in the door at the new peering centers. Q. Are the Tier-1’s really committed to this? A. Yes… It’s the only viable long-term option, and they don’t want to waste the investment they’ve already made.

FUD - Metro-Multipoint Private Peering n n Q. Early Entry is scary… why not wait? A. Very low commitment is needed; can start below 10 M Q. Does the service have the robustness of Private Line? A. All vendors use diversity wherever available. Most use same or similar configuration to the standard SONET/Lambda services we all rely on

FUD – Metro-Multipoint Peering, cont’d n n Q. What about the financial stability of the vendors? Isn’t the cheap pricing based on gaining market share? We saw what happened in the last year… What about today’s vendors? (Applies to Metro-Ether and Peering-Collocation vendors) A. Reduce your risk with short contract terms…Multiple vendors means you can switch if need be.

FUD – Metro-Multipoint Peering, cont’d n n Q. What technical gotchas should we be aware of? A. Management overhead similar to old NAP, and Broadcast and Congestion issues need to be resolved Bottom line: Mitigate risk with short contract terms, and do your own financial analyses… Caveat Emptor!

Points of Contact: Earthlink n Jeb Linton (Architecture/Peering Strategy) – jeblinton@corp. earthlink. net n Josh Fleishman (Peering Coordinator) – flyman 2@corp. earthlink. net

Backup: What Happens Next?

Backup: What Happens Next 1. 2. 3. 4. 5. 6. 7. In “NFL Cities”, expect most large Internet companies (and many small ones) to migrate to the Nex. Gen Peering Centers. In all areas with Metro-Ethernet, expect off-net Collocation Centers to die off. Expect ATM-based NAPs to continue losing popularity. Metro-Ethernet will predominate for new P-to-P Private Peering links and links to the Nex. Gen Peering Centers. Metro-E will be used for Multipoint Peering. (Services already emerging from several vendors) One Metro-E vendor will begin to predominate for Peering in each Metro area. Peers will migrate from MM-PP to the Centers as they grow.

Backup: What Happens Next… the Big Picture Large Internet companies interested in significant Peering will migrate toward the Nex. Gen Peering Centers Mid-sized companies will use both: n n Nex. Gen Peering Centers, for Tier-1 and High-Speed peering Metro-Multipoint for smaller peers, to maximize savings q q Small companies will: n Start with a Metro Ether port for Transit, B-B, etc. Add peering VLANs on the same port for cost savings q q n n q Point-to-Point for B-B links and opportunistic private peering Metro-Multipoint for additional cost savings Use Metro Ethernet as a bridge to the Nex. Gen Peering Centers as they grow

Backup - Terminology (for convenience only) n Tier-1 – Providers of Internet Access to Autonomous Systems (a. k. a. “Leased Transit Service”, “Transit”) n Tier-2 – Providers of Internet Access to Individuals n Nex. Gen Peering Centers – Collocation Centers chosen as common locations for Peering by the Largest Tier-1 Providers

Backup - Who wants to peer with whom? n Largest Tier-1 q n Other Tier-1 q n Content, all Tier-1, and each other Content q n Largest Tier-1, each other, large Tier-2, and some Content Large Tier-2 q n Each Other Everybody but each other Other (Who is Other? ) q Academe, Enterprises, R&D, etc… the Usual Suspects

Backup: The “Peering Promiscuity Chart” Arrows point toward Targeted Peers Other Similar Largest Tier-1 Other Similar Other Tier-1 Largest Tier-2 Content-Heavy Traffic Mostly Outbound Eyeball-Heavy Traffic Mostly Inbound Other Tier-2 Content Other Similar Other

Backup – Metro Ethernet

Backup – Point-to-Point Metro Ethernet

Backup – Multipoint Metro Ethernet

Backup - Evaluating Available Methods of Peering n Questions to Ask: q q q n How much Peering traffic is locally available using this peering method? How much does this Peering Method cost per Mb for the expected traffic level? What does the company pay per Mb for Transit in this metro area? Things to consider: q q Should be half the cost of Transit (per Mb) or better Methods of Peering are complimentary, not exclusive