CENIC PACIFIC WAVE NLR IEEAF Infrastructure for Research

CENIC, PACIFIC WAVE, NLR, IEEAF: Infrastructure for Research and Education John Silvester Chair of the CENIC Board Vice-Provost for Scholarly Technology, University of Southern California LISHEP: Digital Divide and HEPGRID Workshop Rio de Janeiro, Brasil February 19 th, 2004

1. CENIC History 2. Cal. REN and DCP 3. Cal. REN ONI 4. Cal. REN Today 5. National Lambda. Rail (aka Lightrail) 6. Pacific Wave 7. IEEAF 8. Pointers

Education in California – Overview n University of California – 9 (10) campuses n 3 Private Research Universities – Caltech, Stanford, University of Southern California State University – 23 campuses n Community Colleges – over 100 n Other independent institutions of higher education – over 100 n K-12 schools – over 9000 n Various government labs and university affiliated research institutes

State of Networking in 1996 n 4 -CNET connected the CSU system with extension out to community colleges n Most institutions had their own commodity internet connections n UC operated some private leased lines n No statewide K-12 network n Some individual county and school district networks

Impetus to Establish R&E Network in California n Expansion of the NSF Connections program to facilitate access to the super-computer centers n Broadened to allow access to the high performance backbone (v. BNS) for meritorious research n Acted as a stimulus to bring the Research Universities to create a joint proposal n Two initial goals: – Reduce costs of access to high performance national networks by a collaborative approach – Facilitate communication and collaboration between institutions by building a California REesearch Network

How to Proceed? n n Decision that the network should be funded by the institutions rather than looking for a handout from the state or relying on Federal Government funding Startup funds came from a joint proposal on behalf of the R-1’s to the NSF connections program (12 campuses). This allowed the institutions to ease into the burden of covering the network costs over a 3 year period Founded a non-profit public benefit entity – CENIC – creating some separation from publicly funded entities (UC, CSU) Lightweight organization (1. 5 FTE for first 3 years)

1. CENIC History 2. Cal. REN and DCP 3. Cal. REN ONI 4. Cal. REN Today 5. National Lambda. Rail (aka Lightrail) 6. Pacific Wave 7. IEEAF 8. Pointers

Phase 1 - 1998

CENIC Initial 4 -Year Funding Charter Associates CENIC Partners Corporate Discounts Federal Funding $16 M 42% $2 M 5% $14 M 37% $6 M 16%

Developments n Direct funding from UC for a central valley link to link Cal. REN-north (SF) and Cal. REN-south (LA-SD) n ISP – took advantage of the aggregated buying power to leverage ISP contracts with multiple vendors – significant savings (additional savings from Quilt pricing) n Peering – took advantage of peering opportunities at PAIX and MAE-LA n DCP – Digital California Project – extend connectivity to (public) K-12 schools

Digital California Project n n Funded from California State in FY 2000/01 $32 M through University of California Implementation through CENIC n Program Steering Committee - Advisory board of involved constituents from K-20 n Subsequent budget reductions -- $26 M -$21 M - $14 M – eliminated as direct funding for 2004 -05

1. CENIC History 2. Cal. REN and DCP 3. Cal. REN ONI 4. Cal. REN Today 5. National Lambda. Rail (aka Lightrail) 6. Pacific Wave 7. IEEAF 8. Pointers

California R&E Networks in 2000 n CALREN-2 – advanced services network, owned and operated by CENIC (Corporation for Educational Network Initiatives in California) n 4 CNet – owned and operated by Cal. State University n DCP – K-12 network- owned and operated by CENIC; being implemented n Los Nettos – dark fiber based CWDM Metro network owned and operated by USC and Caltech - last of the NSFnet regionals.

Phase 2 - 2000

Redesigning Cal. REN n n In late 1999, with the approaching end of current SONET contracts (late 2002), CENIC began thinking about the next generation Cal. REN User demand: – – n Reliable ‘commodity’ network High bandwidth (IP) network in support of research (Abilene) Some demand for dedicated resources Significant demand for experimental and research networks at level 3, level 2 and even level 1 This formed the thinking for an integrated infrastructure built on dark fiber

CALREN-DC Digital California n IP based network. 2. 5 -10 GB n Serves-140 H. E institutions; 8000+ elementary and high schools n 8. 0 million+ student, faculty and staff users n I 2 connectivity and commodity ISP services.

CALREN-HPR High Performance Research Network n IP network: 10 Gb, potentially several wavelengths n 50+ Research institutions, National Laboratories and San Diego Super-computing Center in California component of Internet 2 with 10 G and OC 12 connections n Serves hundreds of researchers, demanding applications

CALREN-XD Experimental/Development Network n 10. 0 Gb Wavelengths and Dark Fiber n Potential for Wavelength Switching and Special Network Configurations n Special applications, e. g. Teragrid n Serves Network Researchers in California Research Institutions – primarily four UC Institutes; USC’s ISI; Stanford; and Caltech

1. CENIC History 2. Cal. REN and DCP 3. Cal. REN ONI 4. Cal. REN Today 5. National Lambda. Rail (aka Lightrail) 6. Pacific Wave 7. IEEAF 8. Pointers

CALREN - today n 3 backbones – one commodity, one production, one research oriented, sharing physical resources where applicable n Integrated at the physical and operations level, separable at the link and network levels n Separate local solution from long-haul solution (due to different possibilities, players) n Combination of dark fiber and wavelengths

Cal. REN Backbone Network Phase 3 - 2003

Cal. REN Waves UC Davis Sacramento Cal. REN DC HPR Teragrid HPR & DC Triangle Court Oakland Fergus Sunnyvale Soledad Fresno San Luis Obispo Bakersfield Los Angeles Santa Barbara Tustin San Diego

Digital California Overlay

Calren/DC/HPR/XD POP Architecture Cal. Ren DC HPR XD Long Haul OC 48/OC 192/10 Gig. E DWDM 10 Gig E or OC 192 Gig E switch/mux Cal. REN/DC HPR 15500 Campus or Metro Interconnect XD

Historical Review of Cal. REN 1996 - Initial meetings 1997 - NSF proposal for Cal. REN 2 funded; CENIC incorporated 1998 - Cal. REN 2 operational, connect to v. BNS 1999 - Connect to Abilene; ISP service launched 2000 - DCP project launched 2001 - ONI project launched; First DCP nodes operational 2002 - ONI vendors selected; DCP 90% completed 2003 - Cal. REN-DCP (optical edition) operational 2004 - Cal. REN-HPR operational 2004 - Pacific Wave linkup 2004 - XD links operational as NLR comes up

1. CENIC History 2. Cal. REN and DCP 3. Cal. REN ONI 4. Cal. REN Today 5. National Lambda. Rail (aka Lightrail) 6. Pacific Wave 7. IEEAF 8. Pointers

Expanding the reach? n Multi-State interest in a larger geographic fiber based infrastructure n Fiber vendor very interested in selling National footprint

National Light Rail - concept n Dark Fiber National footprint n Serves very high-end experimental and research applications including network research n 4 - 10 GB Wavelengths initially n Capable of 40 10 Gb wavelengths at build-out n Partnership model (including corporate partners)

Many different perspectives n “NLR… … aims to reenergize innovative R&D into next generation networking technologies, protocols, services, and apps. ” … is a virtual laboratory. ” … will contribute to the (NSF) Cyberinfrastructure that is critical to progress in every field of science & engineering. ” … will help fuel the growth of the Tera. Grid and computational science in general. ” … motivates our planned regional optical network. ” … is a hedge timed to hit the trough of the Internet/telecom economy. ” … is both an experimental facility and a complex, multi-dimensional experiment in and of itself. ” ALL OF THE ABOVE!

Distinguishing features - I n n n NLR, Inc. has been established as a non-profit organization (May 2003, 501(c)(3) tax-exempt status being sought) Partnership with industry (CISCO is a major partner/contributor with special interest in promoting network research) Largest higher-ed owned/managed optical networking & research facility in the world – ~10, 000 route-miles of dark fiber (Level 3) – Four 10 -Gbps ’s provisioned at outset n First & foremost, an experimental platform for research – – Optical, switching & network layers Research committee (with 2 board seats) Experimental support center 50% of capacity available for network research

Distinguishing features - II n n Sparse backbone’ topology Use of high speed Ethernet for WAN transport – 1 O Gigabit Ethernet LAN PHY is primary interface – Traditional OC-192 SONET available, too – Additional ’s (up to 40 on a given segment) can be provisioned with pricing tied to incremental cost n Unprecedented level of self-capitalization for national networking initiative by higher ed participants – ~$80 M budget for full national backbone (Cap. Ex + 5 -yrs Op. Ex) – Each contribution (typically $5 M) assumed to be ‘sunk cost’ n Each participant/node has concurrent responsibility for developing optical networking capabilities and sustaining performance in their region

Current Members and Associates n CENIC n Florida Lambda. Rail n Pacific Northwest Gigapop n Georgia Tech n CIC n Pending: n Pittsburgh SC n Duke Univ. /NCLR – Texas Consortium n MATP/Va. Tech n Cisco Systems – New York Area Consortium n Internet 2 – Mountain region consortium

NLR Project Segment Deployment Schedule (approx) n Chicago to Pittsburgh 11/03 n Sunnyvale to Seattle 4/04 n Pittsburgh to Washington DC n Washington DC to Atlanta 5/04 n Atlanta to Jacksonville n Seattle to Denver 7/04 n Denver to Chicago 8/04 3/04

Related Developments n O(10 k) miles of dark fiber have been acquired by the community for national and regional optical networks by CENIC, Fiber. Co (via Level 3), and others. n ATT makes major donation of fiber to USA Waves n The Abilene Network has been upgraded to a 10 -Gbps backbone and supports the research university community through initiatives such as IPv 6 deployment and the Observatory. n Internet 2 plans to use one NLR wave for development of HOPI (hybrid optical packet interconnect)

1. CENIC History 2. Cal. REN and DCP 3. Cal. REN ONI 4. Cal. REN Today 5. National Lambda. Rail (aka Lightrail) 6. Pacific Wave 7. IEEAF 8. Pointers

U. S. Pacific Coast Peering Collaboration n CENIC (Corporation for Education Network Initiatives in California) and PNWGP (Pacific Northwest Gigapop) have combined efforts to create an advanced, extended peering facility on the U. S. West Coast. n Concept: an extensible, geographically dispersed peering fabric n Result: you connect at any one location on the fabric and have the option to peer with any other participant, regardless of where they are connected

Current Model

Summer 2004 - LA & Seattle

Multi-Node Pacific Wave n The Pacific Wave International Peering exchange facility will offer connection points initially in Los Angeles and Seattle, proximal to submarine cable landing sites on the U. S. Pacific Coast. n Connection points to be connected by 10 GE link derived from NLR n Expected operation by Summer 2004

Future?

1. CENIC History 2. Cal. REN and DCP 3. Cal. REN ONI 4. Cal. REN Today 5. National Lambda. Rail (aka Lightrail) 6. Pacific Wave 7. IEEAF 8. Pointers

New Public-Private Partnerships n Global telecomm build-out of technical infrastructure provides new possibilities for economic development n Current market conditions have resulted in capacity which is currently going unused -- cannot be sold. n As a matter of social responsibility, this unused capacity could be made available for stimulating future applications and markets -- by donation for use by research and education institutions.

Example: IEEAF n The IEEAF represents one such partnership whose goal is to obtain donations of international bandwidth to enable a global collaboration in research and education. n Current donations have already linked US and Europe, and are linking US and Asia-Pacific. n This bandwidth helps enable global collaborations in research and education, in the true spirit of the Global Quilt Initiative.

IEEAF Vision: The Global Quilt n A Network of Networks, “stitched together” to create a common single fabric, and shared equally by all. This will be achieved through collaboration and community effort, until it covers the globe. n The IEEAF has no boundaries of “home” territory…. .

IEEAF - What is it? n U. S. 501. c. 3 Not-for-profit corporation n Formed from original MOU between GEO and CENIC (Corporation for Educational Networking in California) n Vision: Accelerate the global growth of advanced networking (Internet 2) to achieve "universal educational access” to: – Enable and stimulate the rapid expansion of research and educational collaboration in many forms between teaching and learning institutions around the world. – Cultivate and promote practical solutions to delivering scalable, universally available and equitable access to suitable bandwidth and necessary network resources in support of these collaborations.

IEEAF - How does it work? n Leverage global deregulation and new entrants into telco business n Leverage private sector business relationships – Geographic Network Affiliates, Inc. (GEO) n Build donations into business deals (contracts) such as no-cost IRUs

IEEAF - What does it do? n Gets donated communications related assets n Makes them available to existing institutions and networking organizations to put to work n Vehicle: Asset Steward Agreement

Tyco Telecomm Donation Summary n Co-location space in NYC for Expanded International Exchange Point n Production R&E Bandwidth: 622 Mbps – NY-London-Groningen (Netherlands) » Connects to IEEAF fiber to Amsterdam and Hamburg – US-Tokyo, Seoul, Shanghai, Taipei, Hong Kong, and Singapore n Research 10 Gbps optical wavelength (preemptable) – NY-London-Groningen (Netherlands) – US-Tokyo, Seoul, Shanghai, Taipei, Hong Kong, and Singapore n 200 sq. ft. Co-location space in each of global facilities n Additional donations as global build-out continues

Tyco Global Network - Donations Connectivity Donations 622 Mbps +10 Gbps

Tyco Transpacific Donation Available last December, 622 Mbps Debut at Busan! 10 G available in March! Donated, Available when lit

TYCO Donation Key Elements n TYCO Committed Assets – Production R&E Bandwidth: 622 Mbps – Research 10 Gbps optical wavelength (preemptable) – 200 sq ft Co-location space in each of global facilities n Trans-Atlantic and trans-Pacific already operational n Others to be made available as business case (for TYCO) becomes feasible.

USA Waves n In U. S. , IEEAF partnership with SURA resulted in 8, 000 miles of trans-USA fiber donated by AT&T to the Southeastern Universities Research Association (SURA) through GEO and IEEAF efforts.

IEEAF 8, 000 Mile Trans. USA Donation with SURA – Potential Footprint

AT&T Donation – Key Elements n No-cost lease of 6, 000 miles of dark fiber pair on Nex. Gen network n No-cost lease of additional 2, 000 miles of Nex. Gen fiber (O&M costs waived) - optical research pilot testbed projects n Donation of existing Cisco equipment n Very low cost IRUs (lease) for additional dark fiber

IEEAF – other Activities n Many other donations in progress n Key is to look for partnership opportunities n GEO as a key player in IEEAF often has access to opportunities at early stages of development and has succeeded in leveraging opportunities.

Developing a Global Quilt n Global patchwork of regional initiatives linked with international initiatives resulting global connectivity for R&E worldwide n Much progress but work still to be done. . .

Think Globally Strategic – Act Locally Opportunistic

1. CENIC History 2. Cal. REN and DCP 3. Cal. REN ONI 4. Cal. REN Today 5. National Lambda. Rail (aka Lightrail) 6. Pacific Wave 7. IEEAF 8. Pointers

For More Information silvester@usc. edu CENIC: www. cenic. org NLR: www. nationallambdarail. org Pacific Wave: www. pacificwave. net IEEAF: www. ieeaf. org Acknowledgements Tom West: CENIC, NLR Steve Corbato: Internet 2, NLR Don Riley: IEEAF Ed Fantegrossi: GEO, IEEAF