National Science Foundation International Research Network Connections Program

National Science Foundation International Research Network Connections Program Kickoff March 11, 2005 Julio Ibarra, PI Heidi Alvarez, Co-PI Chip Cox, Co-PI John Silvester, Co-PI

Outline l l l l The WHREN-LILA project US - Latin America Connectivity Links Interconnecting Latin America (LILA) year 1 The Atlantic. Wave Project Monitoring and Measurement Activities WHREN-LILA Year 1 Milestones E-Science and Engineering Collaborations 2�

The WHREN-LILA Project l l l Proposal submitted by Florida International University (FIU) and the Corporation for Education Network Initiatives in California (CENIC) - Award# 0441095 Links Interconnecting Latin America (LILA) aims to Improve connectivity in the Americas through the establishment of new interregional links Western-Hemisphere Research and Education Networks (WHREN) is a coordinating body of organizations from across North and South America that aims to leverage the network resources of participating members to foster collaborative research and advance education throughout the Western Hemisphere � 3

Project Goals l l l Improve network connectivity between North and South America through the deployment, operation and evolution of LILA links Evolve the LILA links to their fullest capacities as resources and economies permit Foster collaborative research and advance education throughout the Western Hemisphere and other world regions Support the evolving needs of US science and engineering researchers Foster new inter-regional and inter-disciplinary communities of researchers and learners � 4

WHREN - Coordination in the Western Hemisphere l l WHREN will establish a consortium of participating western-hemisphere organizations that will collectively oversee the assignment of lightpaths across administrative domains A Governance Committee (GC) will collectively oversee the assignment and management of lightpaths and provide coordination between member organizations A Research Advisory Committee will be formed to advise the GC on program and network needs for the broad research and education community An Engineering Committee (EC) will be comprised of engineering managers from the various networks participating in WHREN � 5

LILA Project Coordination l l l Participating Organizations q FIU (Awardee) q CUDI (Mexico) q CENIC (Awardee) q RNP (Brazil) q ANSP (Sao Paulo) q REUNA (Chile) q CLARA (Latin America) Project Steering Committee formed, with one member from each participating organization, for project implementation and operational decisions Engineering Committee, comprised of network engineers from each participation organization, to make network engineering and operational recommendations to the Steering Committee � 6

US - Latin America Connectivity before IRNC Argentina, Brazil (national and the State of Sao Paulo), Chile, Panama and Venezuela connections through Miami l Mexico connections through San Diego and El Paso l Peerings with Internet 2 and other US R&E networks through AMPATH, Cal. REN and UTEP l International and Fed. Net peerings at STARTAP/Starlight from Miami provided by AMPATH l � 7

Regional Development l l l Cooperation of Latin American research networks (CLARA) @LIS Alliance of the Internet Society funded program, providing 10 Million Euros for interconnecting R&D communities of Latin America and Europe Creates a regional backbone in Latin America Direct connectivity to Europe from Sao Paulo, Brazil Intraregional connectivity between connected countries in Latin America 3 DS 3 s from AMPATH to support CLARA initiative Argentina (RETINA) Ecuador (CEDIA) El Salvador Brazil (RNP) (RAICES) Chile (REUNA) Costa Rica (CRNET)Guatemala (RAGIE) Mexico (CUDI) Nicaragua (RENIE) Panama (REDCYT) Paraguay (ARANDU) Peru (RAAP) Uruguay (RAU) Venezuela (REACCIUN) (NRENs in formation indicated in RED) � 8

Expected CLARA network topology Network Characteristics: • 155 Mbps backbone ring • 622 Mbps connection to Europe • local traffic remains within the region • 10 to 45 Mbps spur links • 4 Mbps satellite link to Cuba • Network to be operated by CLARA (through CUDI and RNP) � 9

Red. CLARA Routed Network � 10

Current US - Latin America Topology Red. CLARA network starts operating in August 2004 l Brazil/RNP and Chile/REUNA today transit CLARA, then GEANT to reach US R&E networks l NSF CHEPREO project and collaboration with Sao Paulo R&E community establishes STM-4 link between US and Brazil l ITN services and transit to Fed. Nets through Abilene l Argentina, Panama and Venezuela maintaining direct connections to US through AMPATH l Mexico has direct connection to US through UTEP l 11�

Links Interconnecting Latin America (LILA) Year 1 Increases Miami - Sao Paulo link from 622 Mbps to 1. 2 Gbps l q Q 2 2005 q Evolving to 2. 5 Gbps Establishes a dark fiber segment between San Diego and Tijuana for a 1 Gbps link l q May 2005 Enables interregional peerings through east and west coasts l � 12

LILA Miami - Sao Paulo link design Abilene Provides dedicated Gig-E interface to ANSP l. Provides shared Gig-E interface to CLARA and RNP l. Support planned for lightpath provisioning and deterministic transport services through Atlantic. Wave l. Peering with Internet 2’s Abilene and other R&E networks through AMPATH l NAP of the Americas Miami Atlantic. Wave AMPATH 1/10 Gig. E Switch (Foundry Fast. Iron 400) Gig. E AMPATH Router (Cisco GSR 12012) Gig. E LILA-Miami Optical Mux (Cisco ONS 15454) OC 192 2 x VC 4 -4 C (2 * 622 Mbps = 1. 2 Gbps) LILA-Brasil Optical Mux (Cisco ONS 15454) Gig. E ANSP router COTIA Sao Paulo Gig. E Ethernet switch Gig. E POP Red. CLARA POP RNP USP 13�

LILA San Diego - Tijuana Link l Provides dedicated Gig-E interface to CLARA l Provides dedicated Gig-E interface to CUDI l l l Growth across border possible through purchase of additional Gig-Es up to maximum of 6 Connections are to CENIC’s Cal. REN/HPR routed network Peering through Cal. REN to Internet 2 and other R&E networks � 14

LILA San Diego - Tijuana link design Physical Diagram CENIC at UCSD/SDSC Logical Diagram Cal. REN/HPR switch Cal. REN/HPR GX POP @ 3180 University switch CLARA router CUDI router Cisco 7206 VXR CLARA @ BESTEL in Tijuana CUDI @ Telnor in Tijuana CUDI Provided Equipment/fiber CLARA Provided WHREN-LILA Provided � 15

US - Latin America Year 1 Topology LILA links reestablish direct connectivity to South America from east and west coasts l Reduces delay reaching sites in Chile and Brazil from the US and Asia-Pacific l Introduces an infrastructure to develop a distributed international exchange and peering l Leverages network resources to provide route diversity and high-availability production services l 16�

Atlantic. Wave l Atlantic. Wave is an International Peering Fabric q US, Canada, Europe, South America q Distributed IP peering points: Ø NYC, WDC, ATL, MIA, SPB l l l SURA, FIU-AMPATH-CHEPREO, the IEEAF, MAX, So. X/SLR, MANLAN, and in partnership with the Academic Network of Sao Paulo (ANSP) are combining efforts to establish Atlantic. Wave Described as an integral component of the WHREN-LILA proposal to extend LILA on the Atlantic side to MANLAN in NYC Complements the Pacific. Wave distributed peering facility on the west coast � 17

Atlantic. Wave Topology A-Wave provides multilayer/multi-protocol services between participating networks l q Layer 3 peering services q q over ethernet GLIF “light path” services Others TBD A-Wave will to provide a Layer 3 distributed exchange capability l q Ethernet based q Best effort packet exchange q Linear topology – unprotected (NLR based) q 1 GE, 10 GE LAN, 10 GE WAN client access q Jumbo frame support 18�

Atlantic. Wave Design Initial East Coast US A-Wave Backbone Future backbone Extensions within NA & EU To South America To Europe & Canada, MIA ATL Miami, FL Atlanta, GA WDC Washington, DC NYC New York City, NY Sonet services Prepared by Jerry Sobieski Ethernet services 19�

Generic A-Wave Node Architecture (using separate switching fabrics) NLR Node A-Wave Backbone OC 192 c Sonet switch Ethernet WAN PHY Regional ethernet handoff: 10 Gbs LAN or 1 Gbs Prepared by Jerry Sobieski Regional sonet handoff: OC 192 c or OC 48 c Payload agnostic Ethernet switch � 20

Deployment Plans & Timeline l Phase 1: Deploy backbone OC 192 c Sept 05 q Between MIA-ATL, ATL-WDC, WDC-NYC q 10 Gbs WAN PHY ethernet over NLR wave initially. q Migration of existing exchange switches/networks Ø Regional backhaul Ø Reconfiguration of existing exchange services and networks l Phase 2: Sonet switch deployment Dec 05 q Map IP/Ethernet Peering Fabric across “appropriate” sized VCG (GFP- F & VCAT) q Engineer and deploy GLIF Common Services in conjunction with other GLIF domains l l Phase 3: Deploy dynamic light path services Phase 4: Expansion Mar 06 Aug 06 -> q Integrate links between A-Wave, P-Wave, Northern Tier, etc 21�

Monitoring and Measurement Activities l l l NSF REU Distributed Network Monitoring using Net. Flow and Mon. ALISA project An integrated networking monitoring system using Cisco Netflow, Mon. ALISA and NLANR PMA/AMP tools Ping. ER and NLANR AMP active measurement resources in service for links to Latin America 22�

Distributed Network monitoring using Netflow and Mon. ALISA l l l l Development project that integrates Net. Flow monitoring data with the Mon. ALISA framework Collaboration between FIU, Caltech and CERN Builds a lightweight, scalable and customizable utility that enables viewing flows based on source-destination pairs Allows for a historical trend analysis of network behavior as well as a close-torealtime look at flow data Builds a framework for an Integrated Passive Flow Measurement Toolkit (next slide) Support from NSF REU Award SCI-0231844 REU Report available at http: //www. ampath. net/publications. htm 23�

Integrated Passive Flow Measurement Toolkit l l l A real-world deployable network monitoring toolkit that improves the functionality of Mon. ALISA by integrating Net. Flow and NLANR PMA Cisco University Research Program (URP) Award to Caltech and FIU Leverages NSF REU project that integrates Net. Flow into Mon. ALISA Integrates Mon. ALISA, Cisco Net. Flow and NLANR PMA tools to facilitate efficient network planning and operation for the at large networking community Being deployed to monitor traffic flows for CHEPREO between the US and Brazil Toolkit will be available by Sept. 2005 24�

Measurement and Monitoring Plan l l l Active measurement q SLAC Ping. ER (deployed and available, a Telnet server and a Ping. ER collector for delay/loss information from/to AMPATH) Passive measurement q NLANR PMA (now taking a 90 -second trace every hour, tunable upon request) q Net. Flow (available via the Mon. ALISA service module and SNMP/MRTG graphs) Implementation and Deployment q Service discovery and service integration based on Mon. ALISA q Modular design to allow flexible and robust deployment and extensibility 25�

Year 1 Milestones l l l Implement LILA links Implement interregional peering through Cal. REN and AMPATH Establish Coordination and Control mechanisms for service management Deploy Atlantic. Wave OC 192 c backbone Deploy Next-Gen SONET switches May 05 June 05 Sept. 05 Dec. 05 26�

E-Science and Engineering Collaborations 27�

Global LHC Data Grid Hierarchy ~10 s of Petabytes/yr by 2007 -8 CMS Experiment ~1000 Petabytes in < 10 yrs? Online System Tier 0 Tier 1 0. 1 - 1. 5 GBytes/s 10 -40 Gb/s USA CERN Computer Center Mexico Brazil Argentina >10 Gb/s UF Tier 2 Caltech USP 2. 5 -10 Gb/s Tier 3 Tier 4 Physics caches FIU PCs UBA UNAM 28�

Grand Challenge Research: CHEPREO l l l An interregional grid-enabled Center for High. Energy Physics Research and Educational Outreach (CHEPREO) Fosters an integrated program of research, network infrastructure development, and education and outreach q Collaboration with FIU, Caltech, University of Florida, Florida State University, the State University of Rio de Janeiro, University of Sao Paulo q Augments bandwidth capacity to Brazil Joint funding by U. S. NSF (MPS-0312038) and State of Sao Paulo Research Foundation (FAPESP) 29�

u Caltech, UF, UMich, SLAC, FNAL, CERN, FIU, NLR, CENIC, UCAID, Translight, UKLight, Netherlight, Uv. A, UCLondon, KEK, Taiwan, KNU (Korea), UERJ (Rio), USP (Sao Paolo) u Cisco u u u Ultra. Light Collaboration: http: //ultralight. caltech. edu Integrated hybrid experimental network, leveraging Transatlantic R&D network partnerships; packet-switched + dynamic optical paths q 10 Gb. E across US and the Atlantic: NLR, LHCNet, Nether. Light, UKLight, etc. ; Extensions to Korea, Brazil, Taiwan End-to-end monitoring; Realtime tracking and optimization; Dynamic bandwidth provisioning Agent-based services spanning all layers of the system, from the 30� optical cross-connects to the applications.

Grid 3 Open Science Grid l Build on Grid 3 experience q Persistent, production-quality Grid, national + international scope l Ensure U. S. leading role in international science q Grid infrastructure for large-scale collaborative scientific research l Create large computing infrastructure q Combine resources at DOE labs and universities to effectively become a single national computing infrastructure for science l Provide opportunities for educators and students q Participate in building and exploiting this grid infrastructure q Develop and train scientific and technical workforce q Transform the integration of education and research at all levels http: //www. opensciencegrid. org 31�

32� Source: Cindy Zheng

Grand Challenge Science Instruments Gemini-South Optical Observatory NRAO telescopes La Serrena, Chile Atacama Large Millimeter Array (ALMA) Atacama plains in Chile 33�

Traditional VLBI The Very-Long Baseline Interferometry (VLBI) Technique (with traditional data recording) The Global VLBI Array (up to ~20 stations can be used simultaneously) 34�

Pan-American Advanced Studies Institute • NSF sponsored program to offer a series of lectures at the advanced graduate and postgraduate level involving domain researchers, students and practitioners. Award# 0418366, OISE Americas Program • Aims to disseminate advanced scientific and engineering knowledge, stimulate collaborative Mendoza Argentina learning and cooperation among the research communities of the Americas • CIARA, along with collaborators from the U. S. , Argentina and Brazil, is organizing a PASI to offer a series of lectures on the role of Grid Computing and Advanced Networking for High-Energy Physics and Astronomy • Our PASI is planned for May 15 -20, 2005 in Mendoza, Argentina • Approximately 40 students from the Americas will learn of the major experiments, Grid and advanced networking technologies and how the growing interdependence between the science and the technologies are 35� forming global collaborations

Thank You! l WHREN-LILA, AMPATH infrastructure, CHEPREO, science application support, education, outreach and community building efforts are made possible by funding and support from: q National Science Foundation (NSF) awards STI- 0231844, MPS-0312038, OISE-0418366 and SCI 0441095 q Florida International University q Latin American Research and Education community q The many national and international collaborators who support our efforts 36�