LCG Phase2 Planning David Foster ITCS 14 th

LCG Phase-2 Planning David Foster IT/CS 14 th April 2005 Thanks to Dante, ASNet and ESnet for material presented at the T 0/T 1 Networking meeting in Amsterdam on 8 th April 2005

European Tier 0/Tier 1/Tier 2 Connectivity Overview • A total of 7 NRENs serving Tier 1 sites in Europe plus one in Asia Pacific • All currently have at least 1 Gbps IP connections to CERN • 10 Gbps lambda available on timescales ranging from July 2005 to autumn 2006 • Number, location and bandwidth requirements of Tier 2 sites unclear to many NRENs

GÉANT 2 Project Partners

Red. IRIS (Spain) • Connecting the PIC Tier 1 site in Barcelona • Traffic crosses 3 domains prior to reaching GÉANT 2: – PIC network – Anella Cientifica (Catalan regional network) – Red. IRIS • Currently 1 Gbps VPN is supported • Upgrade planned for Red. IRIS connection to Catalan network, date TBD • No request has yet been received from PIC for 10 G lambda • PIC requirement timeline unclear • 7 Tier 2 sites are known in Spain • Bandwidth requirement of Tier 2 sites unknown • Tier 2 sites connectivity varies from GE to STM-4 • Cost sharing TBD with Spanish ministry

DFN (Germany) • DFN will connect the Tier 1 site at Karlsruhe to CERN via GÉANT 2 • Presently 10 G is available over GÉANT (Layer 3), providing LSP Karlsruhe-to-GÉANT-to-CERN • Testing is already taking place and high-datarate transfers have been shown Karlsruhe to CERN • Tier 2 centres are not yet known so provision is unclear • Cost sharing: Karlsruhe will pay a subscription to DFN, a proportion of which will be passed to GÉANT 2

GARR (Italy) • GARR will connect the Bologna Tier 1 site to CERN via GÉANT 2 • 10 Gbps lambda ring provided by GARR, connecting INFN-CNAF (Tier 1) and GÉANT 2 Po. P in Milan will be operational by September 2005 • By the end of 2005, multiple lambdas will be available from this site to GÉANT 2, allowing as many 10 Gbps connections as required • GARR connects Bologna Tier 1 to other Tier 1 s via GÉANT 2 • 12 Italian Tier 2 sites identified, all with DF to GARR backbone • 8 Tier 2 sites already have 1 Gbps connection. All will have 1 Gbps connectivity by September 2005 • GARR will bill INFN for all services provided, details of the cost sharing TBD

UKERNA (UK) • • • UKERNA will connect the RAL Tier 1 site to CERN via GÉANT 2 2 x 1 Gbps RAL-CERN via UKLight possible now 10 G lambda RAL-UKLight (switched port)-GÉANT 2 by end 2006 Cost will be addressed by UK national funding (discussions ongoing) -a proportion being channelled to GÉANT 2 Four distributed Tier 2 sites: North. Grid, South. Grid, Scot. Grid, London. Grid: bandwidth requirements unknown

SURFnet (Netherlands) • SURFnet will connect the Tier 1 site at SARA, Amsterdam • SURFnet 6 will provide a 10 G lambda to SARA by July 2005 • Initially 10 G Lambda to CERN will be provided by SURFnet, later by GÉANT 2 when available • Tier 2 sites in the Netherlands will be connected via 10 G lambdas by January 2006 • 1 G lightpaths will be provided over Nether. Light and/or GÉANT from Dutch Tier 2 s to non-Dutch Tier 1 s • SURFnet will absorb networking costs of the NL-access to CERN via GÉANT 2 and all costs inside NL for accessing the Tier 1 and Tier 2 s

RENATER (France) • RENATER will connect the IN 2 P 3 (Lyon) Tier 1 site directly to CERN (not via GÉANT 2) • RENATER will procure dark fibre between Paris, Lyon and CERN • 10 G lightpath will be provided Lyon-CERN by July 2005 • Tier 1 -Tier 1 traffic TBD • Traffic to/from the 3 French Tier 2 sites will pass over the RENATER network • Cost sharing TBD

NORDUnet (Nordic Countries) • NORDUnet will connect the ‘distributed’ Tier 1 site in the Nordic countries • Connectivity via lambdas can be provided by mid-2006 for all the sites concerned • Cost sharing TBD

SWITCH (Switzerland) • • • The Tier 1 site at CERN is connected directly to Tier 0 There are no Tier 1 sites connected by SWITCH The Tier 2 site, CSCS has 10 GE if necessary CSCS will connect directly to CERN (ie not via GÉANT 2) Cost of this connection will be borne by SWITCH

European Tier 1 SUMMARY • High data rate transfer tests Karlsruhe to CERN already underway • 10 G available now (L 3) from Bologna to CERN • Testing of 10 G lambdas from Lyon and Amsterdam can commence from July 2005 • Amsterdam (and Taipei) will use Netherlight link to CERN until GÉANT 2 paths are available • Testing of Barcelona link at 10 G from October 2005 • Nordic distributed facility restricted to 1 G until late-2006 when 10 G available • RAL could operate between 2 and 4 x 1 GE (subject to scheduling and Nether. Light/CERN agreement) until late-2006 when 10 G available. Interconnection of UKLight to GEANT 2 might make an earlier transition to higher capacities possible.

SC test bed @ Taipei

Taipei Status • ASNet* runs one STM 16 IPLC from Taipei to Amsterdam • One GE Local loop from Amsterdam to Geneva (via Nether. Light. Path Service), second GE is waiting to turn on (I already signed the xconnect order form) • will double the IPLC on July 1 st (contract will be signed on May 27 th) • ASCC LCG facilities now have multiple GE uplinks to ASNet’s core router; will be replaced by one or two 10 GE during this summer vacation • ASNet connects (10 GE*2 + STM 64*n) to domestic backbone, a. k. a. TANet/TWAREN joint backbone, to reach T 2’s in Taiwan. • Every T 2 in Taiwan has it’s own 10 GE link to the domestic backbone * ASNet (Academic Services Network, as#9264) is the network division and is also the network name that registered in APNIC.

Asia Pacific Link Status • How many T 2 s in Asia Pacific region? • Taipei – Tokyo – STM-4 IPLC – 2*GE to APAN-JP (for JP Universities) – Will have GE to Si. Net/Super. Si. Net (for KEK) – IPLC should be increased before LCG production run. • Taipei – Singapore – STM-1 IPLC to Sing. AREN (National Grid Office of Singapore)

Asia Pacific Link Status • Taipei – Hong Kong – STM-4 IPLC, will upgrade to STM-16 on Feb 1 st, 2006 (contract already been signed on March 30 th) – GE link to CERnet, GE link to CSTnet • CERnet : China Education and Research network • CSTnet : China Science and Technology network – Maybe have GE link with KOREN (still work with KOREN engineer)

ASNet (Taiwan) Summary • ASNet connect the Taipei Tier 1 site • Currently Taipei is connected via an STM 16 link (IPLC) to Amsterdam, this will double in July 2005 • Currently traffic is routed over GE local loop and via Nether. Light to CERN • Quality is assured by MPLS and PIP service • A move to a GÉANT 2 connection Amsterdam-CERN is planned • A WDM solution is being considered, no decision as yet • Tier 2 sites are connected in Taiwan (by 10 GE to ASNet), Japan and China

SINet (Japan) Japan – Russia (BINP) GPo P CA*net 4 France GLORIAD Kreonet 2 MREN Netherlands Star. Tap TANet 2 Taiwan (ASCC) PNW Australia CA*net 4 Taiwan (TANet 2) Singaren CERN (LHCnet – part DOE funded) GEANT - Germany, France, Italy, UK, etc SEA HUB LIGO e hi L TWC QWEST ATM LLNL SNLL NA P AMES SNV HUB PAIX-PA V L-N Equinix, etc. HTE C E B L SNLA B HU ALB high-speed peering points MAE-E Equinix Allied Signal DOE-ALB PA DC HUB ORNL NOAA EX JLAB MA XG ORAU Po P ATL HUB Abilene NT 42 end user sites HUB Office Of Science Sponsored (22) ELP NNSA Sponsored (12) Joint Sponsored (3) Other Sponsored (NSF LIGO, NOAA) ESnet IP core: Packet over Laboratory Sponsored (6) SONET Optical Ring and Hubs peering points SNV SDN HUB SND core hubs SNV HUB IP core hubs Abilene PPPL 4 x. LAB-DC OSTI BNL LLNL/LANL-DC KCP LANL MIT INEEL-DC ORAU-DC GTN&NNSA ARM GA FNAL ANL CHI HUB YUCCA MT SDSC HUB len e INEE Abi C CHI-SL HUB NRE SNV SDN HUB B U H ilen JGI LBNL NERSC SLAC YC N t gh rli a St ESnet IP core MA N Ab LA ile N ne PNNL Ab ESnet Science Data Network (SDN) core ESnet High-Speed Physical Connectivity to DOE Facilities and Collaborators, Summer 2005 SRS So XG Po P International (high speed) 10 Gb/s SDN core 10 G/s IP core 2. 5 Gb/s IP core MAN rings (≥ 10 G/s) OC 12 ATM (622 Mb/s) OC 12 / Gig. Ethernet OC 3 (155 Mb/s) 45 Mb/s and less

ESnet • The IP core is primarily a layer 3 infrastructure – However, supports layer 2 via MPLS – Directly connects sites – Provides global peering for sites • The SDN core is primarily a layer 2 infrastructure – Targeted at providing virtual circuit services

ESnet Target Architecture: IP Core + Science Data Network + MANs CERN Asia. Pacific Aus. Seattle (SEA) GEANT (Europe) ESnet Science Data Network (2 nd Core) o Chicag Sunnyvale (SNV) Aus. Metropolitan Area Rings Core loops ESnet IP Core Albuquerque (ALB) Existing IP core hubs SDN hubs New hubs Primary DOE Labs Possible new hubs (CHI) New York (AOA) Washington, DC (DC) Atlanta (ATL) El Paso (ELP) Production IP core Science Data Network core Metropolitan Area Networks Lab supplied International connections

ESnet Near-Term Planning for FNAL CERN ORNL OC 192 NRL, Ultra. Science. Net, etc. T 3 20 T 320 ESnet IP core ESnet SDN core IWire Qwest hub (NBC Bld. ) Starlight Notes FNAL Qwest FNAL Shared w/ IWire site equip. ESnet Switch/RTR FNAL ESnet site gateway router All circuits are 10 Gb/s ESnet fiber ESnet/ Qwest

ESnet Planning for BNL (Long Island MAN Ring) Other connections Engineering Study for LI MAN

Asia. Pac ESnet Goal – 2007/2008 SEA • 10 Gbps enterprise IP traffic • 40 -60 Gbps circuit based transport CERN Aus. ESnet Science Data Network (2 nd Core – 30 -50 Gbps, National Lambda Rail) Europe Japan SNV CHI NYC DEN Aus. DC Metropolitan Area Rings ALB SDG ESnet IP Core (≥ 10 Gbps) ATL ESnet hubs New ESnet hubs ELP Metropolitan Area Rings Major DOE Office of Science Sites High-speed cross connects with Internet 2/Abilene Production IP ESnet core Science Data Network core Lab supplied Major international 10 Gb/s 30 Gb/s