CF Computing Facilities CERN Computing Facilities Evolution Status

CF Computing Facilities CERN Computing Facilities’ Evolution Status and Business Continuity Plans GDB January 16 th 2013 CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it

CF Overview • • • CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it Computer Centre Upgrade Project Status Reminder of Remote Hosting Selection Remote Hosting Status and Issues Business Continuity Plans Conclusion GDB January 16 th 2013 - 2

CF Overview • • • CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it Computer Centre Upgrade Project Status Reminder of Remote Hosting Selection Remote Hosting Status and Issues Business Continuity Plans Conclusion GDB January 16 th 2013 - 3

CF Goals of the CC Upgrade Project • Solve the cooling issue for the critical UPS room – New UPS systems in a different location CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it • New, independent and redundant cooling installation for critical equipment, including the UPS system. • Increase critical capacity to 600 k. W • Increase overall power capacity to 3. 5 MW • Restore N+1 redundancy for both critical and physics UPS systems • Secure cooling for critical equipment when running on UPS and extended stored cooling capacity for physics • Move of AHUs to UPS (not in original scope but added later) • Decouple the A/C for CC from the adjacent office building GDB January 16 2013 - 4 th

CF CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it GDB January 16 th 2013 - 5

CF Current Status • Civil engineering works completed except for a further water tank and sound proofing around the new chillers • New electrical systems fully operational • Ventilation systems for: – The various electrical rooms operational – R-050 and Telecom and other labs installed but not yet operational • New mixed water station operational – Full pressure test with racks recently done CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it • New IT room ready, racks installed and cabled and servers being installed • R-050 existing AHUs now on UPS • AC in B 31 decoupled from B 513 GDB January 16 th 2013 - 6

CF Overview • • • CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it Computer Centre Upgrade Project Status Reminder of Remote Hosting Selection Remote Hosting Status and Issues Business Continuity Plans Conclusion GDB January 16 th 2013 - 7

CF Wigner Data Centre Location • • CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it Easy reach to the airport and city centre Huge area within the fence Highly secure area Area only for research institution

CF CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it Wigner Area GDB January 16 th 2013 - 9

CF CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it Layout GDB January 16 th 2013 - 10

CF CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it Artist’s Impression GDB January 16 th 2013 - 12

CF Operational Model • Wigner to be considered as an extension of the CERN Tier 0, i. e. should be transparent for the experiments • IT Operations to be done as if systems were at CERN – All operations not requiring physical intervention to be done remotely from CERN – All physical interventions to be done by Wigner staff on request from CERN using CERN ticketing system • Wigner responsible for all facility operations CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it GDB January 16 th 2013 - 14

CF CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it Data Centre Layout & Ramp-up GDB January 16 th 2013 - 15

CF Overview • • • CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it Computer Centre Upgrade Project Status Reminder of Remote Hosting Selection Remote Hosting Status and Issues Business Continuity Plans Conclusion GDB January 16 th 2013 - 17

CF Overview of Status and Issues - I • Adjudicated in March FC and contract signed early May • Works started on 21 st May 2012 and everything required for the first room expected to be operational at beginning of 2013. • Remaining works to carry on until June 2013. • Two 100 Gbps links have been ordered – One commercial provider and DANTE – Due to be operational at beginning of Feb 2013 • Extensive discussions of rack layout and networking topology – 5 rack grouping chosen (see following slides) CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it GDB January 16 th 2013 - 18

CF CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it WAN Routing GDB January 16 th 2013 - 19

CF CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it Status and Issues PES Group Meeting 20 th November 2012 - 20

CF CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it Status and Issues GDB January 16 th 2013 - 21

CF Overview of Status and Issues - II • Brocade routers ordered and delivered CERN • HP switches ordered and delivered to CERN • Routers and switches (and networking servers) are being delivered to Wigner mid-January – Networking team will assist with first installation • CPU servers and disk storage units ordered directly after the December FC – – Two suppliers for each 100 (80+20) 2 U CPU servers and 80 4 U disk storage units Delivery foreseen end February/beginning of March Operations team will assist with first installation • Operational Handbook being developed • Draft SLA exists • Handling of CERN immunities and privileges (VAT and customs) – Has been very difficult (not the principle but the logistics) CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it GDB January 16 th 2013 - 22

CF Overview • • • CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it Computer Centre Upgrade Project Status Reminder of Remote Hosting Selection Remote Hosting Status and Issues Business Continuity Plans Conclusion GDB January 16 th 2013 - 23

CF Business Continuity • With a 2 nd DC it makes sense to implement a comprehensive BC approach – Safe. Host provides only very limited BC capability • First classification of services against three BC options: 1. Backup (quick switch over) 2. Load balancing (split over the two sites) 3. Re-installation (taking an existing machine) • An internal study was conducted to see what would be required to implement BC from a networking point of view – Requires a network hub to be setup independently of the CC – Requires an air-conditioned room of about 40 -50 sqm for ~ 10 racks with 50 -60 k. W of UPS power and good external fibre connectivity – Several locations were studied and the proposed location is B 272 CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it GDB January 16 th 2013 - 24

CF Second Network Hub • Diagram from Eric CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it GDB January 16 th 2013 - 25

CF Possible Network Topology US GEANT ISPs 300 M T 1 s 100 Gb. E B. 513 AS 513 CERN users Wigner Budapest GPN: 137. 138 LCG: 128. 142 AS xx 100 Gb. E GPN: 188. 184 LCG: 188. 185 B. 272 CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it ISP 10 G T 1 s (Renater, Surf. Net, SWITCH) GDB January 16 th 2013 - 26

CF Business Continuity • Will start with the second installation in second half of 2013. Nonetheless: – Wigner will be completely dependent on CERN for whole of 2013 (CS needs to understand implement correct configuration and routing) – Internet link only foreseen for end of 2013 • Not the same differentiation as at CERN, i. e. no difference in location, cooling or powering. – Same electrical and cooling infrastructure. – Except with a 10 minutes warning to shutdown. • Second network hub (foreseen in B 272) will not be available until after LS 1. – Requires a lot of fibres to be pulled by EN-EL. • Survey of groups for first services to be set-up in Wigner, and their requirements, underway. • Once services are established at Wigner, Safe. Host will be gradually phased out. CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it GDB January 16 th 2013 - 27

CF Conclusion • Computer Centre upgrade project nearing completion • Remote Hosting – Adjudicated in March and contract signed early May – Construction • Started 21 st May 2012 • First room operational and remaining works due to be finished June 2013 • Minor opening ceremony 18 th January and more major May/June – Delivery • Networking equipment being delivered • First servers in February/March 2013 – Goal for 1 Q 2014 to be in production as Iaa. S with some level of BC to be gradually extended • Business Continuity CERN IT Department CH-1211 Geneva 23 Switzerland www. cern. ch/it – To start with second delivery 2013 – Full BC requires second network hub (not before 2015) and a dedicated Internet connection at Wigner (not before 2014) GDB January 16 th 2013 - 29