Grid PP Prof David Britton PPAP Community Meeting
Grid. PP Prof. David Britton PPAP Community Meeting Imperial, 24 th Sep 2015 David Britton, University of Glasgow IET, Oct 09 Grid. PP Project leader University of Glasgow Slide
Outline The aim of this meeting is to review the experimental and theoretical particle physics projects currently being pursued by our UK community and to discuss possible future directions. • • Grid. PP: The context. The Grid. PP 5 proposal. What we do and how we plan to evolve. Looking to the future. David Britton, University of Glasgow PPAP Slide 2
The Grid Higgs Discovery 2012 2013 2015 The Grid is an essential, though preferably invisible, part of the infrastructure that enabled the Higgs discovery. It has delivered on-time, over-spec, and within budget. David Britton, University of Glasgow PPAP Slide 3
Grid. PP in 2015 WLCG: 170+ sites in 42 countries 515 K CPU; 290 PB Disk; 150 PB Tape Grid. PP is ~10% of WLCG 18 sites; 45 K CPU; 32 PB Disk; 14 PB Tape CPU Delivered during Grid. PP 4 2011 -15 9% 11% David Britton, University of Glasgow PPAP 10% 9% Slide 4
LHC and More 9% of Tier-2 CPU and 4% of Tier-1 CPU was delivered to 32 non-LHC VOs between Jan 2012 and Dec 2014 Past Future See Pete Clarke’s talk tomorrow. David Britton, University of Glasgow PPAP Slide 5
Grid. PP 5 Proposal: Process • Grid. PP 5 proposal originally submitted in early 2014 to the PPRP where it was effectively tensioned against the upgrades. Proved impossible to fund both out of the envelope provided. • Grid. PP 4 was given a 1 -year extension to March 2016 and a new Grid. PP 5 proposal requested in early 2015 to go to the PPGP. This time, tensioning was against the consolidated grants. • Grid. PP 5 proposal addressing 3 scenarios: – – – Flat Cash (i. e. 10% reduction) 90% FC (i. e 19% reduction) 70% FC (i. e 37% reduction) • A special review committee provided the PPGP with advice on the funding Grid. PP required and provided a report to STFC Science Board. David Britton, University of Glasgow PPAP Slide 6
Grid. PP 5: Brief Tier-1 and Tier-2 operations User support; Integration with experiments; and shape the future. Adapt to changes. Support LHC and non-LHC physics. Increased importance of ecosystem. Development and innovation necessary! David Britton, University of Glasgow PPAP Slide 7
Strategic Objectives 1. To meet STFC's Mo. U commitment to CERN and the WLCG by ensuring that Grid. PP is able to handle the challenge of higher data rates and volumes of LHC Run-2. 2. To prepare Grid. PP for the 2020 start of LHC Run-3 by exploiting developments in IT technology and by influencing and contributing to WLCG's future technical direction and development. 3. To reduce the cost to STFC of operation of Grid. PP by developing cost-sharing collaborations and by evolving the infrastructure to reduce the operational cost. David Britton, University of Glasgow PPAP Slide 8
Our Summary to the Review Panel • With Flat Cash we can meet our strategic objectives… continuing to deliver appropriate resources and excellent service to the LHC experiment and the broader community. In addition we can play a leading role in developing the UK/EU-T 0 agenda with the ultimate goal of creating a more general and sustainable infrastructure. • With 90% of Flat Cash, we can mainly meet our strategic goals… but we will not have resources to lead the development of the UK e-infrastructure. • With 70% of Flat Cash, we will not meet our strategic goals, except in the sense that we reduce the cost of Grid. PP. We will not deliver the resources or service level expected by the WLCG Mo. U. We will reduce the capability of the RAL Tier-1 to respond to requirements from other collaborations. David Britton, University of Glasgow PPAP Slide 9
Feedback from Science Board • Recommend that the 90% of Flat Cash scenario be funded but with various specific reductions/normalisations. David Britton, University of Glasgow PPAP Slide 10
Service Decomposition In order to develop and present the various Grid. PP 5 scenarios we looked closely at how Grid. PP effort contributes. In particular, we noted that: • The majority of Grid. PP effort is not used to run hardware. • Grid. PP staff effort does not write experiment software. Ai. BM In fact, Gridd. PP’s value is the support and provision of a service layer that lays between the hardware and the software. After the submission of the Grid. PP 5 proposal, WLCG also examined the effort required worldwide and we were able to use some of this information in Grid. PP 5 review process. David Britton, University of Glasgow Grid. PP 5 Review Slide 11
EGI Services required by WLCG that are provided via the EGI project. The UK leads two of these, co-leads a third, and makes contributions to two more. UK provides 14% of effort: 7. 3% funded by Grid. PP; 1. 3% by other UK sources; and 5. 3% directly by EGI. UK certainly does not contribute more than a fair share: We host 17% of EGI resources (denominator does not include US WLCG resources). David Britton, University of Glasgow PPAP Slide 12
WLCG Services WLCG services not provided via EGI: The UK contributes to 13 services and co-leads two. Most WLCG partners contribute to most of these shared international tasks. Grid. PP funds 9. 7% of the effort required to deliver the WLCG service infrastructure; an additional 1. 6% is provided by the Tier-2 s (leverage). Contribution consistent with size of UK (we provide ~12% of LHC resources) David Britton, University of Glasgow PPAP Slide 13
UK Services: These are services that every country needs to perform for the benefit of their own infrastructure. These 14 tasks that are core business for Grid. PP. Some of these services do (and increasingly can) benefit other UK infrastructures. The effort corresponds to 9. 5 FTE funded from Grid. PP, complemented by 0. 51 FTE funded from other UK sources. David Britton, University of Glasgow PPAP Slide 14
Tier-1 Plan • WLCG Site Survey suggested that the manpower at RAL is already lower than comparable sites. • The plan is to reduce this by 10% at the start of Grid. PP 5 and by a further 15% after the first two years. • We plan to achieve this by making things easier to run and broadening the user-base to share costs. We need to invest effort now into both these areas in order to benefit in the future. • This is an ambitious plan. David Britton, University of Glasgow PPAP Slide 15
Tier-2 Plan • The Tier-2 infrastructure needs to balance the advantages of leverage, local support and engagement, redundancy, and the use of distributed bandwidth, with the economies of scale that may arise from less fragmentation of resources. • We have pointed out that much of Grid. PP effort delivers a service, rather than runs hardware, so the required effort does not scale linearly with the number of sites. • Nevertheless, we must reduce manpower and we have a plan that reduces the effort at Tier-2 sites by 28% over Grid. PP 5. • The plan is to evolve to 5 major Tier-2 sites and up to 6 smaller Tier-2 sites that can be run with minimal effort. • We hope sites with no Grid. PP funded effort will still be engaged. • This is an ambitious plan. David Britton, University of Glasgow PPAP Slide 16
Tier-2 Evolution • Fully convert sites to Virtual Machine operation with VM definitions supplied and maintained by the experiments. • Minimise need for sites to operate job execution grid middleware or tune batch systems. • Make use of Cloud infrastructures that might exist using Grid. PP -developed Vcycle to manage VMs. • Alternatively, use Grid. PP-developed Vac system to manage VMs on dedicated resources. • Ease burden of managing storage by making use of technologies like xrootd and Web. DAV that help remove the dependence on any specific copy of the data. develop to e rc fo sk a T n o ti lu vo p an E Contacts Grid. PP has now set u. is th ve ie ch a to se o we prop detailed plans on how vid Colling. a D d n a b a c. N M w re d are An David Britton, University of Glasgow PPAP Slide 17
WLCG Survey: Tier-2 s The WLCG site-survey received responses from 85 WLCG Tier-2 sites and showed that the UK Tier-2 sites were run with about half the average effort. We concluded that the Grid. PP Tier-2 infrastructure is extremely costeffective and that the Grid. PP 5 proposal presents an ambitious plan to take a lead in further reducing the effort required to run WLCG Tier-2 infrastructure. David Britton, University of Glasgow PPAP Slide 18
Looking Forwards Over the next four years we need to: – Deliver the steadily increasing Mo. U commitments of resources at the Tier-1 and Tier-2 s. – Maintain and improve the services we provide. – Contribute to, and in places lead, the evolution of the WLCG infrastructure. – Reduce the effort required to run our infrastructure. (WG) – Take a significant role in developing/promoting UK-T 0. – Engage and support new user groups. – Engage with other e-infrastructures in the UK. – Engage with European initiatives such as EGI, EU-T 0, HNSci. Cloud. – Respond appropriately to the RCUK Impact agenda. – Prepare for the period after Grid. PP 5 (very different? Grid. PP? ) David Britton, University of Glasgow PPAP Slide 19
Last Words • Significant evolution required throughout Grid. PP 5. • We have to run hard to stand still. • Increased emphasis on engaging with new groups, particularly STFC-funded. • Increased emphasis on linked infrastructures, nationally and internationally. • Reality check required – after 8 years of flat cash and now less, Grid. PP can not do significantly more with significantly less. David Britton, University of Glasgow PPAP Slide 20
Back up Slides David Britton, University of Glasgow PPAP Slide 21
Hardware Statement Grid. PP anticipates that some sites with no Grid. PP-funded effort would still be interested in running Grid. PP-funded hardware during Grid. PP 5. In principle, we feel that this offers potential benefits in terms of leverage of additional resources, engagement with local groups, and opportunities to develop additional synergies with other computing infrastructures. However, these potential advantages must be balanced against the potential effort required by both Grid. PP and, in particular, by the global experiment computing infrastructure (such as the ATLAS ADC), to keep these sites functioning at the required level of service. Particularly with new technologies, Grid. PP believes that this will be feasible and see the provision of hardware at non-staffed sites as a good opportunity to access additional hardware when it available, as well as engaging with the wider community. However, Grid. PP will need to discuss with the experiments the appropriate level of capacity hardware that might be located outside of the dedicated centers and will need monitoring/processes in place to protect the experiment infrastructures from additional work and maintain the UK reputation. David Britton, University of Glasgow PPAP Slide 22
Leverage • Capital costs primarily reflect contributions in the form of dedicated machine rooms. • Hardware is estimated by looking at Tier-2 resources reported in the Grid Accounting system, compared to the resources Grid. PP has funded. • Electricity has been estimated from the total resources reported. • Manpower is that reported in the Grid. PP quarterly reports, which is not funded by Grid. PP. David Britton, University of Glasgow Grid. PP Review Slide 23
Tier 1/2 Functional Services David Britton, University of Glasgow Grid. PP 5 Review Slide 24
- Slides: 24