Grid PP 24 Data Prof David Britton Grid

Grid. PP 24 – Data! Prof. David Britton Grid. PP 24 Collaboration Meeting 14 th April 2010 David Britton, University of Glasgow IET, Oct 09 Grid. PP Project leader University of Glasgow Slide

Data Arrives! David Britton, University of Glasgow PPRP Slide 3

The Long Road… 383 PMB Meetings 24 th Grid. PP Collaboration Meeting, RHUL, 14 -15 April 2010 - Data. 23 rd Grid. PP Collaboration Meeting, University of Cambridge , 8 -9 September 2009 – Final Steps 22 nd Grid. PP Collaboration Meeting, UCL, 1 -2 April 2009 - Service Resilience 21 st Grid. PP Collaboration Meeting, Swansea University, 3 -4 September 2008 - 21: Coming of Age 20 th Grid. PP Collaboration Meeting, Trinity College, Dublin, 11 -12 March 2008 - Unreasonableness? 19 th Grid. PP Collaboration Meeting, Ambleside, 29 -31 August 2007 – The end of (Grid. PP 2) as we know it 18 th Grid. PP Collaboration Meeting, Glasgow, 20 -21 March 2007 17 th Grid. PP Collaboration Meeting, Ne. SC, Edinburgh, 1 -2 November 2006 16 th Grid. PP Collaboration Meeting, QMUL, 27 -29 June 2006 15 th Grid. PP Collaboration Meeting, RAL, 11 -12 January 2006 14 th Grid. PP Collaboration Meeting, University of Birmingham, 6 -7 September 2005 13 th Grid. PP Collaboration Meeting, Grey College Durham, 4 -6 July 2005 12 th Grid. PP Collaboration Meeting, Brunel, 31 January-1 February 2005 11 th Grid. PP Collaboration Meeting, Liverpool, 14 -15 September 2004 10 th Grid. PP Collaboration Meeting, CERN, Geneva, 2 -4 June 2004 9 th Grid. PP Collaboration Meeting, Ne. SC, Edinburgh, 4 -5 February 2004 8 th Grid. PP Collaboration Meeting, Bristol, 22 -23 September 2003 7 th Grid. PP Collaboration Meeting, Oxford, 30 June-2 July 2003 6 th Grid. PP Collaboration Meeting, Coseners House, 30 -31 January 2003 5 th Grid. PP Collaboration Meeting, Imperial College, 16 -17 September 2002 4 th Grid. PP Collaboration Meeting, Manchester, 9 -10 May 2002 3 rd Grid. PP Collaboration Meeting, Cambridge, 14 -15 February 2002 2 nd Grid. PP Collaboration Meeting, Edinburgh, 5 -6 November 2001 1 st Grid. PP Collaboration Meeting, Coseners House, 24 -25 May 2001 David Britton, University of Glasgow Grid. PP 24 Slide 4

… to the Start! David Britton, University of Glasgow Grid. PP 24 Slide 5

The Time is Now David Britton, University of Glasgow Grid. PP 24 Slide 6

Since Grid. PP 23 Ø First Data Ø Tier-1 Review Ø RCUK International Review of e-Science Ø Tier-1 Opening Ø … and of course the Grid. PP 4 proposal! David Britton, University of Glasgow Grid. PP 24 Slide 7

First Data From an Atlas presentation to w. LCG MB David Britton, University of Glasgow Grid. PP 24 Slide 8

Tier-1 Review Third in a series of annual reviews of the Tier-1. Reviewers included three people from CERN. Ø It was noted that the weekly Tier-1/experiments meeting had enhanced communications. Ø The out-of-hour service had been successfully developed. Ø The excellent performance of RAL during STEP 09 was noted. Ø The management processes had been successfully developed. Ø Many (but not all) of the problems last autumn were related to the new building R 89. Ø The Tier-1 was encouraged to develop staff-agility to provide more cover. Ø Concerns were voiced about the CASTOR upgrade path (Grid. PP 24 discussion). David Britton, University of Glasgow Grid. PP 24 Slide 9

RCUK e-Science Review Report explicitly mentions: • Grid. PP’s expertise in large-scale distributed data management and analysis. • Our work with start-up companies. • The substantial secondary economic benefit arising from the ability to rapidly screen drugs "in-silico”. Grid. PP resources were used in this way to screen potential agents in the fight against bird-flu and malaria. “NGS and Grid. PP have been highly successful, providing many users with access to more computing power than they could otherwise easily obtain. Looking forward, we recommend that these efforts, including enhanced capacity and function of distributed storage, be sustained and expanded. ” David Britton, University of Glasgow RAL Tier-1 Slide 10

Tier-1 Opening David Britton, University of Glasgow Grid. PP 24 Slide 11

Grid. PP 4 Time Line 12 Weeks 3 Weeks Ø Nov 5 th – invitation to bid. Ø Dec 10 th – Face-to-face PMB to agree structure Ø Dec 11 th – CB meeting to agree structure Ø Jan 15 th – Face-to-face PMB to agree draft v 6 Ø Jan 21 st – CB meeting to discuss v 6. Ø Jan 28 th – Submission of draft to Oversight Committee Ø Feb 4 th – Meeting with Oversight Committee + STFC Ø Feb 12 th – Near-final draft incorporating feedback Ø Feb 22 nd – Final comments/typos/corrections done Ø Feb 24 th – Submitted! Ø Mar 4 th – Last possible submission date Ø Apr 15 th - PPRP Ø May 14 th – PPRP Visiting Panel. David Britton, University of Glasgow Grid. PP 24 Slide 12

Proposal Structure WP-A: Tier-1 infrastructure and manpower; WP-B: Tier-2 infrastructure and manpower; WP-C: Grid deployment, operations and support; WP-D: Experiment-specific technical support; WP-E: Management team; WP-F: Impact Plan (Economic Impact, Knowledge Transfer and Outreach) WP-G: Travel and other costs. In the midst of writing the proposal we were told to reduce the project by 20%. This was very complex but is now built into submitted document. David Britton, University of Glasgow Grid. PP 24 Slide 13

Project Map David Britton, University of Glasgow Grid. PP 24 Slide 14

Project Deliverables David Britton, University of Glasgow RCUK e-Science Review Slide 15

Hardware Requirements Chamonix 2010 (Jan 2010) LHC Schedule Experiment Computing Models UK Collaboration sizes Global Resource Requirements Scrutiny by C-RSG Experience with real data (April 2010), CRRB, LHCC (May 2010). UK Resource Requirements Hardware Costings UK Resource Request (March+ 2010) A complex and evolving situation requiring constant management. Historically we have seen the integrated requirements are reasonably “good-and-improving” estimates of actual needs. David Britton, University of Glasgow Grid. PP 24 Slide 16

Hardware Scaling Tier-1 CPU Capacity Hardware requirements have increased exponentially – fortunately Moore’s Law has held true with about the same time constant. KSI 2 K vs Year RAL CPU costs Logarithmic scale ~2 -year Moore’s Law Ramp for LHC start David Britton, University of Glasgow Grid. PP 24 Slide 17

Hardware Costs For CPU, there is a remarkably good (in fact, too-good) agreement between the CERN and Grid. PP predictions. For Disk, factoring in different RAID strategies etc, estimates agree at about 5% level, which is better than the exchange-rate uncertainty alone. 27 pages Comparing predictions is even more difficult… David Britton, University of Glasgow Grid. PP 24 Conclude: Grid. PP’s empirical approach leads to the same costing as an extensive topdown investigation of market and technology trends suggesting the Moore’s law assumption continues to be valid over the this time period. Slide 18

The EGI/NGI Structure Grid. PP operates the largest scientific Grid in the UK and must ensure the UK NGI meets the needs of the large community we serve. We have identified certain tasks that are absolutely required by Grid. PP and w. LCG, which have wider UK applicability and would sensibly be done in close collaboration as part of a larger team within a UK NGI. We have used this as a way to optimize our manpower, our leadership, and our influence in these areas. We believe that JISC is committed to funding an NGI at some level (signed up to EGI) but we have examined alternative scenarios and included in contingency. David Britton, University of Glasgow European Framework-7 proposals EGI. eu EMI EGI. inspire NGI Management Board NGI Operations Coordinator (John Gordon) Grid. PP Security Grid. PP 24 NGI Operations Helpdesk and Regional Support NGS Accounting and Monitoring Slide Training/ Outreach 19

A UK NGI NGS 4 and EGI funded Grid. PP 4 funded EGI not funded NGS 4 not funded David Britton, University of Glasgow RCUK e-Science Review Slide 20

Risks Register Covered by: W. A. (0. 5 FTE) WP-D Expt. Support WP-C Data Man. Old (atlas) building and other measures Tier-2 strategy W. A. (0. 5 FTE) WP-C security team W. A. (1. 0 FTE) … 22 high level risks analysed in proposal. David Britton, University of Glasgow Grid. PP 24 Slide 21

Risks Register - 2 These risks are covered by the Contingency David Britton, University of Glasgow PPRP Slide 22

Publications Maintain a list of publications (besteffort basis, so incomplete) on the Grid. PP website at: http: //www. gridpp. ac. uk/papers/. Currently 220 publications (2001 – 2009). 2009 papers: 22 papers, 105 (~70 unique) authors David Britton, University of Glasgow Grid. PP 24 Slide 23

In Summary David Britton, University of Glasgow PPRP Slide 24