Proposal for a Dynamic Megatable Flavia Donno CERNIT
Proposal for a Dynamic Megatable Flavia Donno CERN/IT WLCG Management Board, CERN 24 June 2008
The requirements n The goal n Provide the management with information about installed capacity (per VO). n Provide the management with information about resource usage (per VO). n This info can also be used by VO operations people in order to "monitor" their usage of the resources n Main focus n Storage resources n Computing resources n Dynamic view n Information should be retrieved as dynamically as possible in order to be reliable n The information will be made available through the WLCG accounting system WLCG Management Board, CERN 24 June 2008 2
The documentation n The technical specifications are available in the CCRC twiki in the SSWG section: https: //twiki. cern. ch/twiki/bin/view/LCG/WLCGCommon. Computing. Readiness. Challen ges#Storage_Solution_Working_Group_S n Storage only: n The documentation available reports on the conclusions reached during focused meetings with developers and information providers n Specific solutions found to cover d. Cache internal specialized buffers and avoid double counting n The document has been agreed by storage developers, storage information providers, data management developers, … WLCG Management Board, CERN 24 June 2008 3
The Storage Resources n Current accounting harvests data from BDII n … but many errors in data. n combination of Glue 1. 2 and 1. 3 n Need to improve information providers and configuration. n Existing Reports n Current status (last hour) n Monthly Report (like manual Tier 1 reports). WLCG Management Board, CERN 24 June 2008 4
The Storage Resources n Glue. SA class n Online resources only. Glue. SAs do not overlap. n Shared spaces as one Glue. SA with multiple Glue. SAAccess. Control. Base. Rule n Glue. SAReserved. Online. Size n Space physically allocated to a VO (or a set of VOs) – Installed capacity n Glue. SATotal. Online. Size (in GB = 109 bytes) n Total Online Space available at a given moment (it does not account for broken disk servers, draining pools, etc. ) n Glue. SAUsed. Online. Size (in GB = 109 bytes) n Size occupied by files that are not candidates for garbage collection – Resources usage n Glue. SAFree. Online. Size (in GB = 109 bytes) n Free = Total - Used n Glue. SACapability n Key/value pair to publish agreed characteristics of Storage Area, i. e. access=read, network=WAN-LAN, etc. WLCG Management Board, CERN 24 June 2008 5
The Computing Resources n Expressed in terms of KSI 2000 per federation in current Megatable n APEL provides already the information about resources usage n Ongoing work to make information published about installed capacity more reliable (see presentation of Steve Traylen at GDB) n Grid. Map already provides the information per region. Ongoing work (in collaboration) to provide a view per federation and an interface to retrieve data. n Proposal: Publish static pledges and the dynamic view in the current Megatable format (by federation) – provide APEL plug-in n It allows for easily spotting discrepancies n Profit of the presence of a technical student who will be at CERN till the end of August. WLCG Management Board, CERN 24 June 2008 6
The schedule n Storage n Dynamic information providers for CASTOR already available by J. Jensen n They will be installed at CERN n Few minor corrections needed n Dynamic information providers for d. Cache available by R. Trompert n Need to verify with d. Cache developers that pinned files usage info is available, and other details n 2 -3 weeks to implement the proposal n The changes will be reflected also in the new official d. Cache information providers by P. Millar n Dynamic information providers for DPM available n A few weeks to implement the proposal n Information already available for Sto. RM n Probable minor additions for VOInfo. Path n Computing resources n Provide a working prototype by the end of July 2008 ? WLCG Management Board, CERN 24 June 2008 7
Thank You WLCG Management Board, CERN 24 June 2008
- Slides: 8