Storage accounting with DGAS and HLRmon Andrea Cristofori

Storage accounting with DGAS and HLRmon Andrea Cristofori, Enrico Fattibene 22 September 2011, OGF 33, Cité Internationale Lyon (France)

Accounting layout in IGI • • Distributed Grid Accounting System (DGAS) sensors collect accounting information at Grid site level Site data are sent to site or multisite Home Location Registers (HLRs) A top-level HLR receives Usage Records (URs) from all the HLRs HLRmon retrieves data from the top level HLR and presents them through a web interface. Aggregated data are also sent to the EGI Accounting Portal HLRmon – WORLDCOMP GCA’ 11, Las Vegas, July 18 th 2011 2

Storage accounting approach • Storage accounting system integrated in DGAS under development • Non standard schema has been used to generate storage URs no standard schema exists we joined the Open Grid Forum (OGF) working group on the storage UR definition • The information on the used space is collected from the BDII once a day specific sensors can be developed to retrieve more accurate data making the system independent from the BDII • Storage URs are stored on the HLR server just like the computing URs • HLRmon is able to retrieve storage information directly from the desired HLR • Current implementation of the sensor does not generate an XML (this will be changed in future implementations) HLRmon – WORLDCOMP GCA’ 11, Las Vegas, July 18 th 2011 3

Storage accounting approach • Some differences with the St. AR (Storage Accounting Record) defined in EMI: There is not a validity field but the Time. Duration that records the amount of time the storage has been used The UR provides a field for storing the specific file to account, if a per file account is chosen HLRmon – WORLDCOMP GCA’ 11, Las Vegas, July 18 th 2011 4

St. AR Record. Identity vs. DGAS+HLRmon Record. Identity Local. File. Id Global. File. Id Storage. System Storage. Media Storage. Class Directory. Path Local. User Local. Group User. Identity Group. Attribute Measure. Time Valid. Duration Resource. Capacity. Used Logical. Capacity. Used File. Count Host Storage. Type Service. Level Local. User. Id Local. Group. Id Global. Username Global. Group Project. Name Project. Partition Time. Instant Time. Duration Disk Charge Status Submit. Host Protocol. Type Network

Storage report • HLRmon shows the daily disk consumption in terms of used space • Data are aggregated per site, SE and Storage Area (SA) • Used space will be compared to the total available space (at SA, SE and site level) • The plotting of these data allows to detect wrong values published in the BDII Sites are notified in order to correct the values HLRmon – WORLDCOMP GCA’ 11, Las Vegas, July 18 th 2011 6

Storage charts Used space per site per day Used space per SA Used per SA per day HLRmon – WORLDCOMP GCA’ 11, Las Vegas, July 18 th 2011 7

Storage table Example of tabular view where the site, the SE and all the SAs related to the SE are shown. HLRmon – WORLDCOMP GCA’ 11, Las Vegas, July 18 th 2011 8