HSA Metrics Tasks from March Meeting Determine metrics
- Slides: 8
HSA Metrics
Tasks from March Meeting • Determine metrics that can be gathered by the HSA • Consider the use of a database to record metrics
Metrics Gathering • HSA logs “raw” metrics only, such as – Start/stop date time – Subsetting criteria • HSA does not record derived metrics like – Duration of subsetting job – Ratio of output file sizes to input file sizes • Derived metrics are not recorded because – They can easily be computed from raw metrics – Each site will have different requirements
Metrics Mechanism • Each HSA component calls a common routine to record a metric “event” • Each event has a corresponding event number, text, and optional parameters • Events are logged on a “per-order” basis using a separate, dated file for each order • Event files are ASCII text for easier reading and manipulation
Metrics File Example 2002 -05 -31 T 15: 12: 36 Z: 1000 Order accepted 2002 -05 -31 T 15: 12: 36 Z: 1002 REX started 2002 -05 -31 T 15: 12: 36 Z: 1003 Input data ordered 2002 -05 -31 T 15: 14: 43 Z: 1013 Input granule: SC: MOD 11_L 2. 003: 2004699928, file MOD 11 A 2. A 2001065. cs_konza. 003. 2001193080124. hdf (0. 680873 MB) 2002 -05 -31 T 15: 14: 43 Z: 1013 Input granule: SC: MOD 11_L 2. 003: 2004697909, file MOD 11 A 1. A 2001079. cs_konza. 003. 2001201211857. hdf (0. 696910 MB) 2002 -05 -31 T 15: 14: 44 Z: 1009 Lat/ lon bounds: (25. 000000, 54. 000000) – (-125. 000000, 66. 000000) 2002 -05 -31 T 15: 14: 44 Z: 1011 Selected field: “ 2380. 37 MHz” 2002 -05 -31 T 15: 14: 44 Z: 1011 Selected field: “ 53595. 41 +- 115 MHz” 2002 -05 -31 T 15: 14: 44 Z: 1011 Selected field: “ 88997. 00 MHz” 2002 -05 -31 T 15: 14: 44 Z: 1004 HEW started 2002 -05 -31 T 15: 14: 50 Z: 1007 6. 070000 seconds CPU time used 2002 -05 -31 T 15: 14: 56 Z: 1005 Delivering output data 2002 -05 -31 T 15: 14: 56 Z: 1014 Output file: HSA 0800000972_0800001012. 20020531151456. REPT (0. 00077 MB) 2002 -05 -31 T 15: 14: 56 Z: 1014 Output file: HSA 0800000972_0800001012. 20020531151456. REQ (0. 003764 MB) 2002 -05 -31 T 15: 14: 56 Z: 1014 Output file: MOD 11 A 2. A 2001065. cs_konza. 003. 2001193080124. hdf (0. 173699 MB) 2002 -05 -31 T 15: 56 Z: 1006 Order completed 2002 -05 -31 T 15: 56 Z: 1008 Completion status: Successful
Use of a Database • Not implemented within HSA because – It severely affects portability of the HSA – It ties the HSA to a DBM – It makes the recording of metrics dependent on an external process which may not be available • Sites may write their own routines to load the metrics into a database of their own design
Operations Tasks • Metrics gathering is controlled by the STATISTICS parameter in the HSA parameters file, which specifies the output directory to contain the metrics files • Metrics files are never deleted by the HSA; that is a site-specific procedure
Notes • All metrics messages are documented in the HSA Operator’s Guide • Parsing of messages is easy because messages are uniquely numbered • Every line is date and timestamped • Routines that process metrics files must be capable of handling – Incomplete files (i. e. , order never completed) – Missing lines (e. g. , no “Lat/lon bounds” line because spatial subsetting was not requested) – Lines not in anticipated order
