Grid Event Management Using R GMA Monitoring Framework

Grid Event Management Using R -GMA Monitoring Framework ISGC April 27 -29, 2005 Min Tsai ASCC/CERN

Overview l l l Introduction to R-GMA Monitoring Framework Event Management System

R-GMA Introduction (Slides from Steve Fisher) l Background l l R-GMA developed in EDG Deployed by LCG Re-engineered for the EGEE project Uniform data transport mechanism for: l l Measurement Resource information Logging Monitoring

R-GMA – one cloud Producer l A relational implementation of GMA (from GGF) l Powerful data model and query language l R-GMA l l Consumer All data modelled as tables SQL can express complex queries in one expression Creates impression that you have one RDBMS

Example Deployment Site A Site B Server Registry Service Server Schema Service Registry Service Server Producer Service API Site C Server Consumer Service Consumer application Producer Service Key Consumer Service API control data Producer application Site D

The Producer Service l Producer created Key control data API Producer application declare Producer Service

The Producer Service l Registry Service contacted to register producer Key control data Registry Service register producer API Producer application declare Producer Service

The Producer Service l l Data transferred from the application to Producer Service If there are no consumers then the data goes no further than the Producer Service Key control data Registry Service register producer API Producer application declare insert Producer Service

Producers l Primary Producers (PP) l Initial source of data l Data published by user code stored by Producer Service l Secondary Producers (SP) l Used to republish data to: l l PP PP PP SP co-locate information to speed up queries reduce network traffic

The Consumer Service l l Query issued by user code Consumer Service carries out all of the work on its behalf Key control data API Consumer application query Consumer Service

The Consumer Service l l Registry Service contacted to identify relevant producers Mediator works out list of producers to answer query Key Registry Service control data Mediator query API Consumer application query list of producers Consumer Service

The Consumer Service l Consumer Service contacts relevant producers Key Registry Service control data Mediator query API Consumer application query list of producers Consumer Service query Producer Service

The Consumer Service l Data transferred directly from Producer Services to the Consumer Service Key Registry Service control data Mediator query API Consumer application query tuples list of producers Consumer Service query tuples Producer Service

The Consumer Service l Query types: l Continuous queries l l History Queries l l as soon as new data becomes available it is broadcast to all interested parties return time sequenced data Latest Queries l correspond to intuitive idea of current information

Using R-GMA l Three steps to Producing l l Three Steps to Consuming l l Create Producer Declare table Insert data Create a Consumer with your query Start it Read data More information l JRA 1 -UK web site: http: //hepunx. rl. ac. uk/egee/jra 1 -uk/

R-GMA Monitoring Framework

Problems with Monitoring Tools l Inconsistent site configuration sources l l Results in different site coverage Difficult to correlate test results l l l Different site Identification methods Search through many web pages Time consuming especially with 100’s sites!

Unified Monitoring l Site configurations are consistent l l Only from GOCDB Data is sent to single data transport (RGMA) l l l Shared data schema Shared client applications (UI, Alarm Sys. ) Data can be selectively collected to create custom view (CIC, ROC, site admin) Application can share results and build dependencies Application can more easily be distributed using R-GMA to aggregate the data

Unified Schema l Test. Def l l Test are registered here Provides information describing each test l l test. Name, friendly. Name, is. Virtual data. Type, Unit test. Help, test. Title Test. Def. Relation l Defines logical test groups and organize tests l l super. Test. Name, sub. Test. Name display. Priority, flavour

Unified Schema (cont. ) l Test. Data l Holds data produced by test l l l test. Name node. Name summary. Data detailed. Data Status § l 0: NA, 10: OK, 20: INFO, etc. Test. Data. Relation l Defines test data dependencies l l super. Test. Name, super. Node. Name sub. Test. Name, sub. Node. Name

Monitoring Console

Event Management System (EMS)

Introduction to EMS l l Goal of Operation Centers to maximize availability of Grid services EMS will help facilitate this by providing: l l immediate notification as faults and performance degradation is identified Shared EMS services that enables l l Correlation and analysis of historical events of disparate monitoring tools Reduce redundant development of notification modules for Grid monitoring applications

EMS Prototype l l Web Services event transport and command client developed as a proof of concept Individual monitoring application send events directly to a centralized EMS Core Service. Client applications can then query the EMS Core for new events Events are stored in an XML format and filtered using XPath expressions

Migration to R-GMA Transport l l Already fully deployed in LCG 2 R-GMA benefits l l l No additional work required by applications using RGMA Monitoring Framework Built-in data archiving Automatic timestamps Flexible SQL queries Planned features l l Security Fully redundant directory and schema service

Event Management System

EMS Java Client

Additional Details and Help Information

Sorting by Columns

What Lies Ahead l l Not ready for operations use yet Display l l l Event reduction l l l Based on test dependencies Damping of flapping events Notification l l l Filtering mechanism to customize event received Display list of current active alarms Email Audio Communication l Broadcast comments for a particular event

Questions? l R-GMA Monitoring Framework l http: //goc. grid. sinica. edu. tw/gocwiki/Rgma. Unified. Monitoring. System l Piotr Nyczyk Judit Novak Andrey Kiryanov Dave Kant l l EMS l http: //goc. grid. sinica. edu. tw/gocwiki/Event. Management. System l Antun Gusev Doug Chen Mark Ho Jeng-Hsueh Wu l l l