Adaptive Component Based System Architecture for Monitoring Data

Adaptive, Component Based System Architecture for Monitoring Data Storing Distributed Systems Research Group Department of Computer Science AGH-UST Cracow, Poland Dominik Radziszowski, Krzysztof Zieliński [radzisz, kz]@ics. agh. edu. pl

Adaptive, Component Based System Architecture for Monitoring Data Storing DSRG The need and the problems • The need: More and more resources which must be monitored and monitoring data which must be stored. • Current solutions for Grid: • MDS, EDG NMA, Ganglia, Nagios, Map. Center, SNMP, JIMS • The problems: – cannot exchange data and store in one common database (lack of common information model), or does not support monitoring data storing at all, – are not adaptable to: • different monitoring environments, • various resources types, • possibility of dynamic resource attachment and dynamic attributes changes, • different monitoring modes. – monitoring data storing processes is not scalable/efficient enough to store massive data. Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński

Adaptive, Component Based System Architecture for Monitoring Data Storing DSRG Wishes list - functionality • • Heterogeneity of monitored resources, Dynamic resource attachment, Runtime modification (extension, removal) of attributes, Support for compound attributes and multi-valued attributes, Different monitoring modes (push, pull, tracing), Universal interfaces for data upload and query, A new system Monitoring domain independency, have to be created Make use of existing monitoring agents. Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński

Adaptive, Component Based System Architecture for Monitoring Data Storing DSRG Wishes list – other features • • High availability, Load balancing, Automatic failover, Scalability, Efficiency, Nodes heterogeneity, Dynamic reconfiguration. Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński Can be meet, Let’s use component architecture!

Adaptive, Component Based System Architecture for Monitoring Data Storing DSRG Usher – general architecture Usher – uniform storage for heterogeneous environment monitoring Problems: • common data model, • agent, • universal access (upload and query) interfaces. Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński

Adaptive, Component Based System Architecture for Monitoring Data Storing DSRG Common object data model META DATA Type, Resource, Attribute: – Simple – Structured – Multi-valued Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński Data Meta Data DATA Values: - String - Double - Long

Adaptive, Component Based System Architecture for Monitoring Data Storing DSRG Data upload conceptions 1. Values are being send together with all meta-information (eg. Ganglia) Big overhead for simple data 2. Separated mode - at the beginning meta information is configured - data are being send with proper attribute identifier only Low overhead, possible detection of meta data inconsistency at the agent side Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński

Adaptive, Component Based System Architecture for Monitoring Data Storing DSRG Data upload interface Operations sequence: • Configure meta data: – get. Resource() • If resource does not exist in the system create it with add. Resource() – get. Root. Attributes. For. Resource() • If resource exposes attributes from outside the list, add attribute with add. Root. Attribute() or extend structure attribute with extend. Attribute() • Upload monitored values – periodically call add. Monitoring. Data() Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński

Adaptive, Component Based System Architecture for Monitoring Data Storing DSRG Usher agent Agent responsibility: • interaction with a resource or an external monitoring agent • meta data structure setup in USHER • data conversion and periodical storing in USHER system • resource’s attributes changes tracing and appropriate meta data structure modifications Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński

Adaptive, Component Based System Architecture for Monitoring Data Storing DSRG Data query conception Copyright

Adaptive, Component Based System Architecture for Monitoring Data Storing DSRG Data query interface Operations sequence: 1. 2. 3. 4. call get. Root. DTO() to obtain root object of the meta data tree, mark nodes to be extended, call get. Subcomponents(), repeate 2, 3 until you will have desired simple. Attributes in leaves – metadata tree is completed, 5. call get. Data() to obtain values of the requested attributes according to specified query. Clauses (time, number or results etc. ). Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński

Adaptive, Component Based System Architecture for Monitoring Data Storing DSRG Component implementation Implementation technology details: – component technology: Java, J 2 EE, – application server: JBoss, BEA Web. Logic (can be any), – data base: Oracle, Postgre. SQL (can by any with JDBC support). Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński

Adaptive, Component Based System Architecture for Monitoring Data Storing DSRG Conclusions • Proposed architecture is based on: – conception meta data for monitoring data, – universal access interfaces, – component technology for non functional requirements meeting. • USHER advantages: – integrates data from different monitoring agents, – is adaptative to different monitoring environments, various resources types, possibility of dynamic resource attachment, dynamic attributes changes and different monitoring modes. • Further work – non functional characteristic need to be experimentally verified (lots of work has been made in this area but it is not covered in this presentation), – usher agents have to be build for various monitoring systems (ongoing integration with JIMs and WBEM). Copyright © 2006 Dominik Radziszowski, Krzysztof Zieliński

Thank you for your attention Questions please. .