GRIDVIEW A Visualization Tool for LCG Rajesh Kalmady

GRIDVIEW A Visualization Tool for LCG Rajesh Kalmady, Phool Chand, Kislay Bhatt, D. D. Sonvane, Kumar Vaibhav (BARC-Mumbai) 9/18/2020 BARC-CERN/LCG Meeting

Gridview : Work Areas • • • Gridftp File Transfer Monitoring Service Availability Monitoring Job Monitoring GOCDB Replication and Synchronization Enhancement of RGMA Archiver module Gridview on New Machines 9/18/2020 BARC-CERN/LCG Meeting

Gridftp File Transfer Monitoring • New Features: – Report page displaying Latest Transfers (used for configuring FTS channels) – Summarization of current Hour • Summarization Module now – Filters duplicates reported by both source and dest. – More Optimized Database Access as suggested by DB group • Support during SC 3 Rerun in Jan and SC 4 • Use of Gridview for monitoring non-SC tier-1 to tier-2 transfers – From RAL to UKI T 2 sites – From PIC Tier-1 to Tier-2 sites 9/18/2020 BARC-CERN/LCG Meeting

Gridview in SC 4 9/18/2020 BARC-CERN/LCG Meeting

Service Availability Monitoring Environment (SAME) • New Data Transport Mode for archiving SAME data using Web Service and custom data serialization • A fully optimized transport and storage schema for sensor data and summary data tables for SAME • Visualization of SAME data is in progress 9/18/2020 BARC-CERN/LCG Meeting

Job Monitoring Major Features of Job Monitoring in Gridview • • Visualize jobs count graph per VO, RB or Site. Jobs count graphs for combinations of above. State-wise details of jobs. Parameters like Job Success rate, Resource utilization, Job Turnaround time, RB waiting time etc. • Reports of above Parameters for VO, RB, Site 9/18/2020 BARC-CERN/LCG Meeting

Job Monitoring in Gridview 9/18/2020 BARC-CERN/LCG Meeting

Job Monitoring… 9/18/2020 BARC-CERN/LCG Meeting (cond. )

GOCDB Replication & Synchronization • Gridview and SAME requires data of Grid sites, Regions, Nodes, Services, VOs, Maintenance Schedules, etc centrally maintained by GOCDB. • Development of GOCDB Synchronization Module maintains an updated local Replica of required GOCDB tables in Gridview Database. 9/18/2020 BARC-CERN/LCG Meeting

Enhancements to R-GMA Archiver Module • Some newly discovered problems regarding creation of consumers are fixed • Archiver is made more configurable to setup for multiple tables • Archiver uses latest API • Use of secure connections 9/18/2020 BARC-CERN/LCG Meeting

Gridview on New Machines • New machines grvw 00[123] for Gridview • Shifted production instance to grvw 001 (frontend) and grvw 002 (backend) • Shifted development instance to grvw 003 9/18/2020 BARC-CERN/LCG Meeting

Current Status of Gridview Development • Gridftp File Transfer Monitoring – – – Design, Development is Over Fully functional system is Implementation and Deployed Initial problems due to RGMA are resolved Used successfully in SC 3 and SC 4 Features like latest transfers and current hour summary are added after users Feedback – Support for the system • Service Availability Monitoring – Design of the modules for SAME is done. – Archiver module using web services is implemented after noticing problems with RGMA – Summarization module with database schema is implemented – Development of Visualization Module is in progress. 9/18/2020 BARC-CERN/LCG Meeting

Current Status … (Contd. ) • Job Monitoring – – – Design of Job Monitoring module is completed Archiving module is developed and deployed Design of database schema is over Implementation of database is in progress Summarization module for hourly and daily report for job status are implemented. – Implementation of Full Summarization module is going on – Implementation of Visualization module is in progress • GOCDB Replication and Synchronization – GOCDB synchronization module is developed and deployed. 9/18/2020 BARC-CERN/LCG Meeting

Grid. View: Problems • No clear rules to tie hosts to sites – Hostnames for many sites do not conform to GOCDB registered domains • Unregistered VOs in Collected Job Status Data 9/18/2020 BARC-CERN/LCG Meeting

Future Work / New Requirements • Visualizing FTS File transfer failure rates on Gridview • Interfacing Gridview with Information System (Top level BDII) 9/18/2020 BARC-CERN/LCG Meeting

Thank You Comments and Suggestions are invited 9/18/2020 BARC-CERN/LCG Meeting