The Quality Assurance Metric Infrastructure in the EMI

The Quality Assurance Metric Infrastructure in the EMI Project Eamonn Kenny, TCD Manchester, 2013 -04 -10 EMI is partially funded by the European Commission under Grant Agreement RI-261611

EMI Quality Assurance Activities • Quality Assurance Policies – definition and monitoring • Metrics and KPIs – definition and collection • Quality Control – verification and reporting • Build/Test Tools and Repositories – selection, maintenance and integration – setup, maintenance and deployment 29/03/2012 TCD 2 EMI INFSO-RI-261611 • Certification and Integration Testbeds

Description Multiple Institutions Company/Large Organisation Programming Languages Many Few or one Bug Trackers Many One Feature Trackers Many One (may be the bug-tracker) Unit testing Variety of analysers Discrete, well defined, run over full code base Build systems Many One You cannot apply all of SQua. RE ISO 25000 to a working well established system 29/03/2012 TCD 3 EMI INFSO-RI-261611 What’s different in EMI?

Trackers Format Metrics Savannah RT 29/03/2012 TCD 4 EMI INFSO-RI-261611 Metrics

Request for Change (Rf. C) lifecycle Open https: //twiki. cern. ch/twiki/bin/view/EMITracker. Mappings Accept? YES Accepted NO NO Rejected • Won’t Fix • Duplicate • Invalid Fixed YES Not Tested Successful? Hiding and managing the complexity YES Tested Closed 29/03/2012 TCD 5 EMI INFSO-RI-261611 • Integration Test? Candidate • Fixed NO • Resolved

Implemented Framework QA Validation Policy emi-rfc. cern. ch Tabulated Formats Dashboard Pull CSV /XSL emiqa. web. cern. ch Tracker to Product Mappings 29/03/2012 Manual Validation Quality Control TCD Release Manager 6 EMI INFSO-RI-261611 Chart Generator (Java Framework)

29/03/2012 TCD 7 EMI INFSO-RI-261611 Dashboard for Validation/Reports

Framework Actors Bug/Feature Trackers Common formats Common Format Product Teams Common Format Dump/Export of Rf. C Tracker Common Format QA Metrics Common Format Exported files in Common Format Tabulated Formats dashboard QA Metrics Other formats Multiple Builds Product Teams Common Format QA Tools Common Format Common. Dated Format Builds Results with Static Analysers Verification & Tests Per Product Calculations Releases Dashboard Trend Graphs Product information QA Metrics & Tools QA Policy & Testing + Release Manager Outputs XY Bar/Stacked Charts Stacked, Trend Graphs, Distributions 29/03/2012 TCD Release Manager, QA, QC 8 EMI INFSO-RI-261611 Metric Calculation & Visualisation

Java Framework Implementation Build System XML Metrics Associated Tests per Update SLOC per Update (single and trend) Rf. C/Tracker XML Trend Graphs Deployment KPIs RPMlint/Lintian tables 29/03/2012 Stacked Bar Charts Defects per SLOC Development task Vertification/Testing XML Bar Charts Associated Tests per Release + Updates TCD Line plots 9 EMI INFSO-RI-261611 Data Plotting Tools

29/03/2012 TCD 10 EMI INFSO-RI-261611 Sample Results

EMI Year 3 work summary 29/03/2012 TCD 11 EMI INFSO-RI-261611 • New dashboard layout and fixes • Combined static analyser reporting • Density plots for quality control • Git. Hub/EMI product team integration • Ongoing release/update metrics

Questions? EMI INFSO-RI-261611 Any Questions?