SPI Software Process Infrastructure Quality Assurance QA http

SPI Software Process & Infrastructure Quality Assurance (QA) http: //spi. cern. ch/qa QA reports generation - 22 June 2005 Johanne BENARD johanne. benard@cern. ch J. Benard SPI - Software Process & Infrastructure

QA reports generation Test coverage reports • Basics • Some examples • Other utilities Savannah Quality Assurance (QA) reports • The web interface • Capabilities • Some examples J. Benard SPI - Software Process & Infrastructure 2

Test coverage reports - Basics • Aim at: • Showing what percentage of an application has been executed by the test process, • Measuring effectiveness of a system test. • Two different principles in GCOV: Statement coverage analysis: • Code break down into basic blocs, • Block solicitations counting J. Benard Branch coverage analysis: • Conditional possible paths • Path usage frequency SPI - Software Process & Infrastructure 3

Test coverage reports - Basics Code instrumentalisation: LTP GCOV Suite (*): lcov, genhtml, etc. -fprofile-arcs -ftest-coverage Compilation time File. bb File. cxx, . c , . h, etc. Runtime Coverage analysis File. da File. info File. bbg (*) LTP: Linux Test Project LCOV is an extension of GCOV, a GNU tool which provides information about what parts of a program are actually executed while running a particular test case. J. Benard SPI - Software Process & Infrastructure 4

Some examples - POOL J. Benard SPI - Software Process & Infrastructure 5

Some examples - POOL J. Benard SPI - Software Process & Infrastructure 6

Some examples - CLHEP J. Benard SPI - Software Process & Infrastructure 7

Some examples - PI J. Benard SPI - Software Process & Infrastructure 8

Some examples - SEAL J. Benard SPI - Software Process & Infrastructure 9

Some examples - ROOT J. Benard SPI - Software Process & Infrastructure 10

Test Coverage - Other utilities Tools • Genhtml: • Creates an HTML view from coverage data found in INFOFILE • Geninfo: • Converts. da files into. info files • Genpgn: • Creates an overview image for a given source code file of either plain text or. gcov file format • Gendesc: • Converts a test case description file into a format as understood by genhtml INPUTFILE J. Benard SPI - Software Process & Infrastructure 11

QA Savannah Reports J. Benard SPI - Software Process & Infrastructure 12

QA Savannah report – Web Form Bugs, Tasks, Support and Patches Default: Project beginning Today. ini file: header, Style sheet and options Category, Assigned To Severity, etc. J. Benard SPI - Software Process & Infrastructure 13

QA Savannah report – Web Form • Potential users: projects or experiments that actively use Savannah trackers (bugs, tasks, support, patch) • Current users: JRA 1 coordination, JRA 1 Middleware, LCG 2 Sites • Ready for LCG projects: - POOL, - PI, - SEAL • Tried on several other projects: - J. Benard Athena, Gaudi, Cobra, Iguana SPI - Software Process & Infrastructure 14

QA Savannah report - Capabilities Options used Global processing Breakdown Into Savannah fields J. Benard SPI - Software Process & Infrastructure 15

QA Savannah report - Capabilities Statistics on a given period Translated into graph J. Benard SPI - Software Process & Infrastructure 16

Some examples – LCG 2 Sites J. Benard SPI - Software Process & Infrastructure 17

Some examples – JRA 1 Middleware J. Benard SPI - Software Process & Infrastructure 18

Some examples - POOL J. Benard SPI - Software Process & Infrastructure 19

Some examples - SEAL J. Benard SPI - Software Process & Infrastructure 20

Some example - PI J. Benard SPI - Software Process & Infrastructure 21

SPI Quality Assurance J. Benard SPI - Software Process & Infrastructure 22

Web site: http: //spi. cern. ch/qa J. Benard SPI - Software Process & Infrastructure 23

Conclusion • Proposed services: • To generate customized test coverage reports • To set parameters for Savannah QA reports • To repeat this talk for projects / experiments Thank you for your attention! Question? J. Benard SPI - Software Process & Infrastructure 24