Ali Root Build and Test System ALICEFAIR Computing

Ali. Root Build and Test System ALICE-FAIR Computing Meeting 28/04/2008 Presented by P. Hristov

History l l l Ali. En build system developed by Catalin Cirstoiu and Predrag Buncic in 2005 First port of Ali. Root: Vagner Morais in 2006 Web interface and monitoring: Costin Grigoras in 2006 Test examples: Peter Hristov in 2006/2007 Final build system: Catalin Cirstoiu in 2007 Support and future development: Alina Grigoras and Marco Meoni 27/04/2008 P. Hristov 2

Goals l l l Automate the process of building Ali. Root Have a current, up to date, installable binary image of Ali. Root for different architectures Better understand the dependencies between the packages Provide software that works to the end-users Reuse the development done for the Ali. En build system (Ali. En. BITS) 27/04/2008 P. Hristov 3

Building System l l l Based on the Konstruct framework used to build KDE and Ali. En. Bits (originally GAR system) Each package has a directory in a hierarchy Each Makefile define a set of variables: l l l l Name, version, author, web page License information Build and runtime dependencies Sites to download the package source Patches Checksums (integrity of the files) The Makefiles are executed recursively 27/04/2008 P. Hristov 4

Building Steps [1] l Get the available Releases l Select the one to be (re)built l Fetch the latest changes for this release l Do a svn update l Determine the build order Analyze the dependencies between packages (Ali. En -> Root -> Geant 3 -> Ali. Root) l l Select the packages that have to be rebuilt l l 27/04/2008 Based on modifications received on svn update Based on last build status Based on package dependencies Always rebuild and test Ali. Root packages P. Hristov 5

Building Steps [2] l For each package l l l Clean its environment Fetch source archive from master sites Generate SLOC statistics Build, install and run tests Create the binary image for binary installations l Generate web pages l Build / SLOC / Graph dependencies l Generate the pages for Ali. Root tests l Cleanup environment l Send notification email when status changes 27/04/2008 P. Hristov 6

Web Pages of the Build System l l The system generates web pages automatically (Build / SLOC / Graph dependencies) Package dependencies: l l l Better and faster understanding of the project To see easily how many packages a certain package brings in Provide statistics on the size of the project: l l 27/04/2008 For each package For package’s dependencies For the whole group Global view P. Hristov 7

The Ali. Root BITS l Ali. Root is being built nightly on all major platforms: l l l i 686, x 86_64, ia 64, Intel. Mac The i 686 build includes SHUTTLE Building is performed only if needed l Ali. Root or one or more dependencies were modified § version or tag or the tag was moved Several Ali. Root cvs tags can be tracked and rebuilt if changed Ali. En packages (Ali. Root/Geant 3) are automatically generated l l l If build is successful, tests are run l l l 27/04/2008 Tests provided in the Ali. Root’s repository Currently: 20 pp, 1 Pb. Pb, sim&rec Monitoring information about the run is collected in ML P. Hristov 8

Ali. Root tests l Both sim/rec tests, for both pp and Pb. Pb send monitoring information to ML l l Using the ROOT ML plugin developed by Andreas Peters Parameters l Cpu/Wall Time, Total/RSS Memory, raw. root and Ali. ESD. root file sizes l l Test success status l 27/04/2008 Sent only if the test succeeds Sent only if Ali. Root is successfully built P. Hristov 9

http: //pcalimonitor. cern. ch: 8889/bits_benchmark. jsp Thanks to Costin Grigoras 27/04/2008 P. Hristov 10

Useful pointers Ali. Root BITS: http: //pcalibuildintel. cern. ch: 8889/globa l_index. html Ali. Root Benchmarks: http: //pcalimonitor. cern. ch/bits_ben chmark. jsp 27/04/2008 P. Hristov 11

Building a new release l Used to involve manual editing of the versions, committing & tagging in BITS cvs l Now requires only few clicks l New tags are discovered automatically 27/04/2008 a P. Hristov 12

Work in progress [1] l l l Transition to SLC 4 Upgrade of the Mac. OSX servers New tests l Backward compatibility l l 27/04/2008 Reconstruction of “reference” RAW samples Use of “reference” OCDB data (calibration/alignment) Code checker and reverse engineering tool More “alarms” in case of failure P. Hristov 13

Work in progress [2] l “Incremental” builds from the SVN trunk l l The trunk rarely passes all the tests (some times it even does not compile) Instead of using one single revision (the latest one) l l l 27/04/2008 Prepare (manually) set of revisions (one per module) that is known to work correctly Revert to the last working revision if a module has a problem Replace the working revision of a module if the version from the trunk is OK P. Hristov 14

Summary l The tool for automatic builds has proven its usefulness in two different cases: Ali. En - hundreds of packages, Ali. Root - 4 packages + Ali. En. It l l l Intelligently selects packages that have to be rebuilt Offers different views for packages dependencies - very useful in understanding and managing large projects Provides SLOC statistics Saves and presents relevant logs and statistics on components testing Has “one click” interface for the build management Building and continuous testing is important for delivering high quality software to end-users 27/04/2008 P. Hristov 15