Ali Root and Ali En Build Integration and

Ali. Root and Ali. En Build Integration and Testing System (BITS) Alina Grigoras alina. gabriela. grigoras@cern. ch Olga Vladimirovna Datskova olga. vladimirovna. datskova@cern. ch 1 / 22 http: //alienbuild. cern. ch: 8888 http: //alirootbuild. cern. ch: 8889

BITS purpose 2 / 22 Having a current, up to date, installable binary image of Ali. En and Ali. Root for different architectures Automate the process of building Ali. En and Ali. Root components Introduce a better understanding of the dependencies between packages Perform unit and functional tests during build time, deploying error alerts immediately Provide a web interface that allows monitoring of the current build status and remote access to logs http: //alienbuild. cern. ch: 8888 http: //alirootbuild. cern. ch: 8889

Supported Platforms Ali. En and Ali. Root are supported for the following platforms: SLC 5 32 and 64 bits SLC 5 Itanium Intel. Mac 32 and 64 bits Leopard Intel. Mac 32 and 64 bits Snow Leopard (to be added Ubuntu 9. 10 64 bits (testing purpose) 3 / 22 Ali. En. BITS Web Interface Main Menu http: //alienbuild. cern. ch: 8888 http: //alirootbuild. cern. ch: 8889

Build environment overview Based on the Konstruct framework used to build KDE and dependencies which in itself is based on BSD ports system Each package is defined as a directory in a hierarchy 4 / 22 For each package there is a Makefile containing a set of variables: Name, version, author, web page License information Build and runtime dependencies Master sites for downloading the package source Additional patches needed for Ali. En/Ali. Root environment Checksums for verifying the integrity of the downloaded sources http: //alienbuild. cern. ch: 8888 http: //alirootbuild. cern. ch: 8889

Build process[1] Get the available Releases Fetch the latest changes for this release Analise the dependencies between packages Select the packages that have to be rebuilt 5 / 22 Do a cvs update Determine the build order Select the one to be (re)built Based on modifications received on cvs update Based on last build status Based on package dependencies http: //alienbuild. cern. ch: 8888 http: //alirootbuild. cern. ch: 8889

Build process [2] For each package Clean its environment Fetch source archive from master sites Build, install and run tests Create the binary image for binary installations Generate package web page Build / SLOC / Graph dependencies Cleanup environment Send notification email when status changes 6 / 22 http: //alienbuild. cern. ch: 8888 http: //alirootbuild. cern. ch: 8889

Ali. En: packages graph 7 / 22 http: //alienbuild. cern. ch: 8888 http: //alirootbuild. cern. ch: 8889

Ali. En: available builds http: //alienbuild. cern. ch: 8888 The current build system maintains the following versions of Ali. En across all supported platforms: The build also provides the following Ali. En distributions (the sizes are for v 2 -18 SLC 5 64 bits): 8 / 22 Distribution Packed size (MB) Unpacked size (MB) user 18. 7 71. 1 workernode 44. 8 156. 4 client 62. 2 279. 7 vobox 126. 0 476. 1 all 160. 1 660. 6 http: //alienbuild. cern. ch: 8888 http: //alirootbuild. cern. ch: 8889

Ali. En: Release Web Page 9 / 22 http: //alienbuild. cern. ch: 8888 http: //alirootbuild. cern. ch: 8889

Package Web Page Ali. En Package Details - general information about the package - links to the Makefile + the applied patches Ali. En Package Dependencies - list of packages that have to be build before building this package Ali. En Packages Depending on it - the list of packages that can't be build before building this package 10 / 22 http: //alienbuild. cern. ch: 8888 http: //alirootbuild. cern. ch: 8889

Ali. En: user distribution Latest installation instructions for the user distribution can be found at http: //alien 2. cern. ch wget http: //alien. cern/alien-installer chmod +x alien-installer. /alien-installer [pcepalice 10] /home/alienmaster >. /alien-installer ########### Automated Ali. En user installer ######## Installing in the default directory: /home/alienmaster/alien If you wish to install in a specific directory do: . /alien-installer -install-dir Platform set as: x 86_64 -unknown-linux-gnu Waiting 10 seconds. Press 'Ctrl+c' to exit the installation. . . . Starting installation Downloading user distribution. . . . . Done Extracting the files. . . . . Done Relocating the user installation. . . . Done Installation finished! User distribution contains the necessary libraries for compiling ROOT with Ali. En Grid support. The detailed instructions can be found on the same page as above. 11 / 22 http: //alienbuild. cern. ch: 8888 http: //alirootbuild. cern. ch: 8889

Ali. En Tests – Ali. En Tests are performed after each build • Test the installation procedure • Perform different functional tests • Save the log files for the failed tests • Save the log files of all Ali. En services during the test • Publish relevant logs 12 / 22 http: //alienbuild. cern. ch: 8888 http: //alirootbuild. cern. ch: 8889

Release Test Web Page General information Main test steps and their results Results per test types Logs: - Ali. En services logs - failed tests logs Global log of the build process 13 / 22 http: //alienbuild. cern. ch: 8888 http: //alirootbuild. cern. ch: 8889

Ali. Root: build specifics http: //alirootbuild. cern. ch: 8889 Same build process as Ali. En – Checks for new releases or changes in the current development releases – After each build performs quality and functional tests 14 / 22 http: //alienbuild. cern. ch: 8888 http: //alirootbuild. cern. ch: 8889

Ali. Root: web interface http: //alirootbuild. cern. ch: 8889 15 / 22 http: //alienbuild. cern. ch: 8888 http: //alirootbuild. cern. ch: 8889

Release administration http: //alirootbuild. cern. ch: 8889 Adding a new Ali. Root release new releases can be added using a web interface For each Ali. Root release we need to specify the dependencies – Ali. En version Available version list extracted – Geant 3 version from source repositories – Root version 16 / 22 http: //alienbuild. cern. ch: 8888 http: //alirootbuild. cern. ch: 8889

Ali. Root benchmarks are triggered after each build The results are published on a web page The logs are available from any remote location History charts to compare with previous builds / releases 17 / 22 http: //alienbuild. cern. ch: 8888 http: //alirootbuild. cern. ch: 8889

Ali. Root: on the GRID After a complete build and in case of no errors during the tests, Ali. Root packages can be registered into Ali. En using a web interface from Mon. Alisa 18 / 22 http: //alienbuild. cern. ch: 8888 http: //alirootbuild. cern. ch: 8889

Ali. Root: Geant 4 unlike its predecessor is written in C++ and has a completely new installation procedure The Grid Ali. Root BITS for Geant 4 packages was adjusted in order to facilitate automatic compilation and installation procedures The final Geant 4 package Geant 4 -v 2010 xxxx installed on the GRID (see left diagram) contains a Geant 4_vmc number of dependent packages, libraries and data Data Files files. Geant 4 CLHEP 19 / 22 Build dependency ROOT Installation point http: //alienbuild. cern. ch: 8888 http: //alirootbuild. cern. ch: 8889

Ali. En: current work [1] Perl upgrade from 5. 8. 8 to 5. 10. 1 Always use the latest packages Enabling thread support for improving Ali. En performance Issues Some packages are really old and they are not supported anymore Rebuilding a distribution takes ~a couple of hours thus each iteration is slow Problems are really difficult to spot Packages that don't have a clear building and installation script Because of the big number of packages and dependencies one change in a package can trigger a whole chain of problems Some packages are not supported at all for some platforms, usually for Mac 20 / 22 http: //alienbuild. cern. ch: 8888 http: //alirootbuild. cern. ch: 8889

Ali. En: future plans Adding a g. Lite test environment, without installing g. Lite with Ali. En Issues: installing g. Lite on all build servers independent of the build and testing process, certificates etc Synchronizing the build number between different platforms You can have Ali. En 2 -18. 23 for 32 bits and Ali. En 2 -18. 30 for 64 bits Major cleaning of Ali. En packages (somewhere in the future) 21 / 22 http: //alienbuild. cern. ch: 8888 http: //alirootbuild. cern. ch: 8889

Ali. Root: present and future 22 / 22 Usual work: Building and registering to Ali. En different tags of Ali. Root Current work: Implementing an automatic email alert system If Ali. Root fails, check where it failed and email the responsible persons about it This applies only to the trunk, that is build every night We are in test phase Future plans : Automatically register packages to Ali. En, now it is done using an intermediate Mon. Alisa webpage http: //alienbuild. cern. ch: 8888 http: //alirootbuild. cern. ch: 8889