Software Review panel ALICE Process Planning Training Milestones

Software Review panel ALICE Process, Planning, Training, Milestones René Brun, Federico Carminati, Andreas Morsch for the ALICE collaboration 6/5/99 Alice Offline Meeting

A brief history New software organisation since June 1998 Adoption of a unique framework (Sep 98) Software before June 1998 the galice package (Geant 3, Zebra, PAW) Creation of Ali. Root (Nov 98) Online & Offline use ROOT (99) no coordination Very fast transition to C++/OO between detectors All detectors included MDCs, Physics Performance Report (00/01) ALICE 13/3/2000 Software Panel Computing Review 2

Some numbers (1) u Ali. Root u u Rapidly growing more than 200 classes More than 100, 000 lines in C++ +70, 000 automatically generated by rootcint About 5000 lines in f 77 About to disappear u Geant 3 u u 130, 000 lines f 77 (Gean. T, Gean. G, Gean. H) 10, 000 lines f 77 (mini Zebra) Wrapped in class TGeant 3 ALICE 13/3/2000 Software Panel Computing Review 3

Some numbers (2) u Geant 4 u u GGG classes 500, 000 lines C++ u Root u u 480 classes 600, 000 lines C++ u FLUKA u 700, 000 lines f 77 u +Event u ALICE 13/3/2000 ALICE sotfware Framework Services Experiment Independent generators 100, 000 lines f 77 Software Panel Computing Review 4

Which elements of the computing and software organisation participate, and interact, to effect the design and development of the software ? u Central u u team at CERN 4 staff + 3 PA + 2 fellow + 1 VSNA infrastructure framework development DAQ interface u Decomposition u u ALICE 13/3/2000 per detector simulation and reconstruction cooperation between labs for major detectors Software Panel Computing Review 5

Software project organisation Representatives from Alice Projects + Physics Board Representatives from Institutions Alice Offline Project Representatives from major Software Work-packages ALICE 13/3/2000 Software Panel Computing Review 6

Which parts of the software will physicists write and which parts software engineers ? u Present situation u not many software engineers in Alice, mainly as fellows and cooperants. u Most software has been written by physicists (90 per cent) u About 250 physicists could (intend) to contribute. u Currently more than 200 subscribed to the Offline mailing list u More than 50 actively involved in the process today u Future u evolution Computer scientists involvement in databases, graphics. . . ALICE 13/3/2000 Software Panel Computing Review 7

How do you stimulate and control contributions from authors spread worldwide ? u u One Single and Modular framework u persistency, containers u histogramming services u UI, GUI, 2 -d, 3 -d graphics u interpreter + dynamic compilation and linking u same for interactive and batch u coding rules Fundamental and successful decision Regular collaboration-wide software meetings Weekly meetings with phone conferencing Dedicated detector specific meetings ALICE 13/3/2000 Software Panel Computing Review 8

How do you stimulate and control contributions from authors spread worldwide ? u u u Provide mature and productive framework and clear development plans Training u on the framework u detector specific Extensive web based documentation u u u ALICE 13/3/2000 including pages from detector group Central CVS server Coding conventions + checking tool Planning code reviews Physics Performance Report coming in 2001 is a challenging milestone Software Panel Computing Review 9

http: //Ali. Soft. cern. ch/offline/ ALICE 13/3/2000 Software Panel Computing Review 10

What mechanisms are you using to assure that the core infrastructure components of the software have broad experiment input, validation and testing? u While the current software evolves towards its final form, all groups are involved and motivated to come with comments and suggestions. u All Alice teams co-operating to this single framework. u Detector representatives are involved in the building of detectors. ALICE 13/3/2000 Software Panel Computing Review 11

How does the interface with physics groups work? How is the responsibility divided between Software and Computing groups and Physics groups? For physics algorithms, physics object definition and identification? u u Software developers are a subset of the physics group Close collaboration between physics, technical and computing board Physics algorithms and objects are defined by discussions in the offline group and offline computing board No complicated hierarchy u u easier to take decisions strong integration ALICE 13/3/2000 Software Panel Computing Review 12

What design methodology and design process does the experiment use ? Why? How well does this work ? u. A growing and encouraged use of design techniques u UML u ROOT tools for reverse engineering u Joint project with IRST Trento for coding convention checking and reverse engineering Ali. Root-aware tools u Combination of macro/micro cycles u Mainly play->try->redesign u Rapid Prototyping u Motivation ALICE 13/3/2000 for high quality software Software Panel Computing Review 13

What design methodology and design process does the experiment use ? Why? How well does this work ? u Tools to show class inheritance in a working program, also class relationships. u Code checking and reverse engineering tools gradually being introduced u Automatically generated browsable code documentation u with embedded UML diagrams, HTML and graphics This process has proved to work very well ALICE 13/3/2000 Software Panel Computing Review 14

Code Checker example First pass on Ali. Root modules ALICE 13/3/2000 Software Panel Computing Review 15

ROOT Quality assurance More about this was presented at the ROOT workshop ALICE 13/3/2000 Software Panel Computing Review 16

How did you arrive at your workplan, or work breakdown structure? What planning process has taken place to map out the work to be done in the next two years? Who is responsible for the work breakdown structure and for keeping it up to date? u Pragmatic u u Immediate migration to C++/OO Discourage Fortran/Zebra/PAW. Adoption of a unique framework: ROOT Rapid prototyping u Work u u u approach: breakdown is per subdetector (simulation, part of reconstruction) central tasks (framework, global reconstruction) specific workpackages assigned to interested groups (visualisation, geometry database …) u The idea is to grow the current prototype gradually into a production system. ALICE 13/3/2000 Software Panel Computing Review 17

How have you arrived at your workplan, or work breakdown structure? What planning process has taken place to map out the work to be done in the next two years? Who is responsible for the work breakdown structure and for keeping it up to date? u Important milestones are the Physics Performance Reports foreseen for 2001. u u u Full detailed simulation for June 2000 Full sub-detector reconstruction for June 2000 Global reconstruction for fall 2000 u Mock Data Challenges scheduled at regular intervals to monitor the progress. u u u MDC 1 April 1999 (7 TBytes) MDC 2 March 2000 (100 Mb/s + LV 3) MDC 3 planned fall 2000 (remote centres involvement) u Offline ALICE 13/3/2000 board + central co-ordination at CERN Software Panel Computing Review 18

What are your milestones in the area of development and how do these interact with other experiment milestones? How will you measure the success of your milestones and evaluate progress in carrying out the work plan ? u The Ali. Root software chain has been effective already for the TDRs u Ali. Root has been used to take important decisions about the layout of the detector. The effects of various detector configurations to important parameters, like the vertex resolution, the tracking efficiency and the track resolution, have been studied, balancing physical and economical issues u Online and Offline both make use of ROOT u As a framework for Off-line u As a visualisation and monitoring toolkit for On-line ALICE 13/3/2000 Software Panel Computing Review 19

What are your milestones in the area of development and how do these interact with other experiment milestones? How will you measure the success of your milestones and evaluate progress in carrying out the work plan ? u Online u u u and Offline share several key projects ALICE mass storage ALICE Mock Data Challenges Euro. Store II (if approved) u Milestones are discussed and reviewed in the Offline computing board and in the technical board. u No interest in toy milestones u No software milestones per se ALICE 13/3/2000 Software Panel Computing Review 20

Which areas of planning and work are the highest areas of risk, in that lateness or poor quality will have far reaching effects? What is being done to mitigate these risks ? u u u u Manpower. ROOT medium/long term support. G 4: worries about Physics, intermediate and low energy hadronic interactions. Fluka integration: medium long term support Coherent handling of geometry between simulation and reconstruction. HSM/MSS Design and deployment of the worldwide computing model. Alice is specific for the huge amount of raw data. ALICE 13/3/2000 Software Panel Computing Review 21

What is being done to mitigate these risks ? u Manpower. u u u Take advantage from a mature and working framework to reduce the start time of newcomers Use temporary personnel Decentralise development to collaboration institutes Keep at CERN: framework development, MSS prototyping and co-ordination collaboration contracts with external Software providers u ROOT u u medium/long term support. Repeated requests to CERN management Take advantage of the large acceptance of Root in a large number of experiments (cross-fertilisation and OS) ALICE 13/3/2000 Software Panel Computing Review 22

What is being done to mitigate these risks ? u G 4: worries about Physics, intermediate and low energy hadronic interactions. u Integration of G 3, G 4 and Fluka (planned) in our framework u Comparison between the 3 MCs and test beams data (planned) u Fluka u support CERN management has promised CERN support u Coherent handling of geometry between simulation and reconstruction. u Opportunity for a common development u but a decision must be taken soon ALICE 13/3/2000 Software Panel Computing Review 23

What is being done to mitigate these risks ? u HSM/MSS u Development of the Alice mass storage project u Evaluation of CASTOR u Participation in Eurostore II u Design and deployment of the worldwide computing model. Alice is specific for the huge amount of raw data. u Participation in MONARC u Participation in Euro. Grid ALICE 13/3/2000 Software Panel Computing Review 24

What is the plan for training? What are the various types of training required -design? Use of tools? C++, other? What have you learned so far about the successes and failures of training programs and what do you intend to do in the next two years? Do you expect that any of this be in common with other experiments? What role do you expect CERN IT to play? u Training courses on Ali. Root, the Alice framework. u u u Typically one training course during Alice software week At CERN and outside: GSI, Heildelberg including courses on ROOT Typically 20 attendants per course (in CERN training room) These one day courses have been highly successful. We should do more ALICE 13/3/2000 Software Panel Computing Review 25

What is the plan for training? What are the various types of training required -design? Use of tools? C++, other? What have you learned so far about the successes and failures of training programs and what do you intend to do in the next two years? Do you expect that any of this be in common with other experiments? What role do you expect CERN IT to play? u C++ & OO courses (CERN or home institutions) u u These courses FOLLOW and not precede the training on the framework IT should be more active in C++/OO training and support u u Too much focus on Java in IT More help on C++/Java integration ALICE 13/3/2000 Software Panel Computing Review 26

How will technology choices for languages, tools, database products, etc. be made? What provisions are being made for rapidly changing technology? u u Privilege HEP/industry main-stream C++ is the chosen language u restrict features of C++ to facilitate co-operation with some parts possibly written in Java u Root I/O as basic data base in combination with an RDBMS system (My. SQL, Oracle? ) for the run catalogue. u Extensive comparison Root vs Objectivity u Intensive testing/prototyping via the ALICE MDCs ALICE 13/3/2000 Software Panel Computing Review 27

What plans do you have for the long-term support of your software? u Our basic framework (ROOT) is an Open Source project. u u We need an in house team Problem well understood at FNAL (3 staff devoted to local ROOT support) u Expect to take advantage of experience and developments in other experiments u u ALICE 13/3/2000 Graphics Geometry packages and data base Physics algorithms Parallelism Software Panel Computing Review 28

What plans do you have for the long-term support of your software? u Support u for ALICE specific software See next slide and manpower planning u Support for non ALICE specific framework u Large number of experiments (~30) using ROOT worldwide ensures open-source style support u Request official support of Root at CERN ALICE 13/3/2000 u The large number of users should be an additional motivation for this! u Can be a small team (3 staffs) u But has urgently to be put in place Software Panel Computing Review 29

What quality assurance and control mechanisms are being put in place, and in which stages of the design, implementation and testing processes? All testbeds with production code u Mock Data Challenges essential u Periodic Code reviews planned u Code checking tools (from IRST Trento) u Code systematically reviewed by senior scientists u As much as possible documentation in the code with automatic html generation u u Enforce documentation with code checking tools u html and pictures in the C++ code directly Require portability on different platforms u currently : Linux, Solaris, HP-UX, Alpha/Unix ALICE 13/3/2000 Software Panel Computing Review 30

What decisions on software technology and implementation choices have to be taken in the future and when do you plan to take them? u u u Major choices already done in 1998 u Language u Framework u Event database and object store u Data analysis package Run Catalogue and conditions data base u consensus on RDBMS u follow experience of other experiments(STAR, FNAL) u My. SQL prototype in MDCs (choice in 2001/2002) Geometry data base u ALICE 13/3/2000 XML ? Common with other experiments ? Software Panel Computing Review 31

What decisions on software technology and implementation choices have to be taken in the future and when do you plan to take them? u Decision on MSS/HSM u Eurostore II , CASTOR, Inter. Stage? u Prototyping via ALICE Mock Data Challenges u Decision in 2003 u Understanding access patterns u ratio tape/disk u analysis patterns u Event by Event and conventional multi-event ALICE 13/3/2000 Software Panel Computing Review 32

What decisions on software technology and implementation choices have to be taken in the future and when do you plan to take them? u Simulation packages u what is the role of fast simulation u when and if replace GEANT 3 with GEANT 4 u FLUKA will be integrated in the framework and kept for ALICE 13/3/2000 u comparison with and complement to GEANT 4 u having two options for full detector simulation in the same framework and with common geometry, kinematics and hits u detailed detector understanding u radiation studies Software Panel Computing Review 33

What decisions on software technology and implementation choices have to be taken in the future and when do you plan to take them? u WAN data access u LAN and WAN data access tools u We assume a constant network latency and a rapidly growing bandwidth excluding object-level access (the DREAM=illusion) u rootd a possible choice u want to follow/understand GRID middleware u decision to be taken in the scope of MONARC, GRID projects ALICE 13/3/2000 Software Panel Computing Review 34

What decisions on software technology and implementation choices have to be taken in the future and when do you plan to take them? u Parallelism u batch job level parallelism assumed to be solved by LSF/Condor (or like) evolution u Interactive data analysis requires a tight integration between the analysis tools and the data bases u Prototyping based on PROOF (Parallel Root Facility), possibly in the framework of the GRID project u Important to understand the implication on the analysis model and tools (prototyping necessary) u Final choice in 2003 ALICE 13/3/2000 Software Panel Computing Review 35

What decisions on software technology and implementation choices have to be taken in the future and when do you plan to take them? u Role of Java u C++ is our language. We invest on our decisions. u It would be unfair to impose a change in the short/medium term u Timescale & priority depending on the perceived role of Java in and outside HEP u Worries about numerical precision and standardisation process (http: //www. cs. berkeley. edu/~wkahan/JAVAhurt. pdf) u We expect complementarity with C++, not an alternative u Our Object model is simple (no templates) u It cannot be an obstacle to a migration u Already compliant with Java (object base class, containers) ALICE 13/3/2000 Software Panel Computing Review 36

What decisions on software technology and implementation choices have to be taken in the future and when do you plan to take them? u Role of Java u step 1: Transparent access to C++/Root services via web applets or JRE. (In development with FNAL) u step 2: Exchange of objects (data objects only) between C++, Java. Easy to do via the CINT RTTI. Could be very quickly implemented. u Step 3: Full Java integration. Implement C++ methods on the Java classes where it makes sense ALICE 13/3/2000 Software Panel Computing Review 37

What decisions on software technology and implementation choices have to be taken in the future and when do you plan to take them? u Possible role of XML u A possible role for detector geometry u Watch developments in other collaborations u Exchange format for low-volume objects u Documentation u Electronic Logbook u ROOT & XML u Trivial for ROOT to generate XML output for objects u Replace ROOT HTML with ROOT XML documentation ALICE 13/3/2000 Software Panel Computing Review 38

How will it evolve? How will it meet demands for parallel processing? distributed processing? security and authentication needs? Language evolution? u Parallel processing will be vital. Thinking to use an evolution of PROOF and rootd. u u A state of the art authentication package is part of ROOT (SRP from Stanford). u u WAN prototyping with MONARC and Grids Possible role for the Grid software It is essential to support multi-threading and asynchronous I/O u Thread package developed jointly with the GSI GO 4 project u Developments in Root I/O to support multi-threading. ALICE 13/3/2000 Software Panel Computing Review 39

How do you plan to provide working software and do development at the same time? How do you plan to transition from existing software to final production software? u Open Software spirit u Micro/Macro u We development cycles gradually develop the production system. u At each step, the software is used by the community in production for the day- to- day work. u Not simply a prototype with the idea to start from scratch in two years. u We exploit the power of abstract interfaces u We make sure that enough modularity is built in our system. ALICE 13/3/2000 Software Panel Computing Review 40

Manpower Support for ALICE specific software u u No chance to have enough people at CERN to develop the majority of the code of ALICE, as it was the case for LEP. Planning based on maximum decentralisation. u u u All detector specific software developed at outside institutes Around 40 people directly involved today CERN Off-line team responsible for u u central coordination, software distribution and framework development prototyping, (in particular of MSS in collaboration with IT) ALICE 13/3/2000 Software Panel Computing Review 41

Manpower Support for ALICE specific software u Fulfill this role with Project Associates and Fellows u u u ALICE 13/3/2000 main worry is continuity -- basic support and coordination more difficult with short-term resident staff more attention to the project structure and documentation New people should get acquainted quickly with the structure and be effective in a short time. For this we have setup a web site with all the information on the code and the project structure. . . Software Panel Computing Review 42

Activities ALICE 13/3/2000 Software Panel Computing Review 43

ALICE CERN man-power evolution Needed FTEs Current Staff ALICE 13/3/2000 Software Panel Computing Review 44