LHCb workloads in the benchmarking suite Andrea Valassi

LHCb workloads in the benchmarking suite Andrea Valassi (IT-DI-LCG) With contributions from and many thanks to: Concezio Bozzi, Ben Couturier, Domenico Giordano Benchmarking WG meeting, 1 st February 2019, CERN https: //indico. cern. ch/event/782598/ A. Valassi – Benchmarking LHCb workloads Benchmarking WG meeting – 1 Feb 2019 1

Goals of this (ongoing) work • Extend and improve Domenico’s benchmarking suite – https: //gitlab. cern. ch/giordano/hep-workloads • 1. Integrate one or more LHCb workloads – Start from the MC workload added by Concezio and Ben – Choice and extraction of relevant metrics – Improve (speed up) current workload and foresee adding new ones • 2. Work with Domenico on the underlying infrastructure – Improve robustness of gitlab CI and docker machinery – Discuss standard ways of presenting metrics – Make it easier to add more than one workload per experiment A. Valassi – Benchmarking LHCb workloads Benchmarking WG meeting – 1 Feb 2019 2

LHCb workload • A typical LHCb “simulation job” – Including both event generation and detector simulation • Extremely easy to run – One command line: docker run -v /tmp/results/: /results gitlabregistry. cern. ch/giordano/hep-workloads/lhcb-gen-sim: buildtest – It takes around 30 minutes to process 5 events • Current metric: score is #events per (wall-time) 1000 seconds – Including both generation and simulation – Including the first event (slower due to I/O) – Foresee to keep a single main score but add several sub-scores A. Valassi – Benchmarking LHCb workloads Benchmarking WG meeting – 1 Feb 2019 3

LHCb workload metrics • Current: parse the total event loop for generation and simulation One of 32 job logs: In this job: average 352 s (= 1760 / 5) per event – i. e. 2. 84 events per 1000 s Benchmarking score: Across 32 jobs: average 2. 85 events per 1000 s – i. e. 91. 1 events per 1000 s node throughput (= 2. 85 x 32) • Future: separately parse generation and simulation, skip the first event Event #3 GEN One of 32 job logs: Event #3 SIM Pile-up Metrics for this job: Event #1 SIM: includes slow initialization (e. g. magnetic field, conditions) A. Valassi – Benchmarking LHCb workloads Comments: - GEN and SIM time ~uncorrelated - SIM time ~proportional to #particles (but first event SIM is slower) - SIM time / particle more stable as a throughput metric than time / event Benchmarking WG meeting – 1 Feb 2019 4

Docker and gitlab CI infrastructure • Workload image build phases – 1. Build a generic workload builder image – 2. Within the builder image, run a workload from the cvmfs service • Use the cvmfs_shrinkwrap utility to note which cvmfs files are needed – 3. Build a workload image with a local /cvmfs including only those files – All these steps are automatically executed within the gitlab CI • Ongoing work – Improve robustness of gitlab CI and docker machinery • Add sanity checks, debugging in a slightly different environment – Eventually: simplify adding more than one workload per experiment • Save cvmfs_shrinkwrap lists from existing workloads, run only new workloads and build images using the union of all needed cvmfs files • Speed up build process (avoid rerunning very long benchmarks) A. Valassi – Benchmarking LHCb workloads Benchmarking WG meeting – 1 Feb 2019 5

Conclusions • An LHCb workload exists and can already be used (in one simple line) – Thanks to Concezio, Ben and Domenico • I am working on various possible improvements – By default it uses all available virtual processors to fill the node CPU • Current score is #events per (wall-time) 1000 seconds – Including both generation and simulation – Including the first event (slower due to I/O) – Foresee to keep a single main score but add several sub-scores • It is relatively easy to add new features to an existing test – Especially if you send Domenico a MR and let his CI do the job! – We are working on making this even easier • LHCb sites (e. g. NCBJ. pl) interested in such benchmarks for tendering A. Valassi – Benchmarking LHCb workloads Benchmarking WG meeting – 1 Feb 2019 6

PS (slide added after the meeting) • Need three types of data in the final json • 1. Metadata – Which test suite was executed and under which conditions • #events, #jobs (copies), #processes per job, #threads per process – Node metadata, including #physical and virtual processors… • 2. Control data – e. g. #events actually processed (as opposed to #events scheduled) – Should make it possible to determine if test suite fully succeeded • 3. Performance scores – One clearly accessible overall “single” score – Many detailed scores (e. g. SIM and GEN, e. g. time for each event…) A. Valassi – Benchmarking LHCb workloads Benchmarking WG meeting – 1 Feb 2019 7