CMS Software Architecture Software framework services and persistency

CMS Software Architecture Software framework, services and persistency in high level trigger, reconstruction and analysis Vincenzo Innocente CERN/EP/CMC 1/8/2022 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

CMS (offline) Software Environmental data Request part of event Slow Control Quasi-online Reconstruction Online Monitoring store Event Filter Objectivity Formatter Request part of event Store rec-Obj Request part of event store Persistent Object Store Manager Object Database Management System store Simulation G 3 and or G 4 Software Architecture Vincenzo Innocente, CERN/EP Store rec-Obj and calibrations Data Quality Calibrations Group Analysis Request part of event User Analysis on demand 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

Requirements (from the CTP) u. Multiple Environments: u. Various software modules must be able to run in a variety of environments from level 3 triggering, to individual analysis u. Migration between environments: u. Physics modules should move easily from one environment to another (from individual analysis to level 3 triggering) u. Migration u. Should to new technologies: not affect physics software module Software Architecture Vincenzo Innocente, CERN/EP 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

Requirements (from the CTP) u. Dispersed code development: u The software will be developed by organizationally and geographically dispersed groups of part-time non-professional programmers u. Flexibility: u Not all software requirements will be fully known in advance Not only performance Also modularity, flexibility, maintainability, quality assurance and documentation. Software Architecture Vincenzo Innocente, CERN/EP 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

CMS Software Architecture R&D 95 -96: RD 41 --- OO Detector Reconstruction u Detector model, Local hit cache, Pattern recognition 95 -97: RD 45 --- OO Event Model (persistent) u Event structure, Raw data, Reconstructed objects 95 -97: RD 45 --- Calibration Database u Time dependent data, Versioning, Experience with Objy 96 -98: Implicit Invocation u Event dispatching, Reconstruction on demand 97 -98: Test-Beam (H 2, X 5) u OO Daq, Online filtering, ODB population Software Architecture Vincenzo Innocente, CERN/EP 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

Use Cases L 1 Trigger Simulation Track Reconstruction “Physics” Reconstruction 1/8/2022 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

Track Reconstruction For each “detector element” there are local measurements of trajectory state-vector (just position or more complex) Local measurements are affected by the detector element state (calibrations, alignments) Software Architecture Vincenzo Innocente, CERN/EP Pattern recognition “navigates” in the detector to associate local measurements into a track 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

“Physics” reconstruction u 4 -vector-like objects are built out of trajectories and localized energy deposits u. A wide range of particle identification, jet, vertex etc algorithms can be applied to produce others 4 -vector-like objects u. Access to the “original” detector data maybe required Software Architecture Vincenzo Innocente, CERN/EP 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

Reconstruction Sources Software Architecture Vincenzo Innocente, CERN/EP 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

Reconstruction Scenario Reproduce “Detector Status” at the moment of the interaction: u front-end electronics signals (digis) u calibrations u alignments Perform local reconstruction as a continuation of the frontend data reduction until objects “detachable” from the detectors are obtained Use these objects to perform physics reconstruction and analysis of the “Event” Software Architecture Vincenzo Innocente, CERN/EP 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

Components Reconstruction Algorithms Event Objects Other services (detector objects, environmental data, parameters, etc) Legacy not-OO data (GEANT 3) The instances of these components require to be properly orchestrated to produce the results as specified by the user Software Architecture Vincenzo Innocente, CERN/EP 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

CARF CMS Analysis & Reconstruction Framework Application Physics modules Framework Reconstruction Algorithms Event Filter Physics Analysis Calibration Objects Data Monitoring Visualization Objects Event Objects Utility Toolkit LHC++ ODBMS Software Architecture Vincenzo Innocente, CERN/EP Geant 4 CLHEP Hep. Explorer HTL C++ standard library Extension toolkit 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

Architecture structure An application framework CARF (CMS Analysis & Reconstruction Framework), customisable for each of the computing environments Physics software modules with clearly defined interfaces that can be plugged into the framework A service and utility Toolkit that can be used by any of the physics modules Nothing terribly new, but. . . Traditional architecture can not cope with LHC-collaboration complexity Software Architecture Vincenzo Innocente, CERN/EP 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

Framework Basic Dynamics Implicit Invocation Architecture No central ordering of actions, no explicit control of data flow: only implicit dependencies External dependencies managed through an Event Driven Notification to “subscribers” Internal dependencies through an Action on Demand mechanism Software Architecture Vincenzo Innocente, CERN/EP 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

Event Driven Notification Dispatcher Obs 1 Software Architecture Vincenzo Innocente, CERN/EP Obs 2 Obs 3 Obs 4 Observers 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

Active and Lazy Observers Dispatcher Lazy Obs 1 uptodate obsolete Software Architecture Vincenzo Innocente, CERN/EP Lazy Obs 2 uptodate obsolete Obs 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

Detector Components Load simulated hits from MC Sim Hit Loader Generates Digis from Sim. Hits (or loads them Digitizer from db) Reconstruct measured trajectory state-vector from Digis Software Architecture Vincenzo Innocente, CERN/EP Local Geometry Detector Element Reconstructor Global Geometry Time Dependent Parameters 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

Action “on Demand” Rec Hits Detector Element Hits Event Rec T 1 T 2 Software Architecture Vincenzo Innocente, CERN/EP Rec T 2 Analysis 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

Detector Object Identification A Detector object collection is identified by: u object type (or super-type) u detector element it belongs to implicitly belongs to the “current crossing” Required the detector to be “in place” and “operational” “User” entry point is a Detector Set-up Object Software Architecture Vincenzo Innocente, CERN/EP 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

Event Object Identification A Rec. Obj collection is identified by: u object type (or super-type) u name of the Reconstructor (same as Rec. Unit) u event it belongs to Implicitly belong to the “current detector set-up” Does not require the Rec. Unit to be in place or operational “User” entry point is the Event Object Software Architecture Vincenzo Innocente, CERN/EP 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

Framework Main Services Define the events to be dispatched and link them to their actual source Allow the selection among available resources (user plug-in’s) Integration with the DBMS Manage the “not yet removed” sequential components (coming from legacy code & data) Software Architecture Vincenzo Innocente, CERN/EP 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

Framework Persistency Services transactions metadata and event collections define the logical persistent object structure physical clustering access to persistent event structure effectively shield physics modules from the underlying technology Software Architecture Vincenzo Innocente, CERN/EP 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

Framework Ancillary Services User Interface Error Report (Exception management) Logging facilities Timing facility (statistics gathering) Utility library u Notably Objy utilities, wrappers and generic persistent capable classes Software Architecture Vincenzo Innocente, CERN/EP 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

Framework middle layer A CARF Application is characterized by the events it dispatches (events, geometry for G 3 or Test-Beams) Implementation of generic clients to specific services (events) u simplified API u uniform detailed design u uniform use of ancillary services Requires synergy with detectors’ sub-systems Software Architecture Vincenzo Innocente, CERN/EP 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

CARF layered Structure Core mechanisms and “data structures” G 3 Test. Beam H 2 Raw Data Generic Application T 9/X 5 Raw Data Generic Clients Software Architecture Vincenzo Innocente, CERN/EP 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

“Events” currently dispatched G 3 Geom Start Xing Ready to build new G 3 simulated event Sim. Pile. Up Set. Up Observers are Objects which depend on geometry Software Architecture Vincenzo Innocente, CERN/EP New pile-up event ready in Zebra memory New trigger event Sim. Trigger ready in Zebra memory G 3 Event New event “ready” to be analyzed 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

“Rec. Obj” Object Model Software Architecture Vincenzo Innocente, CERN/EP 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

Results of R&D Traditional software architectures (main&subroutines, pipes&filters) have been found not to be adequate to CMS (long time-scale, large dispersed collaboration) An “implicit invocation” architecture is a flexible software solution which can scale with the complexity of the CMS project. ODBMS, integrated into the framework, provides a coherent management of persistent objects The framework can effectively shield physics modules from the underlying technology Software Architecture Vincenzo Innocente, CERN/EP 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

Summary CMS has developed an Analysis&Reconstruction framework based on a implicit invocation architecture. The present “prototype” u supports both detector- and event-centric reconstruction u provides a fully integrated persistency services based on a commercial ODBMS (Objectivity/DB) u is being validated by several applications ranging from test-beam (Daq and analysis) to high level trigger simulation (digitization, reconstruction, event selection) Software Architecture Vincenzo Innocente, CERN/EP 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN

CMS - Software Components Software Architecture Vincenzo Innocente, CERN/EP 1 st Internal Review of CMS Software and Computing 27 -28 October 1999, CERN