Progress on Design and Implement of Job Management

Progress on Design and Implement of Job Management System Suo Bing, Yan Tian, Zhao Xianghu suob@ihep. ac. cn 2014. 12. 24

Outline Design principle and goals Data Flow Diagram Inspiration from Ganga Profile of JMS Integration of JUNO Integration of CEPC

Design principle and goals for users to split and submit jobs to backend common platform supporting multiple experiments: BES 3, CEPC, JUNO, LHAASO, etc. general, simple, easy to expand Python 2. 7

Data Flow Diagram app input Users sub job input + steering file Receive Input split submit execute input output sandbox Backend output data SE

Inspiration from Ganga Reference (advantages) classical architecture of “Job, Application, Splitter, Handler, Backend” common interface in GPIDev for expanding strong, rigorous Abandonment (disadvantages) complex wrapped class (schema, item) confusing attributes (inputsandbox, inputdata)

JMS architecture user specify and set input Application app params sub job params Adapter backend params Backend Splitter

JMS class diagram

JMS directory structure <repo_i d> param s job app status subjob 0 input job backend 1 . . .

JUNO Several script are supplied(by Lin Tao) for simulation and reconstruction in “offline/Examples/Tutorial/share” Run directly: $ source …/setup. sh $ python tut_detsim. py [--gdml run] Integrate to JMS: job 0 input tut_detsim. py job. sh

JUNO test script #Import Statements j = Job() app = Juno. App() app. set. Command('tut

JUNO parameters app_params job_params

JUNO parameters backend_params

JUNO Sequence Diagram

CEPC Generation + Simulation + Reconstruction Generation→. stdhep→Simulation/Reconstruction Steeringfiles (init. macro, event. macro) for simulation Steeringfiles(init. xml, Pandora. Settings. Default. xml. . . ) for reconstruction Integrate to JMS: job Parsers read users' templates and generate input files for per job 0 input job. sh init. macro event. macro init. xml Pandora. Settings. Default. xml Pandora. Likelihood. Data 9 EBin. xml

CEPC test script app = Cepc. App() app. set. Init. Macro. Path('/home/cc/suob/tem plate/init. macro') app. set. Event. Macro. Path('/home/cc/suob/te mplate/event. macro') #not flexible app. set. Init. Xml. Path('/home/cc/suob/templa te/init. xml') app. set. Pandora. Setting. Path('/home/cc/suo b/template/Pandora. Settings. Default. xml')

Summary Evaluation – Inspired by ganga. BOSS and ILCDIRAC, we designed a general Job Management System for BESIII, CEPC, JUNO and LHAASO. – A simple prototype has been coded and implemented. – CEPC and JUNO are supported and tested in this prototype, BESIII will be added soon. Future work integrate more experiments and cases to