Survey on Users Computing Experience Artem Trunov EKP

Survey on User’s Computing Experience Artem Trunov EKP

Consideration • • Motivation: to improve user's experience with regard to access to data, running jobs… Questions – – – – – Where do you run jobs? How do you submit them? How do ou manage grid submission to different sites? Do you use CRAB? Is it convinient for you? Where do you keep logs from the jobs? If your jobs produce data, how to do you get it back? Where do you store it? How often do you have problems with grid jobs? What kind of problems? Where do you seek help for grid problems? Do you think you'd do a better/faster debugging of your problems if you had access to corresponding sites, etc? Would you really try to debug your jobs at this level? – Your ideal model of your work environment. • 9 full answers, 2 general (from new entrants)

Where and how people run jobs • Grid is used by majority. – One person so far managed to only use local batch exclusively. • Clear preference to limit a number of sites where jobs run – CERN, FNAL are the choice for high avalability – DESY, Aachen, Grid. Ka – as home or “friendly” sites, where the environment is familiar and people are willing to help – Black/white list and “edg-job-submit –r” is used to steer jobs to prefered sites. • CRAB is used by almost all, but some people also or exclusively use direct edg-job-submit through own automation scripts. – CRAB is not (and can not be) used by people running on unofficial datasets or producing own MC. • People use their local (institute) UI to submit jobs.

CRAB experience • 6 – use it – 3 – like it – 2 – report problems • 3 - don’t use it – 1 never used Grid – 1 avoids using Grid and Crab for problems – 1 need to run on official data

Logs and Data • Personal logs – in the home dir – CRAB users keep them mostly where CRAB brings them – in a job dir. • Production logs – in d. Cache • Small data output – back to home • Big data output – copy to an SE – Local or friendly SE is preferable

Notes on storage Individual opinions • Copying from a job to an SE presents a point of failure • Noticed that handling data grid way is not convenient and could be improved. • Prefer to analyze data locally at institute cluster to avoid grid problems. Wi

Grid Problems • 70 – 80 – 90% efficiency • Trouble points – – – CRAB 4 Data or SW published but not available Environment not correctly set 2 Problem with output to an SE 2 “Black holes” 3 RB problems updating job status, not matching sites, SEs, aborts jobs

Grid problem addressing • • • 6 directly to site experts 1 CMS forums 4 GGUS 1 fix him/herself the system 1 fix him/herself a problem with own setup 2 could not resolve at all, don’t believe that problems could be solved

Notes on Grid problems Individual opinions • Simples grid job submission takes 5 -10 minutes • Reporting properly (GGUS, Savannah) is too much work • Many problem are specific and time consuming • Many problems are transient, no one adresses those “usual” problems. • It’s accepted as “natural” that 70% of jobs fail. • Impossible to report every single problem – too many.

Opinions on access to site for debugging • 5 believe in advantage debugging at a site – 2 want access to at least one grid site – 1 want access to worker nodes – 1 want access to a portal machine(s) • 4 doesn’t believe in advantage of debugging at a site

Ideal work environment • 2 Prefered all grid • 3 Prefered all non-grid • 3 Prefered both, split tasks • 1 workgroup servers (incl. UI) • 4 local batch farm • 3 local access to storage • Also – 1 Proof – 1 Better support – 1 Better debugging options

My humble conclusions • Grid is difficult, and in some cases preventing people from successful work • However, most anyway believe in it and would use it if not completely, then partially. • Most would prefer to do analysis w/o the Grid • Generally, people would be happier with local access to our T 2, T 3 resources, where they would find: – Easy access to storage – Access to local batch farm – Debugging options