AMI Status November 2010 Solveig Albrand Jerome Fulachier

AMI – Status November 2010. Solveig Albrand Jerome Fulachier Fabian Lambert 12/11/2021 S. A. 1

AMI Summary • Server & ORACLE problems and improvements. • Bugs. • Metadata Workshop report. • Developments. • Security & Information Protection. 12/11/2021 S. A. 2

AMI No server problems § And the long standing ORACLE bug concerning the incompatibility of case sensitive searches and escaping the underscore character has been fixed. § A search for a string with an underscore in it now works correctly both at CCIN 2 P 3 and CERN.

AMI Performance improvements • We continue to make improvements in the reponse times of AMI. (For more details see TC talk of Jerome Fulachier in SIT ) • All response times were reduced by ~0. 5 seconds by caching the XSLT. (XSLT time now negligible) • Typical response time (99% of commands) – Overview page < 1 s. (35000 executions in Oct 2010) – DQ 2 GUID < 4 s. • The next slides show response times without XSLT!

AMI Ntimes Response time for AMITCEasy. View One server is faster than the other ? n. Vals 23763 24755 secs

AMI Dataset Overview Page Number of Executions Average. Time in seconds There seems to be a slight upturn in the last 2 months which we will be observing carefully!

AMI Some bugs found in Pile Up datasets – • Problem with provenance – due to an over simple algorithm the calculation was missing generated event input when the pile-up was seen first. (FIXED) – Search by keyword = "pile-up" to see them. • Some recent pile-up not marked because the parameter name in the transform was changed. (Not yet fixed). • IT IS VERY IMPORTANT THAT AMI KNOWS ABOUT TRANSFORMATION CHANGES.

AMI Metadata Workshop Report • 30/31 August – report only made available (to me) last week, but we started work already on the requests. • Main request I retained : Get more run based information in AMI. – Read Access on the COMA tables (Elizabeth Gallas). – Rendered these tables browsable in AMI • Using aliases and views to "AMIfy" them. – Can now exploit them with AMI code. – Will be able to link dataset info with run info directly. But don't want to duplicate the run. Browser of course. – Already getting run period info through COMA– and we have plans for closer collaboration COMA/AMI on this issue. (see Elizabeth G. 's talk)

AMI thinks that COMA is an AMI catalogue Can use AMI generic code

AMI Work on Real Data "Show information about missing events and lumi blocks in AMI. " • Requirements from a meeting 2010 -11 -12 with Data Prep + metadata coordinators. • This is a vast project and only the first few steps have been done; comparing the number of events in a RAW dataset with the number of events of all its AOD, ESD and TAG descendants. • Need to synchronize on the file consistency service, COOL, and need "file to file" (LB) provenance tracking.

MC requirements AMI Borut@ MD workshop "Meta-data interface looks a bit technical for the end user" • Transporting cross section values along the MC production chain (less clicks to get the values!) • Some requests are not possible to implement at present. E. g. No "automatic" way of marking datasets "September reprocessing". We cannot (will not) read this information from TWIKI pages or emails. • Related development. : Reworking the "dataset numbers" broker, and extending it to hold production requests.

AMI Other Developments • Synchronizing with DQ 2 : • AMI client for DQ 2 stomp Active MQ service. ( done by a Summer student) • Used for Dataset & Dataset container deletion. • We would like to extend this service to – File consistency – Add/Remove primary datasets from dataset containers.

AMI Security and Information Protection • Until last week most of the information in AMI was world readable. • Following a security audit of the AMI web site at CERN we were asked to put the access to the AMI replica behind SSO and to clean up some rather ugly responses to error conditions or attempts to inject java script. This has all been done with a lot of help from Flavia, and we now have as clean a bill of health as possible from CERN IT security. • But of course this does not protect any information from non-ATLAS members, and the main site remains world readable.

AMI Towards Information Protection • What's more – the replica Web service is no longer accessible from outside CERN • What we plan to do in the short term, is to restrict world readable rights to the top page, and to permit only members of ATLAS VOMS to read AMI catalogues. • When this is done we can probably come out from behind SSO at CERN. LATEST NEWS: CERN security cannot fix the web service problem so agrees that we come out from behind SSO, as long as they can monitor connections. • We would then have the same Authentication & Authorisation mechanism on both sites.