MAX IV Nexus Status MAX IV Status on

MAX IV Nexus Status

MAX IV Status on NEXUS Darren Spruce and KITS team May 2019

MAX IV Beamlines

Project t n e m e g a n Ma 14 Rev G G E D G G Priority Project ID / Name 152 b_Hippie phase 2 216_BLOCH, Endstation PLC 32018 -II_Balder_1 Bl ready incl Sample stage 413_BIOMAX, Humidifier 5505_Gun Test Facility 6517_Cabling for fast dump MPS 6, 1 517_Cabling for fast dump MPS - R 3 6, 2 517_Cabling for fast dump MPS - R 1 754_308, 317, 108, 112 Gas cabinet safety PLC 8308_Cryo system safety PLC 9507_Rebuild of R 3 and R 1 Aluminium Chamber Shuntgroups 9, 1 2018 -II_Femto. MAX_3 (Dectris Detector) 10 MPS review 10, 1 Radiation safety requested valve tests in R 3 and R 1 11 2018 -II_Max. PEEM_1 12 2018 -II_Fin. Est. Be. AMS_1 12, 1 2018 -II_Fin. Est. Be. AMS_1 Branch A 12, 2 2018 -II_Fin. Est. Be. AMS_1 Branch B 13 2018 -II_Veritas_2 13, 1 2018 -II_Veritas_2 Branch B 13, 2 2018 -II_Veritas_2 Branch A 14 ID compensation in R 1 Project Closed ? Yes Yes Yes Yes Yes

Nexus and data workflow at MAX IV

Disclaimer: I am not a Nexus expert NEXUS INFO from - team members - MAX IV staff scientists

MAX IV Overview and status ● Our focus isn’t Nexus right now ● …… ● It’s bringing experimental stations to their baseline operation as soon as possible ● And harnessing the complexity to make stable operational environments!

Why use Nexus? ● FAIR principles! – Once data is findable and accessible • We want to make it interoperable and re-usable • Nexus provides a standard way to give data meaning ● We need to convince the researchers about the benefit of considering the data life cycle.

First, data needs to be Findable: Scientific Data Management Workflow

Instrument Scientific Data Workflow: the devil is in the details! Experimental setup Meta Data Storage (File system) Nexus? Data treatment software

MAX IV standard tools generating Nexus ● Sardana data acquisition tool – Uses recorder modules to write data to file following Nexus conventions ● LIMA (library for image acquisition) – ? ? ? Writes detector data as Nexus?

HDF 5/Nexus at MAX IV Experiment Stations • 4 operational beamlines: • Macromolecular Crystallography (Bio. Max) • Eiger Detector saving data in HDF 5, format close to Nx. Data & NXMX but not fully compliant. • Compatible with MX beam line processing pipelines • Nano Focus and Coherence (Nano. MAX) • Detectors saving data using Lima version from 2016 • Lima writes Nexus (Nx. Detector) • Eiger, XSpress 3, Merlin, Pilatus • Soft X-ray Spectroscopy (Fin. Est. Be. AMS and Hippie ) • Scienta Detector. Saving data in a proprietary format. • In the future ICE project will be used.

HDF 5/Nexus at MAX IV Experiment Stations 6 commissioning beam lines: • Balder (Absorption, EXAFS) • XSpress 3 detector. Saving data in HDF 5/Nexus through Lima. • BLOCH (ARPES) • Scienta detector. Proprietary format • Femtomax (time resolved pump probe, absorption, XRD) • Several detectors. Combination of proprietary formats and HDF 5. Only standard sardana scans in Nexus. • VERITAS (RIXS) • Data in HDF 5. • MAXPEEM (X-ray Electron Microscopy) • Proprietary format. Will be converted to HDF 5 in the future • SPECIES (XPS. . . ) • More than one detector. HDF 5 and proprietary format is generated.

Discussion on Nexus with Staff members at Beam lines in operation

Nano. MAX - Nano focus and coherence ● Typical experiment – snake scans generate >100000 images ● Needs – Data management is more important. . . • save more images per file (handling many files is painful) • Compression (moving large files is painful) • Links between HDF 5 files (file naming conventions specific for detectors is painful) • Custom sardana recorders for specific information for scan – To eliminate wasted space for non-relevant information • Decoration with Nexus (could be useful for others)

Data Flow

BALDER Want to handle dynamic data: • E. g. Write I 0 as the sum of 2 measurement channels • Sardana generates NXMeasurements • These are custom named channels set by the beam line • Same devices can measure fluorescence or transmission data • How to distinguish from the data?

Par. Sec XAS spectrum viewer

HIPPIE (Operando Spectroscopy) discussion on data formats Users/staff write meta-data in a paper logbook. ● • Would be nice to generate automatically as sometimes they can forget ● Scienta analyser/detector has a ’beta’ API currently and proprietary file format – Planned to use new API from Scienta to drive the detector ● The researchers write tools which read from the Scienta ● Visualisation software supplied by detector company

BLOCH (ARPES) discussion with staff scientist • Similar to HIPPIE - Need to write new Scienta Interface • Only two detector suppliers in world, currently • User community de-facto standard defined by detector suppliers • Many ’power’ users • Write their own software to read detector data directly • Would be nice to have • Meta data from instrument available with detector data • ”Supplied as CSV on request”

Nexus Issues at MAX IV

Nexus Compliance Issue? ● Standard acquisition tool (Sardana) generates NXScans to store experimental channels in NXCollections ● NXCollection Channels are configured per beamline using custom naming schemes ● Entries in NXCollections are unstructured • cannot parse automatically

Implementation issue: Where is my data? ● I want to plot my data without knowing how to find it in the Nexus file ● some MAX IV data is missing the default attributes for plotting – Or it isn’t implemented currently

How to find default data

data workflow Ideas to move forward Today is considered low value by scientists and researchers • Refer to Data Management Plans to influence the design of data workflows Knowing the end requirements for the data may help to improve the data management at the various steps of the workflow

Future Plans Internal projects: • Better detector integration -> better data handling - ICE Project • High throughput data - Eiger Data Acquisition • MAX IV Data Storage and Management - Data. Stamp

Nexus stakeholders Researchers They must be able to operate with Ne Staff scientists xus data They need to supp ort nexus when pro curing beam line in devices frastr Controls & IT group ucture and They need to supp ort the data workfl ows with tools to w nexus in a suitable rite and display way for users Management They must support strategic guideline s to focus on stand like nexus in the fa ard data formats cility