The Diamond Mapping Project Automation in Beamline Control

The Diamond Mapping Project Automation in Beamline Control and Data Acquisition Workshop, Helmholtz-Zentrum Berlin (HZB) 17. Jan. 2019 Presenter: Ulrik Pedersen, Head of Beamline Controls Contributions from: Tom Cobb and Mark Basham

Topics • Motivations • Introducing the Mapping Project • Products – Control and Data Acquisition • Lessons Learned • What is next • Acknowledgements and references February 2021 2

Motivations February 2021 3

What is Mapping • Move sample under focused x-rays • Continuous or trajectory motion – “fly scan” • Interactions with the sample are recorded • Process – Reduce data to create a map (image) • Visualization – Show map of reduced data – Click to see raw or detailed data February 2021 Abs DPC DF 1 mm CC O O Fe 4

Motivations • Mapping or scanning-probe beamlines conduct similar scans – Pool the development resources to implement a common solution • Mapping SW step-scans performance – Typical scan: several hours or over-night – Fly-scan or continuous-scan implemented ad-hoc on some beamlines depending on available equipment • Automation – Support top-up, pause, rewind, etc. – Cope with missing/dropped detector frames • Live feedback user interface – progressing scan and data • Live processing – Start analysis at the beginning rather than the end of a scan February 2021 5

The Project February 2021 6

The Project Breakdown • Split into several work packages – – – 1. 2. 2. 1 3. 4. 5. SWMR (HDF 5, area. Detector) Map Visualisation + Processing Mapping Scan Infrastructure Experimental Parameter Control Hardware Middleware Piezo Motor controllers • Roll-out solutions, test & commission on beamlines: – – – I 08 I 05 -1 I 14 I 13 I 18 February 2021 7

The Project Team • Involvement from a wide range of groups – – – – Scientists (users and managers) Project Office (planner) Scientific Software Group (data analysis) Data Acquisition Group (GDA & Ne. Xus) Beamline Controls Group (EPICS & HDF 5 DAQ) Motion Control Team Scientific Computing Group (IT infrastructure) • Over 20 individuals* involved over ~2 years – scientists, developers & engineers – Some new-comers, some leaving throughout the project *Acknowledgements at the end of presentation February 2021 8

GDA Mapping Client What is it? GDA 9 Scanning GDA/DAWN visualisation SWMR Messaging EPICS/ Malcolm Middleware Data DAWN Processing

Products Developed February 2021 10

Control and Data Acquisition • • GDA 9 Malcolm middlelayer • • Consistent interface to GDA EPICS area. Detector • • • configure() run() Malcolm HDF 5 SWMR support Pos Plugin caput camonitor New EPICS pmac driver • • “Model 3” motor driver Trajectory scan support data motion trajectory February 2021 11

HDF 5 SWMR • Single Writer Multiple Reader • Process data while it is being written to file New data elements … Writer Reader …that are added to a dataset in the file… … February 2021 HDF 5 File can be read by a reader… with no IPC necessary. 12

EPICS area. Detector NDFile. HDF 5 • SWMR support – Only available with HDF 5 >= 1. 10 – Optional enable/disable • Multiple dimensions in addition to the 2+1 D – Arrange data by scan dimension February 2021 13

EPICS area. Detector NDPos. Plugin • Snake scan • Write data in snake pattern • Leave gaps for dropped frames • Rewind and overwrite data • Monitor HDF 5 for live updates Detector Driver NDArray Position Labelling Plugin XML positions X: 0 Y: 0 X: 1 Other plugins here HDF Writing Plugin XML layout HDF 5 file

Delta Tau Geobrick LV • • • PMAC 2 based Complex co-ordinate systems with arbitrary kinematics Enough user memory for trajectory scanning

https: //github. com/dls-controls/pmac EPICS pmac model 3 driver • Support for Turbo PMAC and Power PMAC • Move raw and coordinate system motors • PVT scanning of motors • Double buffered points Positions + Build Run. Positions + Append pmac driver Motion Program Motor

Lessons Learned February 2021 17

Managing Upgrade Projects • Upgrades to operational beamlines is challenging – Engineers busy with operational support • New development require focus and few interruptions – Beamline schedule with users • Don’t interfere with working systems • Priority of new upgrades & testing vs. operation • Cross-disciplinary project team – Regular communication – Diverse technology stack • Define interfaces early • Regular “integratathons” February 2021 18

Technology: motion • Stages and actuators – Designed originally by “static” specifications • Range, resolution, stability – Dynamic, continuous scanning is a different requirement • Constant velocity • Settling time • Oscillations • Timing – Feedback signals highly time-sensitive • Micro-seconds matter • Filters: time delay vs. noise • Complex feedback electronics: time delay unknown February 2021 19

Technology: storage • Must be POSIX for SWMR support – DLS storage: Lustre and GPFS – NFS specifically does not work • Chunking configuration is complex – Main (image) dataset: large chunks for performance – Meta-datasets: small chunks for regular updates • Notifiy live view/processing of new data February 2021 20

What is next? February 2021 21

More Projects… • Other beamlines can make use of the core products – Panda/malcolm for hardware triggered scanning – HDF 5 SWMR for live processing/visualization – GDA Scanning and DAWN live visualization • Other techniques require HW triggered scanning – Tomography – Ptychography • New projects on the way – – Ptychography – the main follow-on project from Mapping MX – supporting continuous scans and Eigers I 22 – Time resolved experiments DIAD – tomography & diffraction interleaved February 2021 22

Acknowledgements • Acquisition • • Analysis • Mark Basham (PM) • Rob Walton • Sylvie Da Graca Ramos • Alun Ashton • Matthew Dickie • Jacob Filik • Matt Gerring • • Peter Chang • James Mudd • Baha El Kassaby • Charles Mita • Controls • Colin Palmer • • Ulrik Pedersen • Keith Ralphs • Tom Cobb • Matthew Webber • Alan Greer • Mike Klyne • Nick Rees • Kaz Wanelik • Gary Yenel • Anthony Hull • Nic Bricknell • Giles Knapp Motion Control • Lee Hudson • Trevor Bates • Brian Nutter Science • Andy Dent • Paul Quinn Project Planning • Alison Shepperd • Kathryn Poulter • Sonia Moon • Steven Launchbury

References • • M. Basham et al. (2018) “Software Mapping Project with Nanopositioning Capabilities”, Synchrotron Radiation News, 31: 5, 21 -26, DOI: 10. 1080/08940886. 2018. 1506238 Rees, Nick, et al. "Developing HDF 5 for the Synchrotron community. " (2015): WEPGF 063. J. Filik, On-the-fly data visualisation and processing using the HDF 5 singlewriter/multiple-reader library at Diamond Light Source, Journal of Synchrotron Radiation (in preparation) Links to software – DAWN Contact me: – GDA – EPICS model 3 motor driver "pmac" – malcolm – EPICS area. Detector – HDF 5 SWMR February 2021 24