Brightn ESS WP 5 First Integrated Design Review
Brightn. ESS WP 5 First Integrated Design Review January/February 2017 DMSC, Copenhagen Brightn. ESS is funded by the European Union Framework Programme for Research and Innovation Horizon 2020, under grant agreement 676548
Welcome What do we want this meeting/workshop to be? Brightn. ESS is funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 676548
Overview of ESS Data Pipelines Brightn. ESS is funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 676548
Architecture Design Model Brightn. ESS is funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 676548
per beamline DST DM IDG DAM External Beamline Lund Copenhagen Both/Redundant unknown facility services DB UO ICAT ISPy. B +Glue Auth EPICS Config Engine fast SE “BDI” Event Formation Motion et al SE (fast) SE Detector Interfaces and Responsibilities Analysis AGG ECP Live Mantid Live Analysis Reduction Analysis Service Download Web PFS FW
Data flow from Detectors, SE, choppers and motion control to disk, Mantid & analysis i. e With a latency that is outside of spec for EPICS Fast SE readout High speed environment data interface ERC PSS Status PLC layer Control Box Time stamped signals Motion axis Chi. C ERC DM group DAM group ID group DST group ICS Chopper & motion & SE Control Box Time stamped signals IOC(s) ERC ICS software, CCDB , IOC factory naming, SCR Detector group ERC Event receiver card Data reduction automated reduction data correction visualisation of reduced data Analysis interface Instrument PV access gateway ERC IOC(s) Choppers Mantid subscriber Accelerator data Fast SE data Sample environment Kafka API config Fast SE & motion Neutron data and meta data Aggregator Kafka File Nexus Processed ERC Frames of Events & Aggregation of meta data API config Detectors Apache Kafka Python based experiment control system & visualisation of detector data. DAQ control & configuration DB access Data analysis codes visualisation of analysed data WEB Analysis file in Detector signal processing Pixel positioning Detector data interface Reduction / Analysis API Detector signal File writing service DOI catalogue User Office DB PFS
Science NT Groups Detectors Epics Devices Agg. Mantid AAS PV gate DMSC DM File. Writer ICS CA comm Agg Comm File Writer Comm MAAS Comm Server Abstraction layer Script Engine Logging Alarms Configuration service ECP config Access control layer Client Vis service
Overview of WP 5 Task 5. 1 Creating a standard neutron event data stream for different detector types ESS, KU Task 5. 2 Creating a standard method for streaming meta-data for fast applied fields ESS, PSI Task 5. 3 Software to aggregate and make available neutron event data and sample meta-data ESS, PSI, Elettra Brightn. ESS is funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 676548
Task 5. 1 -- Event Formation/Classification Brightn. ESS is funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 676548
Task 5. 2 – Fast Field Data Joint Task with PSI. • • • Collecting Metadata from fast Sample Environments High Speed – up to MHz Lots of data, close to neutron events No turn key solution available that includes accurate timing Evaluating partial technical solutions that can be combined Integration into data stream Brightn. ESS is funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 676548
Task 5. 3 – Data Aggregation Technology Choice: Client Server Client V 4 Server Facility Control System Client EPICS interaction model General Purpose Communication Library “Big Data” Data Broker Design Model Brightn. ESS is funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 676548
Milestones and Deliverables Brightn. ESS is funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 676548
Agenda Brightn. ESS is funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 676548
Workshop Ask questions. We’re not running a tight programme, there is time. Have a positive attitude but do find and point out problems: If you think something will not work, be unreliable, slow or you just don't understand it, say so! Be concrete in your suggestions. Brightn. ESS is funded by the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 676548
- Slides: 14