NSLSII Control System The Control Group presented by
NSLSII Control System The Control Group – presented by Bob Dalesio EPICS Meeting April 12, 2015 1 BROOKHAVEN SCIENCE
Outline • • • Project Planning Integration shortcomings (Lessons learned) Setting Standards Control System Studio (CSS) Tools Fast Orbit Feedback Components Development Middle Layer Services for High Level Application (Relational) Database Developments Experiment Control, DAQ, Data Analysis, Data Management. Conclusions 2 BROOKHAVEN SCIENCE
Project Planning • Identify critical requirements • Define scope and interfaces • Recruit, Recruit • Project Execution 3 BROOKHAVEN SCIENCE
Control System High Level Requirements – 1 of 3 • • • 2. 642 usec ring revolution Injection system runs up to 1 Hz Top off every 1 minute Top off bunch train ~20 psec Top off damping time 10 -50 msecs (inform beam lines) 80% Fill – 1056 of 1320 buckets filled • Orbit stability to. 3 microns at injection • • 5 Hz updates to operators of up to 1000 chosen parameters Archive up to 20, 000 parameters at a rate of 1 Hz continually Must scale to support 150, 000 physical I/O connections and 400, 000 computed variables 99. 7% availability during operation • different fill patterns • high charge single bunch at mid-gap 4 BROOKHAVEN SCIENCE
Control System Requirements – 2 of 3 • • 80 psecs pulse to pulse timing jitter. • 20 msec equipment protection mitigation • Transient Recording • • Provide data for all control aspects (no hidden parameters) Conventional facilities • During top off, some beam lines will need 1. 1 - 1. 8 psecs of timing jitter Timing resolution • • • 2 nsec for bunch to bunch timing 200 psec for single bunch injection Provide electron detector as event for beam line 10 k. Hz power supply read backs triggered from timing sys All data available to system with revolution identifier for turn by turn data correlation. • Sub 1 msec at other labs when undulator gap is closed – it’s the xrays • Also – slow detection/shut down on RTDs on beam pipe for beam studies when beam emittance is wide • Plc provides voltage to comparator to bpm signal – forward/reflected to give current – with fiber optic transmission of this signal to the RF – one from each cabinet. 10’s of micro seconds • 200 turns to loose energy after RF is off • Photon shutter – can take full energy? • Take coherent turn by turn orbit data for up to 800 channels for 1024 turns (each turn is 2. 36 usec) • Latch the last 10 seconds of data from all parameters in the storage ring • Beam line needs 1 msec archiving over 1 minute for temperatures and positions • Tunnel and experimental floor Vibration < 25 nm from 4 -50 Hz • Thermal stability of storage ring tunnel enc +-. 1 dgc • Experimental floor +-. 5 dgc 5 BROOKHAVEN SCIENCE
Control System Requirements – 3 of 3 • • 5 KHz RF Feedback on beam phase 10 k. Hz orbit feedback, (100 usec loop time) • Slow orbit correction • • • 10’s of Hz Data Collection for RF loop correction. 200 Me. V to 3 Gev. V in 400 msec ramp in booster for gap changes Transverse feed back Bunch to Bunch • BPM Requirements • Requirements at an accelerator evolve!! Standard engineering flow is iterative – Requirements-specification-designimplement-test-integrate, so flexible designs and attention are required. • 360 BPMs (12 per cell) • 90 Corrector PS (3 per cell) • 180 Slow PS (6 per cell) synchronized with fast orbit feedback • 1 Hz model based control • 1 H and 1 V stripline • RF cavity control • • Turn by turn on demand Fast 10 k. Hz output Slow 10 Hz data ADC Raw data 6 BROOKHAVEN SCIENCE
Control System Scope and Interfaces (1/3) Channel Access Protocol EPICS Time – events / data / triggers Timing C E TTTTT CCCCC P V MMMMM U R C E Digitizer P V U R C E P V U G C E P V U R Ethernet – Instrumentation Bus Machine Protection Bus 500 MHz w/ 60 Hz Fid PLC LLRF RF Osc Non-BPM Diagnostics RF Equipment Timing Libera Modules Diagnostics RF 7 BROOKHAVEN SCIENCE
Control System Scope and Interfaces (2/3) Ethernet - Channel Access Protocol Timing – events / data / triggers C E P P V S U R I C E P V U R C P U Ethernet – Instrumentation Bus Machine Protection Bus P L C Facility I/O Temperature & High Prec. DMM Power Supply Controllers Power Supply Control G P BBAA I I OI O B Power GP 307 IG Supplies HP 937 CCG Vacuum Facility 8 P L C A I Valve Controllers G P BBAA I I OI O B A I PPS I/O Personnel Safety Sys. BROOKHAVEN SCIENCE
Control System Scope and Interfaces (3/3) Timing C E I P V / U R O Injection Linac Control Timing C E I P V / U R O Booster Ring Control Beam-line Control Design, Prototype solutions, Produce beamline controls Help write specifications, monitor progress, Particpate in integration testing, take over maintenance 9 C E I P V / U R O Undulator Control Design, Prototype solutions, Produce undulator control BROOKHAVEN SCIENCE
Control System Software Scope Develop Control Algorithms Populate RDB Client Tools Engineering Applications Physics Applications Configuration Tools Engineering Screens XAL/RDB Tools EPICS Database EPICS Device Support Drivers I/O Device test / subsystem integration Commissioning Includes: LINAC, Booster, and Storage Ring, Support Facility, Undulator, Beamline 10 BROOKHAVEN SCIENCE
Project Execution (1/2) • • • Budget: 20. 59 M Controls of 462 M: 349 M accelerator 113 M beam lines One PCR added approximately 2 M for network and computer infrastructure Actuals: 21. 36 M -770 K (CPI =. 96) Project missed early completion date by several months. Staffing: Now 40 - 30 New Hires into Light Source Division, 10 transferred from LS 1. Recruiting and building a team of experts is a major job. • Engineers were given responsibility and authority over complete subsystems (BPM, PS, RF, Physics. . ). • They also were given projects that pushed their technical skills. • The control engineers are co-located to help them keep up with the work of their control colleagues. One person reported EVMS execution has certain knobs for BROOKHAVEN SCIENCE 11
Project Completion (2/2) • • Scope includes: • FPGA code for power supplies (1000), BPMs (300), Fast Orbit Feedback (30), Position Capture / Detector Triggering, Single Channel Correlator, Fast Machine Protection, Timing, and Active Interlock. • PLC code for low speed IO: utilities on the accelerator, all PLC IO on beam lines including EPS • EPICS Driver for all hardware (most provided by community) • EPICS Databases for all subsystem and facility IO • CSS Synoptic Displays • Middle layer services for physics applications including a model server • Network and computer layout • Choose EPICS tools where they are adequate • New EPICS tools and services: channel finder, save/restore, meta. Data. Store, Olog, CSS build, extended Boy widgets, unit conversion, area. Detector PVAccess Plugin, No. of EPICS records 985 K Accelerator + 180 K for 6 beam line delivery systems No. of IOCs 163: 45 VME, 7 c. PCI, 111 IBM Servers (Instr Bus – Ethernet) No. of 19" racks 485 No. of devices 61 unique device types No. of drivers 26 No. of signals 200 K Ines of code for EPICS tools: 1. 8 M BROOKHAVEN SCIENCE 12
NSLS II Integration Lessons Learned I (1/4) Linac LINAC Beam Dump Faraday Cup Bending Magnet Second Dump Faraday Cup Booster Tunnel Restricted Area Radiation Monitor 13 BROOKHAVEN SCIENCE
NSLS II Integration Lessons Learned I (2/4) 14 BROOKHAVEN SCIENCE
NSLS II Integration Lessons Learned I (3/4) 15 BROOKHAVEN SCIENCE
NSLS II Integration Lessons Learned I (4/4) • • The primary issue here is incomplete consideration of machine modes and insufficient training of operators. Good procedures for commissioning avoided any safety issue. Thorough follow up uncovered and resolved this and other shielding issues. Defining the modes of operation with soft limits could have prevented this by limiting power supply range when operating with only one RF system. There was no control design document for PPS from controls as another team was implementing it and controls was integrating it. This could have lead to a deeper questioning of modes of operation. Instruments were integrated, but not tested for operation. Linearization errors resulted in oversaturation of the instrument and no alarm condition flagged in the control room. 16 BROOKHAVEN SCIENCE
NSLS II Integration Lessons Learned II (1/4) • During low power tests, the Active Interlock System did not respond under testing when the beam was moved. • Upon study of the PVPut. Log it was discovered that all BPMs were put into test mode. • This was done by someone unwittingly pressing an ambiguous button. 17 BROOKHAVEN SCIENCE
NSLS II Integration Lessons Learned II (2/4) • Modifications: • Remove the ability to set the BPMs into simulation mode from the screen • Change the FPGA code to disable simulation greater than 2 m. A would need to transmit to BPMs. This requires: • Run a signal from the cell controller nearest the EVG into a signal in the EVG that indicates that we are over 2 m. A. This signal is wired from the safety PLC into a cell controller near the ICT. • Send the machine mode from the EVG to the EVR in each of the BPMs and use it to disable simulation mode in the FPGA. • Modify the AI cell controller to compute BPMs that are not responsive and take appropriate action based on the loss of one or more BPMs. • Other parameters were identified that could render compromise the BPM readings by changing the calibration/conversion constants. • Further tests were listed as a result of an engineering meeting to discuss potential failures through disconnecting signals and removing power from critical devices. 18 BROOKHAVEN SCIENCE
NSLS II Integration Lessons Learned II (3/4) 1‐ Remove each fault signal from a cell controller and note the response from the AIS a. Disconnected the 2 m. A current signal from the cell controller at cell 3. i. No interlock. b. Removed the DCCT heartbeat signal from the Cell controller at cell 3. i. Indication of heartbeat stops on the control screen. ii. No interlock. c. Disconnected the request to trip signals from the front end EPS system. i. Fault indicated on the CSS screen. ii. Interlock. d. Removed the EPS heartbeat signal from the cell controller at cell 3. i. Indication of heartbeat stops on the control screen. ii. No interlock. e. AIS disengaged: removed the ID gap status input from the cell controller at cell 3. i. AIS engages. f. AIS engaged: removed the BM photon shutter input. i. Interlock. g. AIS engaged: removed the ID photon shutter cable. i. No interlock. 2‐ DCCT fault condition test. a. Powered off the DCCT from the control room this caused an audible alarm in the control room as expected. b. Crashed the DCCT IOC and the result is the same audible alarm in the control room as expected. 3‐ Remove timing at cell controller (additional testing added). a. Removed timing fiber from the cell controller at cell 3. i. BPM data stopped updating, ii. No interlock. iii. Restored the timing cable and the system came back on line as expected. iv. Enabled AI and repeated the test, this resulted in an RF trip. b. Removed timing fiber from ID BPM. i. No interlock. c. Removed timing fiber from RF BPM. i. No interlock. d. Removed RF rev tick from ID BPM and AI tripped the RF. 4‐ Remove the SDI cables from the BPM. a. Interlock. b. Repeated the test no interlock. 5‐ Power off any cell controller. Powered off the cell controller at cell 3. a. No interlock. 6‐ Power off ID BPM. a. Interlock. 7‐ Remove the timing fiber from cell 31. a. Interlock after delay. b. Repeated the test same result. 8‐ Remove timing fiber from the EVG. a. Interlock. b. Repeated the test interlock. c. System recovered smoothly after reconnect and beam was quickly restor 19 BROOKHAVEN SCIENCE
NSLS II Integration Lessons Learned II (4/4) • Fault discussion: task lead: “We had these issues as no one was in charge”, response: “You were”. Be clear about subsystem/task ownership across organizational lines. • Good commissioning plans turn up surprises and avoid disaster. • Carefully develop operations screens and hide engineering screens. • Control system engineers should spend 10% of their time understanding what must be done, and 90% of their time considering what is missing and how it will fail. • TEST ALL FAILURE MODES!! 20 BROOKHAVEN SCIENCE
Standards Save Time (1 of 2) • All Hardware Integration is achieved through EPICS • Hardware standards are followed wherever possible through NSLS II • https: //wiki. bnl. gov/nsls 2 controls/index. php/Beamline_Co ntrols_Equipment#Standards • Control System Studio is used for operator tools • Archive Appliance or Channel Archiver to archive time series data • BEAST – alarm server • Channel Finder Service – Provides properties/tag searches of the Process Variables • MASAR – save set manager and user interface • Olog – Electronic log book • Meta. Data. Store – store sweep information to search data sets 21 BROOKHAVEN SCIENCE
Standards Create Clarity (2 of 2) • • • Naming convention https: //ps. bnl. gov/acc/controls/Shared%20 Documents/Naming. Convention/LT-ENG-RSISTD-002_Rev 4. docx https: //ps. bnl. gov/acc/controls/Shared%20 Documents/Naming. Convention/LT-PRJ-STDCO-NAM-001. xlsx Coordinate system https: //ps. bnl. gov/phot/Project. Beamlines/Controls/References%20 and%20 Standards/Coo rdinate%20 System/LT-C-XFD-STD-BL-COORD-001_Ver 3. docx Interfacing standards https: //ps. bnl. gov/phot/Project. Beamlines/Controls/References%20 and%20 Standards/Wiri ng%20 Standards/LT-C-XFD-SPC-CO-IIS-001_Ver 5. docx Controls RSI https: //ps. bnl. gov/phot/Document. Reference. Library/RSIDocuments/1. 04. 02%20%20 Stand ard%20 Local%20 Controls%20 and%20 Data%20 Acquisition%20 Systems/Stnd%20 Local %20 Controls%20 and%20 DAQ%20 RSI%20 Development/LT-C-XFD-RSI-CO 001_Ver 3. docx EPS standards (draft) https: //ps. bnl. gov/acc/controls/Shared%20 Documents/Beamline. Controls/Industrial%20 Co ntrols/LT-C-XFD-STD-BL-EPS-001_Ver 1_draft. docx PMAC programming standards (based on Diamond standards): https: //ps. bnl. gov/acc/controls/Shared%20 Documents/Motion. Systems/Standards/LT-RXFD-STD-BL-MC-002_Rev. A. docx 22 BROOKHAVEN SCIENCE
Control System Studio • • • Adopted CSS for our user interface after evaluation Developed Channel. Finder. Service to return process variables by function, location, device, etc… Developed a channel access package to provide all client functions and time synchronous vectors from an arrays of process variables (PVManager) Developed multichannel visualization tools Developed an Integrated Log Book Developed a multi-controller application (Channel. Orchestrator) Developed a save/restore application (MASAR) Developed standard extension points in CSS for services such as log book and directory service. Developed standard extension points in CSS for plots. This first deployment of CSStudio for a facility is exposing many new requirements and some shortcomings that need to be handled. 23 BROOKHAVEN SCIENCE
Control System Studio Creating a log entry from the alarm tree Create log entry command available is most applications. 24 BROOKHAVEN SCIENCE
Control System Studio Pre initialized log entries The applications define how to effectively capture the state e. g. when creating a log entry from the alarm viewer the alarming PV, the alarm state the time are automatically added to the log entry. Each application provides the most useful way to capture its information into a log entry. 25 BROOKHAVEN SCIENCE
Control System Studio Log viewer Once the entry is made users can later search the log book for a particular alarms e. g. search for : alarms *ACMI* 26 BROOKHAVEN SCIENCE
Control System Studio PV aware entry If log entries contain pv names defined with a specified format, csstudio is able to extract that pv information from the log entry. CS-Studio will add the “Process Variable” sub menu to the context menu when it can extract a PV. The default syntax is PV: <PVName >/n, however each site can define their own regular expression for recognizing pv names. 27 BROOKHAVEN SCIENCE
Control System Studio Open Databrowser to appropriate time The extracted PV can be piped to any other cs-studio applications. In this case we opened the alarming pv in databrowser, with the time axis automatically set to +/- 30 mins from the time the log entry was made. Good tools improve efficiency. 28 BROOKHAVEN SCIENCE
Orbit feedback system architecture Fast orbit feedback system in one cell 29 BROOKHAVEN SCIENCE
Orbit feedback system is in commissioning PI control demonstrated suppressing noise from DC to 200 Hz Cell 1 Cell 30 Cell 2 SDI link Cell 29 Cell 3 Storage Ring SDI link Cell 17 Cell 15 Cell 16 30 BROOKHAVEN SCIENCE
Fast Orbit Feedback – Time Budget • 17. 7 µs for BNL BPM Electronics delay to pipeline • 7 µs measured 300 BPM data distribution by Redundant Data Link • • • 3. 7 µs for analog value averaged from raw data • 14 µs estimated for BNL BPM Electronics • (FYI 57 µs measured on Libera BPM Electronics with minimum filter) • 14 µs measured when one of the redundant links not operational 4 µs measured for FOFB calculation with separate mode compensation 5 µs measured for set power supply by PS Shared Memory link 8 KHz measured for fast corrector PS bandwidth w 10 deg phase shift. 1 KHz measured for corrector magnet/chamber bandwidth. • Measured on with Inconel beam pipe. This value is at 10 degree phase shift. • New magnet/chamber set up will be tested when they are available. BROOKHAVEN SCIENCE 31
Production DFE and BPM AFE BPM: 117 MHz Raw Data BPM: 2. 36 usec TBT Data BPM: Xmit 10 k. Hz fast acquisition data for FOF Cell Controller: 10 k. Hz fast orbit Cell controller: 10 Hz slow orbit 32 BROOKHAVEN SCIENCE
Active Interlock / Cell Controller Virtex-6 FPGA • Embedded Micro. Blaze soft core processor running TCP/IP lw. IP stack in conjunction with EMAC Ethernet core • 2 Gbyte DDR 3 SO-DIMM • 1 Gbps Ethernet Hardware TEMAC Memory throughput 6 GBytes/sec • (6) 6. 5 Gbps SFP modules • 1 Gbit FLASH memory • 4 Lemo differential output • Embedded Event Receiver MRF’s EVR functions embeded to FPGA Very flexible and precise timestamp and time synchronization 16 Digital Inputs 12 Digital Outputs 33 BROOKHAVEN SCIENCE
Beamline IO Position capture and trigger board 34 BROOKHAVEN SCIENCE
Correlator Combo Correlator combo FPGA The input data to correlator is random digital pulse with ~5 ns width. Maximum rate is 100 MHz. Correlator Combo, i. e. multi tau correlator plus linear correlator, where multi tau correlator is to get the decay curve of G 2 in a large correlation lag range and linear correlator is to capture the high frequency oscillation, deploy one correlator combo on Virtex 6 in Cell controller for prototype first, more on Zynq later. Cell controller Digital Front End (DFE) board where Correlator Combo 5 ns digital deployed pulse input from ADP detector 35 Ethernet port RS 232 port for G 2 curve for debug output BROOKHAVEN SCIENCE
Modular Digital Front End Firmware Priovides: EVR decoder Clock at 117 MHz SDI redundant RT data Ethernet to EPICS 4 GB RAM over PLB Filing all paperwork to make it OPEN SOURCE Not license it so we can buy it back w/ markup. Successfully developed a number of instruments Given to LBL where improvements were made for their use. Being evaluated for APS & PLS. 36 BROOKHAVEN SCIENCE
Middle Layer Services Simplify Applications • High level application architecture is client/server based on the EPICS version 4 network protocol, PVAccess • One server provides results to all clients • Tables are used to serve database results • These are being implemented in PVAccess • Servers are completed that provide: • Channel Finder - List of process variables based on function and attributes • MASAR – save set service • Model/lattice services • Developer document for thin clients development • Standard clients in CSS can be used to display HLA data 37 BROOKHAVEN SCIENCE
V 4 HLA Architecture MMLT Client Scripting HLA Client PVAccess/CAC Control System Studio PVAccess / CAC Ethernet Middle Layer Servers PVAccess Model Server Lattice Server Channel Finder Svr Elegant/Tra cy SQL RD B Distributed Front-Ends PVAccess/CACPVAccess/CAC PVAccess PVA CAS Diagnostics Power Supplies Physical Device RD B Multi. Channel Arrays Svr Serves orbit, magnets, any array of channels Save/Comp Save/Retrie are Restore ve Server Svr Magnet, Resp Mtx, etc. . SQL RD B PVA CAS RF Vacuum Utilities etc. . , Diag & PS Physical Device LS 2 Simulation 38 BROOKHAVEN SCIENCE
Channel Finder Overlays Different Views • • • Control Engineers talk about PV/IOC, physicists talk about element/cell/girder. PVs are associated with properties and tags, which are used to construct a lattice familiar to physicist. “the BPM readings in cell 10 girder 2” is easier than PV name 39 BROOKHAVEN SCIENCE
Channel Finder Makes Scale Managable • • HLA uses several thousand PVs. Each have properties like “elementname”, “elementtype”, “cell”, “girder”, “s_position”, … Some PVs are tagged with “HLA. X”, “HLA. EGET”, “HLA. EPUT” which means that PV is a “horizontal reading/setting”, “default reading” and “default setting” The lattice object is a list of elements, each owns several PVs. All of the data is initialized from channel finder service: pv, properties and tags. No need to remember PV when writing a normal script. 40 BROOKHAVEN SCIENCE
Channel Finder Overlays Different Views 41 BROOKHAVEN SCIENCE
Channel Finder w/ CSS For those who cannot remember 1 M PV names or do not have the patience to type them into screen configurations. Dynamically creates hierarchical views based on channel property values 42 BROOKHAVEN SCIENCE
Channel. Finder Viewer CSS Display Channel. Finder. Viewer Searching • Name, Property value, Tags • Regular Expressions using “*”, “? ” • prop=val 1, val 2 is equivalent to prop=val 1 OR prop=val 2 • prop=val 1 tag=my. Tag is equivalent to prop=val 1 AND tag=my. Tag 43 BROOKHAVEN SCIENCE
HLA – Thin Clients built on services 44 BROOKHAVEN SCIENCE
HLA – MASAR Use • Events (by 2014 -07 -14 05: 12: 08) approved: 862 total taken: 1226, 45 45 BROOKHAVEN SCIENCE
Middle Layer Services for High Level Apps • Middle layer services reduce the time and • • • complexity of high level applications. Developing Application Programming Interfaces (APIs) to support thin applications requires time and collaboration that is not needed in monolithic codes. Developing narrow APIs that can be used by applications and tools, such as CSS, requires enough action to get it done and enough collaboration to get it right. (We had mixed execution here). Most physics applications are now layered: 46 BROOKHAVEN SCIENCE
(Relational) Database Experience • Goal: All relational data information integrated in IRMIS • IRMIS table evolution and application development conflicted • IRMIS table redesign broke all JDBC production code ……………. . so domains were separated out except • New IRMIS tools were being developed for • Component type editor • Component editor • Component Browser • Wiring Editor • BUT…… never completed in time to deploy ………… other domains developed into deployed applications 47 BROOKHAVEN SCIENCE
(Relational) Database Experience • Channel Finder application uses My. SQL (moving to Mongo. DB) (middle layer service in development) • Lattice development with My. SQL (middle layer service deployed) • Electronic log uses My. SQL (moving to Mongo. DB) (Python API allows scripts to make log entries) • Save Retrieve uses My. SQL (middle layer service deployed) • Unit conversion uses my. SQL for Magnet Measurements (deployed into an IOC application) • New meta. Data. Store uses Mongo. DB (middle layer service in development) 48 BROOKHAVEN SCIENCE
Relational Database Experience • One large database with all of the facility data was not successful. • There around thirty identified database domains. • Collaboration led by MSU with ITER, ESS, NSLS II and others to develop applications • We have serious questions to answer on cross domain queries and technology evaluations that are ongoing. • Relational databases are not the correct solution for all data storage / retrieval problems. Flexible dictionaries used for searching are not support in RDBs. 49 BROOKHAVEN SCIENCE
• • • Experiment Cntrl / DAQ / Data Management This was not in the scope of the project. However, we did analyze the problem, and do a survey of what was going on in the community. When we scaled some data collection to the frame size (8 MB) and data sets (1 kfps/set) that are expected at NSLS II and ran previous SPEC files we found that the process went from 1 minute to 30 minutes spread evenly over three areas: • Time to find the data sets that were needed for the analysis. • Movement of the data into memory. • Time to do computations Beamlines have approximately 30 K signals each In our survey, no Xray facility has demonstrated an architecture that supports the range of requirements posed by an entire facility. Everyone has their favorite analysis codes and we will never have the staff to support this for all facility needs. (Make good interfaces and release the masses) We designed a modular architecture, brought forward middle layer services from the accelerator, and found funding to support the development in whatever areas we could start. 50 BROOKHAVEN SCIENCE
Data Is Stored in Appropriate Repositories metadata Store Experime nt Log Book Scientific Data Machine Data Beamline Data The metadata store contains all configuration and environment parameters that are used to characterize a data set. Searches for specific data sets must be optimized. The Experiment Log Book contains information about the data collection It may be used to log steps in the data analysis such as export file formats used. Log entries can be made by manually or The Scientific Data is the repository for large data sets. These programmatically data sets include matrices and image sets with the metadata required to plot or view the data. These are stored by sample and time stamp. Data can be written from experiment control or submitted from analysis. Other data sets can also be sent here such as reference or model data. Machine data is all time series data that is taken from the accelerator that is available for use in analysis. Beamline data is all time series data that is taken from the end station, beam line or accelerator that is available for use in analysis. 51 BROOKHAVEN SCIENCE
Experiment Support By Construction End ØA Meta. Data. Store service is being provided to support the tracking of data – Pre Alpha Version that reduced search time from 13 minutes to 40 msec when there were 1 M frames with 120 searchable properties each. ØA Channel. Archiver is provided to store time series data from the beam lines. – Prototype service to provide network access to archive data. ØA Science Data Store is being provided to store large frame rate data sets. – Nothing ØMovement of large arrays – Zero copy implementation for EPICS, PVAccess plug-in prototype to send ND arrays over PVAccess. ØExperiment Control – Nothing. ØStandardization on Python. ØNew facilities are limited by computing infrastructure and software investments 52 BROOKHAVEN SCIENCE
Recently – Sample management/automation Samples are either viewed by their order in the sample changer dewar (Dewar. View), or by the order the user would like to collect (Priority. View). Priority The color coding for now is cyan=done, green=running, checked means requested to run when the "Collect Queue" button is pressed below the tree. 53 BROOKHAVEN SCIENCE
Exp Control, DAQ, Visualization, Analysis 54 BROOKHAVEN SCIENCE
Standard DAQ Library – Alpha release 55 BROOKHAVEN SCIENCE
Data Analysis on Vistrails – Alpha Release 56 BROOKHAVEN SCIENCE
Exp Control integration with Middle Layer 57 BROOKHAVEN SCIENCE
Visualization integrated with meta. Data. Store 58 BROOKHAVEN SCIENCE
Experimental Data Architecture Control & Analysis Clients Web Clients Data. Broker NFS CAC PVAC HTTP File Formatter Data. Broker NFS CAC PVAC Control System Studio with Correlation Plots Channel Archiver View PVManager CAC PVAC Ethernet PVAS Channel SQL* Scienc e Data N-lanes Detector PVAS Experiment Information. Archive Retrieval XML/RPC SQL Mongo. DB SQL* Scienc e Data PAS S Meta. Dat a Store Archive Store/Retrie ve Finder Server RDB REST Experiment Information. PVAS NFS CAS PVAS CAS OLO G XML/RPC Machin e Data Beamlin e Data PVAS Process Database. Device Support Driver Area Detector Driver Instrumentation 59 BROOKHAVEN SCIENCE
Conclusions (1/2) • NSLS II has leveraged off of years of development from the community: • We have invested in several areas of development (3. 15 extensions, V 4, CSStudio, middle layer services, fast orbit feedback hardware) and most of them greatly improved functionality and productivity. • We are starting to deploy these services for data acquisition and have been able to reuse several of the services. • Setbacks/oversights were uncovered painlessly with cautious commissioning plans. They were used to productively resolve the issue and uncover any related issues. 60 BROOKHAVEN SCIENCE
Conclusions (1/2) • There are several key areas that need development in the future: PVAccess as the primary protocol for EPICS, CSStudio data layer maturation, and an open-source hardware platform. • Scientists are unable to do science yet as serious scope that was required was not in the scope of the project nor in the plan for commissioning and operations. • Operations: Feb, through the end of Mar) 842 hrs scheduled, 106. 4 hrs down, 1. 92 hrs (in 2 faults) charged to controls. (network problem accounted for most) 61 BROOKHAVEN SCIENCE
- Slides: 61