Control System of Super KEKB Hiroshi Kaji KEK
Control System of Super. KEKB Hiroshi Kaji (KEK) on behalf of the Super. KEKB control group
Contents - Introduction - Basic Components - Middle-layer Service - Dedicated Optical Network - Conclusion 2
3 The Super. KEKB Accelerators Electron-Positron Collider C. M. S. Energy: √s = 10. 58 Ge. V Luminosity: L = 8 x 1035 cm– 2 s– 1 ( Luminosity Frontier Machine ) Belle-II Main Ring Damping Ring Injector Linac Electron Positron Energy (Ge. V) 7 4 Current (A) 3. 6 2. 6
Operation Status Phase-1: 2016 March - June - Smooth start-up of accelerator - Vacuum scrubbing Phase-2: 2018 March – July - Operation of Damping Ring - Collision experiment is started. - The world’s smallest IP beam size s*y = 0. 4 mm @ b*y = 3 mm Phase-3: 2019 March - - Physics run with top-up operation - L = 1. 23 x 10 34 cm– 2 s– 1 @ b*y = 2 mm on July 1 st. - b*y = 1. 2 mm in last week, plan to squeeze more. This progressive project is strongly supported by the hard and sophisticated works of the control group. 4
Control System based on EPICS The 10000 components installed belonging to ~3 km beamline are operated with the distributed control system based on EPICS. Central Control Building There are ~200 Input/Output Controllers (IOCs) installed on the Central Control Building (CCB) and 26 sub-control buildings (SCBs). 26 sub-control buildings The control system is working properly since 2016 and we are steadily improving it with the increased requests from the operation. 5
Input Output Controller (IOC) VME platform 98 IOCs PLC platform 95 IOCs - CPU: mostly MVME 5500, MVME 4100 and MVME 6100 (EPICS v 3. 14. 8 -v 3. 14. 12) - OS: Vx. Works (real-time OS) - High-spec. and robust system - CPU: F 3 RP 61 (EPICS v 3. 14. 8 -v 3. 14. 12) F 3 RP 71 (EPICS v 3. 15. 6) - OS: Linux - Small size, Low price, low power consumption Others - c. PCI Event Receiver IOC - micro. TCA for LLRF - Software IOC as the middle layer system 6
Operation Interface (OPI) SAD and Python scripts - Tk toolkit for GUI - Channel Access functions - Optics calculation, tracking, . . . Control System Studio (CSS) BOY - All-in-one application for accelerator control 7
Server and Console Control Server - OS: Linux (and one HP-UX) - 14 blade servers - 10 rack-mount servers - Software IOCs - Middle-layser services - OPI based on Python Operation Server (SAD cluster) - OS: Free. BSD - 8 servers for operation and offline - SAD script services - OPI based on SAD - tracking/simulation Operation Console on CCB - on Central Control Building - Linux as X-terminal - OPI based on CSS Mac mini - on each SCB - for maintenance works at the local site - network boot 8
Network The control system network is separated from the KEK office network with the fire-wall. Configuration 2 core switches at Central Control Building 40 edge switch at every Sub-Control Building Redundant network based on Virtual Switching System - Set of 10 Gb. E and 1 Gb. E-backup - Partially, 1 Gb. E and 1 Gb. E, link aggregation Leaky Coaxial cables in the entire accelerator tunnel VLAN segmentation EPICS VLAN + three Hardware VLANs (for protecting the hardware from the large EPICS broadcast) ― EPICS VLAN ― Device 1 VLAN ― Device 2 VLAN 9
10 Middle-layer Service There are middle-layer services which support the smooth beam operations. Operator Middle-layer Service IOC IOC ICT Monitoring System S. Sasaki, et al. , in Proc. PASJ’ 19 , Kyoto, Japan, Aug. 2016, pp. 596 -599. S. Sasaki, et al. , in Proc. ICALEPCS’ 19 , New York, NY, USA, Oct. 2019, WEPHA 134. Alarm System T. Nakamura, et al. , in Proc. PASJ’ 16 , Chiba, Japan, Aug. 2016, pp. 1159 -1162. Electronic Log System K. Yoshii, et al. , in Proc. 11 th ICALEPCS’ 07 , Oak Ridge, TN, USA, Oct. 2007, WOAB 04. Data Archiver System T. T. Nakamura, et al. , in Proc. ICALEPCS’ 05 , Geneva, Switzerland, Oct. 2005, P 01. 077 -7. A. Morita, in Proc. PASJ’ 16 , Chiba, Japan, Aug. 2016, pp. 1150 -1153.
ICT Monitoring System The ICT monitoring system is economically developed with open-source software. The status of the control system can be checked from everywhere via web-browser. Metrics monitoring by Zabbix: - Status of servers, IOCs, and network switches are monitored. - Data are visualized with Grafana. - EPICS Process Variable (PV) monitoring the data are collected via pv. Access RPC of EPICS 7. Log monitoring by Elastic stack: - Broadcast monitoring The “ps”, “ca. Snooper”, “casw” commands are periodically implemented and their logs are recorded with Elastic search. - Data are visualized with Kibana. Network Monitoring Server Monitoring IOC Monitoring The CA packet of EPICS is monitored to investigate the anomaly of IOC. 11
Alarm System The CSS alarm system monitors ~15, 000 PVs of the Super. KEKB control system. The “major” or “minor” statuses are defined for individual PVs. They are stored on the severity field. In our operation policy, if one PV shows the “major” alarm, the expert is called and must clear the problem even though it is mid-night. 12
Electronic Log The Zlog system is developed with the Python-based open-source, Zope. It is a web-based service so that all users can easily access to read/write the operation log. All standard OPIs have the snapshot function. The operators take a snapshot when they implement some important operations or when the alarm status is popped up. 13
Data Archiver System Our archive system collects and records the information of the EPICS PV. KEKBLog: maintained by the control group - House-made archiver since the KEKB project - All necessary information (130, 000 PVs) for accelerator operation is recorded. - NAS is increased as needed. - Cache server based on SSD CSS Archiver: freely maintained by all users - Archiver for the particular usages We plan to integrate the above two systems by Archiver Appliance. The full service will be available since this Autumn run. The new retrieve functions are developed with ROOT. 14
Dedicated Optical Network The dedicated optical network systems realize the fast, robust, and complicated operations. Beam Permission H. Kaji, et al. , in Proc. IPAC 18 , Vancouver, Canada, Apr. /May 2018, WEPAK 015. Abort Trigger System S. Sasaki, et al. , in Proc. ICALEPCS’ 15 , Melbourne, Australia, Oct. 2015, MOPGF 141. Timing System H. Kaji et al. , in Proc. ICALEPCS’ 13 , San Francisco, CA, USA, Oct. 2013, THCOCA 04. H. Sugimura et al. , in Proc. ICALEPCS’ 17 , Barcelona, Spain, Oct. 2017, TUCPL 02. H. Kaji et al. , in Proc. PASJ’ 18 , Nagaoka, Japan, Aug. 2018, pp. 1114 -1116. H. Kaji et al. , in Proc. PASJ’ 18 , Nagaoka, Japan, Aug. 2018, pp. 1117 -1119. 15
Beam Permission System enables the accelerator operation when the proper condition is satisfied. The permission signal is delivered from DR and MRs to LINAC. The entire circuit i s configured with AND/OR logic. The AND logic provides the permission signal when the all necessary statuses (inputs) are OK. The OR logic enables to establish the multiple operation modes. 16
Abort Trigger System collects and transfers ~260 inputs as the abort request signals. Then it implement s the abort kicker magnet to throw the beam from MR. There is the t imestamp function to distinguish the order of inputs. Each Sub-Control Building Central Control Building Total response time ~10 ms module latency(100 ns) + cable The system is working without any serious problem since 2016. The following upgrades are carried out. - Increase the abort request signals from the IP hardware. - Rewired from the CCB module to the abort kicker for reducing the response time. - Beam operation with two abort gaps filling pattern. 17
18 Timing System The timing system controls the beam-injection operation. The complicated operations of LINAC and DR are scheduled by Bucket Selection and implemented by Event Timing System. The 1 st and 2 nd halves of LINAC are operated separately when we carry out the positron injection. “positron damping time (at least 40 ms)” > “LINAC operation interval (20 ms)” The two-layers configuration of Event Generators (EVGs) are developed for satisfying the requirements from the DR operation. e− Gun Successfully operated since 2018 ARC e+ BT (Super. KEKB: 4 Ge. V, 4 n. C) e+ Target e– BT (Super. KEKB: 7 Ge. V, 5 n. C) Further upgrade is required for the two pulses operation at DR. ⇒ Event Timing System controls the RF phase at LINAC in 50 Hz.
EVR Development Project The standalone EVR is developed with open source. The SINAP-type standalone module is upgraded. The Data Buffer (DB) and Distributed Bus Bit (Dbus) functions are integrated. Individual outputs can be controlled with them. - enable/disable by Dbus - delay setting by DB Three modules are installed and operated in our timing system. Standalone EVR IOC with Zynq plan to run EPICS on embedded CPU More complicated operations become possible via the EPICS process. AES-PZCC-FMC-V 2 -G 19
Conclusion Super. KEKB has been operated since 2016 and smoothly accomplish its milestones. We developed a sophisticated control system to support this project. The distributed control system is constructed based on EPICS. - There are ~200 IOCs, mostly built with VME or PLC. - OPI is developed with SAD/Python script and CSS BOY. Several useful middle-layer services are developed/provided with open-source. - Monitoring system: Grafana, Zabbix, Elastic search - Alarm system: CSS alarm - Archiver: KEKBLog, CSS archiver, Archiver Appliance - Electronic log: Zope Dedicated optical network is configured for followings: - Abort Trigger System - Beam Permission System - Timing system The future prospect of the Super. KEKB project is promising since we are steadily upgrading the control system for the increased requests from the operations. 20
Machine Parameters Energy (electron/positron) KEKB achieved Super. KEKB design 8. 0/3. 5 Ge. V 7. 0/4. 0 Ge. V Current (electron/positron) 1. 6/1. 2 A 3. 6/2. 6 A Crossing angle 22 mrad 83 mrad Number of bunches 1584 2500 ex 24/18 nm 1. 7/3. 2 nm by * 5. 9/5. 9 mm 0. 42/0. 27 mm xy 0. 09/0. 13 0. 09/0. 09 sy 0. 94 mm 0. 059 mm sz ~6 mm 5/6 mm Luminosity 2. 11× 1034 cm– 2 s– 1 8× 1035 cm– 2 s– 1 21
s*y history 22
Super. KEKB IOCs In addition. . . - some m. TCA IOCs for LLRF - 4 oscilloscope(windows) IOCs for BT BPMs - standalone EVRs 23
Timing System based on the Event Timing System - launches timing-triggers to the accelerator hardware - launches interruptions at remote IOCs to change the operation parameters. Pulse-to-pulse modulation more than 150 LINAC parameters are switched in 50 Hz for the simultaneous top-up operations at 4 rings. 24
Monitoring System with RPC In addition, we utilize the Remote Procedure Call (RPC) of the EPICS 7 protocol. It enables us to collect more complicated information. The arbitrary constructor of the EPICS PV information can be transferred. Alarm monitoring: The data protocol includes the “severity” and “status” fields. Abort signal sorting: The data protocol includes the “ time” field. 25
26 Overview KEKB rev/49/23/2 TDC EVG EVR BS delay #42 #32 #41 Set EVG PF PF(PFAR) Set EVG 3. 5 ms Set EVG AC 50 #62 #31 #61 #42 #41 #32 EVR #181 Upper-EVG sequence EVG
27 IOC Main Trigger Station Operate LINAC. IOC D 7 IOC KEKB control -1 Bunch Current Measurement. Control injection RFbucket. RM CPU RM BCM e+ BCM e– CPU RM EVG EVR EVG CPU Optical Cable
Idea of Luminosity Upgrade 2 times increase 20 times suppression of Twiss parameter: by Note the condition is satisfied if the beam and accelerator optics are well matched. 5. 9 mm (KEKB) ⇒ 0. 42 mm (Super. KEKB) for electron 5. 9 mm ⇒ 0. 27 mm for positron 2 times larger beam current: I 1. 64 A (KEKB) 1. 19 A ⇒ 3. 6 A (Super. KEKB) for electron ⇒ 2. 6 A for positron 29
Visualization of pv. Access data We have developed Grafana datasource plugin and HTTP / pv. Access API gateway to visualize arbitrary data. These applications allow to visualize the data from pv. Ac-cess RPC servers on Grafana. 30
31 Overview KEKB revolution/49/23/2 KEKB rev/49/23/2 TDC EVG #42 #32 #41 BS delay There is no swapping of order among the red arrow signals. There is no swapping of order among the green arrow signals. However the swapping is possible between red and green signals. Set EVG PF PF(PFAR) Set EVG 3. 5 ms Set EVG AC 50 Hz #62 #31 #61 #42 #41 #32 EVR #181 Upper-EVG sequence EVG
- Slides: 31