HEPS Machine Learning for AcceleratorsHigh Energy Photon Source

HEPS Machine Learning for Accelerators/High Energy Photon Source (HEPS) Paul Chu Institute of High Energy Physics, Chinese Academy of Sciences

Outline 1. Introduction 2. Software Architecture for HEPS ML 3. Preparation for Machine Learning 4. ML Application Examples 5. Summary

Introduction p. HEPS– 4 th generation synchrotron light source, 7 BA-lattice p. Construction period – Jun. 2019 – Dec. 2025, ~US$700 M HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

User Facility Software Architecture (courtesy F. Z. Qi) File Transfer DAQ Temp Storage User Office& Scientific Committee Local Analysis Raw Data Repository Proposal Potential Users Computing Service DAQ Detector Scheduling Online Data Processing Online Data Analysis Users Sample Radiational Instrument Sample Source Environment Publication User Proposal Sample Database Pre-Experiments Ctrl Scientific Metadata Local Users& Instrument Scientists Administrative Metadata Experiments Duration Metadata Catalogue Data Service Data Management Remote Analysis Post-Experiments HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

Central Message Logging System p. Recording every action occurred in the system p. CMLog data format p. CMLog database（no. SQL） p. C/C++/Java/Python API p. CMLog client viewer p. Can serve for MPS postmortem analysis HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

High-level Application Architecture § Collecting as much data as possible § Central messaging logging § Operation logging § … § Data Pre-processing § Clean up § Line up n Programming applications with 3 categories of APIs n Software re-usability, cut development time n General-purpose, physics, and Machine Learning (ML) n Standard data formats & popular algorithms Data Science and Machine Learning Workshop, Oct. 6, 2019 HEPS

Machine Learning Platform p. Getting data p. Pre-processing data p. Applying algorithms p. Displaying results p. Applying results/predictions HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

ML Platform General Ideas Machine Learning in Python Scikit-learn/Tensor. Flow n Simple and efficient tool for data mining & data analysis n Built on Num. Py, Sci. Py, and matplotlib n Open source, commercially usable - BSD license Machine Learning in MATLAB Machine Learning Toolbox HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

Data Preparation Example for Machine Learning p. Data Collection ØArchived accelerator data ØExperiment data from Data Acquisition (DAQ) systems ØPVLogger (EPICS based) or time synchronized meta data n n n Define own EPICS PV list, logging period easily Aligned timestamp PVs My. SQL DB Periodic logging On-demand logging HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

Data Handling Data Sources n EPICS live data n TXT/Excel Files n EPICS Channel Archiver n EPISC Archiver Appliance n Other data sources (e. g. Code Snippet PVLogger) Output Data Format n Pandas Data. Frame n TXT/Excel Files n Other format: HDFS pvnames=['BIBPM: R 1 OBPM 02: XPOS', 'BIBPM: R 1 OBPM 03: XPOS', 'BIBPM: R 1 OBPM 04: XPOS] #also can load pvnames from files engine=Load. Data. get. Key(server_addr, pvnames) data=Load. Data. get. Format. Chan. Arch(server_addr, engine, pvnames, start_time='11/30/2018 14: 15: 00’, end_time='11/30/2018 14: 16: 00’, merge_type='outer’, interpolation_type='linear’, fillna_type=None, how=0) Data Science and Machine Learning Workshop, Oct. 6, 2019 HEPS

Algorithms p. Regression ØLinear Regression ØBayesian Linear Regression ØPolynomial Regression p. Decision Tree p. K-Nearest Neighbors p. Clustering ØK-Means ØDBSCAN p. Multi-layer Perceptron (MLP) p… Data Science and Machine Learning Workshop, Oct. 6, 2019 HEPS

ML for HEPS Lattice Design [1] p. Using DNN for HEPS lattice design ØHighly nonlinear model ØApplying HEPS optimized lattice data ØOptimize brightness (BN) and dynamic aperture (DA) HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

ML for HEPS Lattice Design [2] p. The DNN method can predict the BN close to 100% and DA over 95% at 80 th generation Single thread (s) 62 -thread parallel computing (s) DNN 0. 3944 0. 0092 Particle Tracking 78020 1414. 2 Improvement O(5) DNN is 5 Orders better efficiency than Particle Tracking! HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

Machine Learning at Work p. A test for BEPC-II timestamp correction ØCorrelation function ØObjective function f 1(t)&f 2(t) : The relation between 'value' and 'timestamp' of two systems(such as correctors with BPM). h(): Projection of one group of value to another. ζ(): Integral coefficient. (Remove interference and noise. Keep normalization) HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

Accelerator Intelligent Control System [1] p. AICtrl ØCould be applied to many conditions, such as beam loss reduction, luminosity optimization and so on ØBased on deep learning and reinforcement learning, it is better to find good condition with historical data then merely manual tuning Application on luminosity optimization HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

Accelerator Intelligent Control System [2] p. Application on luminosity optimization by adjusting vertical beam position offset ØComparing with manual tuning, AICtrl can make luminosity always in a good state ØStrange behavior could be avoided by simply turning on the watchdog HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

Next Steps p. An Application “Template” p. Setting up virtual accelerator environment for ML App tests p. Setting up Hadoop environment p. Setting up GPU computing p. Getting more operation data HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

Summary p. HEPS Control/Data Systems designed for ML p. Overall consideration, modularized implementation p. Machine Learning for Accelerator Platform prototyped p. Many application ideas are emerging p. Collaborations are welcome Thanks for your attention! HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

Machine Learning Platform Source Code Repository phttps: //github. com/Nicole. Qiao/MLPlatform HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

Backup Slides HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

Controls and Online/Offline Software p. Databases: online/offline p. Control systems p. Physics Modeling p. Services p. API Øcommonly used modules p. Online Applications p. Offline applications ØLattice Design w/ ML HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

Databases p 17 database domains in plan p. My. SQL or Microsoft Share. Point/SQL Parameter List Logbook and Issue Tracking Cable Naming Convention Maintenance/Operation Security Magnet Inventory Alarm Accelerator Model/Lattice Survey and Alignment Machine Protection/Interlock Equipment and Configuration Work Flow Control/Traveler MPS Postmortem Physics Data and Machine State Document DB Data Science and Machine Learning Workshop, Oct. 6, 2019 HEPS

Data Export & Unification [1] Raw data characteristics n Large amount of PVs as model features n Different PV has different acquisition period n Handling null or abnormal data Data Merging Timestamp Alignment Data Unification Pandas Data. Frame HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

Data Export & Unification [2] For temporal data from archiver PV Timestamp alignment 3 data line-up types: 1. Outer -> smallest time period -> data addition 2. Inner -> biggest time period -> data deletion 3. Defined time period -> data addition & deletion n Standardization n Normalization n Discretization (quantization or binning) n Encoding categorical features HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

Data Pre-processing [1] Data quality check Handling empty, abnormal, inconsistent data n Padding Not changing over time Bad machine status n Interpolation Linear, nearest, polynomial, cubic, spline…… n Neural network Predict uncertain data through NN algorithm based on known data HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

Data Pre-processing [2] n Data feature analysis n Diagnostic functions n distribution analysis n common statistical indicators n comparative analysis n histograms n periodic analysis n scatter matrix diagrams n contribution analysis n correlation tables & n correlation analysis associated heat maps n box plots HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

Where Can ML Be for Accelerators p. Basically, everywhere… p. Best fit: non-linear issues with sufficient and good quality data p. Entire accelerator life cycle needs ML ØAccelerator design ØAccelerator/beamline controls ØBeam Tuning – online and offline optimization ØOperation optimization – productivity, ØMachine reliability – maintenance, ØHuman resources: who performs the best at work HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

ML Application Ideas [1] p. Facility Operation Optimization ØUse regression algorithms to improve the performance of key accelerator systems, such as high frequency cavity, superconducting system, water cooling system, etc. p. Beam Physics or experiment optimization ØApply the machine learning platform to beam physics optimization progress, to realize the automatic optimization of DA, emittance, current intensity, etc. ØData driven data acquisition p. Equipment maintenance ØTo avoid unexpected failures by analyzing equipment running data HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

ML Application Ideas [2] p. Big Data applications ØInstalling various sensors to collect as many data as possible ØData mining, data correlation, interdisciplinary data analysis p. Light source data center for data sharing p. Accelerator data archive center ØDomestic and international data archive for various accelerator related data p… HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019

Project Control (Other Data) p. Procurement, Equipment p. Issue Tracking System, Maintenance, Operation Logbook… p. Work Breakdown Structure (WBS) for project management ØCost and schedule control/monitoring p. Share. Point based document system ØProject Web site ØWork flow control ØDocument and data sharing HEPS Data Science and Machine Learning Workshop, Oct. 6, 2019