LCLS LLRF Distributed Control System Dayle Kotturi Controls

LCLS LLRF Distributed Control System Dayle Kotturi Controls Department SLAC National Accelerator Lab LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 Dayle Kotturi dayle@slac. stanford. edu

LCLS LLRF Distributed Control System • Outline – – – – Scope Global Overview General stability requirements Principal motivator Solutions Throughput measurement Conclusions Additional resources LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 2 2 Dayle Kotturi dayle@slac. stanford. edu

Scope • The low level RF controls system consists of RF phase and amplitude controls at these locations: – – – – – Laser Gun L 0 -A (a. k. a. L 0 -1) L 0 -B (a. k. a. L 0 -2) L 0 Transverse cavity L 1 -S L 1 -X L 2 – using 2 klystrons to control avg phase/ampl of L 2 L 3 Transverse cavity L 3 - here is a bit different (lots of klystrons!) LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 3 3 Dayle Kotturi dayle@slac. stanford. edu

LLRF Global Overview LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 4 4 Dayle Kotturi dayle@slac. stanford. edu

General stability requirements • For LCLS, the general RF stability requirements are: 0. 1 deg phase and 0. 1% amplitude in L 0 and L 1 for S band. LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 5 5 Dayle Kotturi dayle@slac. stanford. edu

Principal motivator • Placing the digitizers next to the low noise RF components eliminates transmission of low noise analog signals outside the chassis. LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 6 6 Dayle Kotturi dayle@slac. stanford. edu

LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 7 7 Dayle Kotturi dayle@slac. stanford. edu

LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 8 8 Dayle Kotturi dayle@slac. stanford. edu

LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 9 9 Dayle Kotturi dayle@slac. stanford. edu

LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 10 10 Dayle Kotturi dayle@slac. stanford. edu

Phase/Amplitude Detector -> VME LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 11 11 Dayle Kotturi dayle@slac. stanford. edu

OS, BSP and EPICS versions • PAD: – rtems 4. 9. 1 – m 68 k u. C 5282 – epics-R 3. 14. 10 • VME: – rtems 4. 9. 1 – powerpc beatnik (mvme 5500/mvme 6100) – epics-R 3. 14. 8. 2 • PAC: – rtems 4. 9. 1 – m 68 k u. C 5282 – epics-R 3. 14. 10 LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 12 12 Dayle Kotturi dayle@slac. stanford. edu

PAD waveform readout LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 13 13 Dayle Kotturi dayle@slac. stanford. edu

PAD IOC Stats LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 14 14 Dayle Kotturi dayle@slac. stanford. edu

VME Feedback Calculation LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 15 15 Dayle Kotturi dayle@slac. stanford. edu

VME IOC Stats LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 16 16 Dayle Kotturi dayle@slac. stanford. edu

PAC waveform control LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 17 17 Dayle Kotturi dayle@slac. stanford. edu

Throughput Time: PAD->VME>PAC LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 18 18 Dayle Kotturi dayle@slac. stanford. edu

Throughput steps: PAD->VME->PAC Event (# matches diagram) Absolute time after trigger (μsec) Description - 0. 0 Trigger delivered to phase/amplitude detector digitizer (PAD) 1 7. 2 ISR: signals “data ready” to wake up DAQ task 2 102. 0 Channel 0 readout and data processing begins (40 Is and 40 Qs) 3 583. 2 Channel 1 readout and data processing begins (40 Is and 40 Qs) 4 1041. 6 Channel 2 readout and data processing begins (40 Is and 40 Qs) 5 1501. 6 Channel 3 readout and data processing begins (40 Is and 40 Qs) 6 1963. 2 PAD starts stream of processed values to VME over private net 7 2349. 6 PAD streaming completed. VME parses, does feedback, sends. 8 2508. 0 Phase/amplitude controller (PAC) receives new setpts from VME 9 2524. 0 PAC writes to FPGA which calcs new WF to send next trigger 10 2529. 2 Data is ready LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 19 19 Dayle Kotturi dayle@slac. stanford. edu

Conclusions • At 120 Hz operation, time budget=8. 333 ms • LLRF PAD->VME->PAC throughput measured=2. 529 ms for 4 channels of 40 points each, with no offsets, – adjust: subtract 2. 5 μsec per pair of IRQ raise/lower calls (8 pairs = 20 μsec) – adjust: one socket sends to multiple PACs; add switching time LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 20 20 Dayle Kotturi dayle@slac. stanford. edu

Acknowledgements • Thanks to Ron Akre and Klystron Department for setting up hardware, scopes and signal generators • Thanks to SLAC NAL Controls Group LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 21 21 Dayle Kotturi dayle@slac. stanford. edu

Additional Information • • • Details of the PAD->VME transfer Details of the VME->PAC transfer RF stability measurement PAD Block diagram LCLS LLRF website: http: //www. slac. stanford. edu/grp/lcls/controls/global/subsystems/llrf LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 22 22 Dayle Kotturi dayle@slac. stanford. edu

Details of the PAD->VME transfer • http: //www. slac. stanford. edu/grp/lcls/controls/glo bal/sw/epics%20 team%20 meetings/prese ntations/lan. Ip. Basic. pdf • Raw ethernet packets with IP and UDP headers. Similar to BSD sockets. • Solution is for low end CPU on small LAN. • Requirement: ship 1 KB of data in ~200 μsec • VME initializes, starts and stops PAD streaming • When PAD is streaming, device support for waveform on VME parses out the values and uses them in the feedback calculations of new setpoints. LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 23 23 Dayle Kotturi dayle@slac. stanford. edu

Details of the VME->PAC transfer • On VME, a subroutine record that has calculated new setpoints calls a driver routine that sends the values to the PAC via udp socket • PAC is has thread waiting to receive packet • When packet arrives, it parses out the setpoints and puts them into mem mapped FPGA LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 24 24 Dayle Kotturi dayle@slac. stanford. edu

LCLS Jitter Specification for 2 Seconds is 0. 14% Amplitude and 0. 14 degree Phase Feedback ON 20 Second Plot shows Phase Jitter 0. 043 degrees Amplitude Jitter 0. 022% Amplitude Jitter 0. 024% Short Term RF Jitter Specification for L 0 B are well Exceeded. This is as good as it gets – Don’t tell Physicists or they will expect it. Ron Akre 2007 LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 Dayle Kotturi dayle@slac. stanford. edu

About the PAD • CPU is MCF 5282 (64 MHz) • The digitizer used is the Linear Technologies LTC 2208. It was the first 16 bit digitizer chip on the market capable of running at 119 MHz, it is specified to run up to 130 MHz. • At SLAC NAL, PAD digitizer used for RF, beam position monitors, beam charge monitors and bunch length monitors. • Pohang Light Source is also using PAD for new RF system. LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 26 26 Dayle Kotturi dayle@slac. stanford. edu

Phase/amplitude detector (PAD) Block Ron Akre LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 27 27 Dayle Kotturi dayle@slac. stanford. edu

LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 28 28 Dayle Kotturi dayle@slac. stanford. edu

LCLS LLRF Distributed Control System • Outline – – – – Scope General stability requirements Principal motivator Solutions Throughput measurement Conclusions Additional resources LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 29 29 Dayle Kotturi dayle@slac. stanford. edu

Scope • The low level RF controls system consists of RF phase and amplitude controls at these locations: – – – – – Laser Gun L 0 -A (a. k. a. L 0 -1) L 0 -B (a. k. a. L 0 -2) L 0 Transverse cavity L 1 -S L 1 -X L 2 – using 2 klystrons to control avg phase/ampl of L 2 L 3 Transverse cavity L 3 - here is a bit different (lots of klystrons!) LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 30 30 Dayle Kotturi dayle@slac. stanford. edu

General stability requirements • For LCLS, the general RF stability requirements are: 0. 1 deg phase and 0. 1% amplitude in L 0 and L 1 for S band. LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 31 31 Dayle Kotturi dayle@slac. stanford. edu

Principal motivator • Placing the digitizers next to the low noise RF components eliminates transmission of low noise analog signals outside the chassis. LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 32 32 Dayle Kotturi dayle@slac. stanford. edu

LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 33 33 Dayle Kotturi dayle@slac. stanford. edu

LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 34 34 Dayle Kotturi dayle@slac. stanford. edu

LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 35 35 Dayle Kotturi dayle@slac. stanford. edu

LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 36 36 Dayle Kotturi dayle@slac. stanford. edu

Phase/Amplitude Detector -> VME LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 37 37 Dayle Kotturi dayle@slac. stanford. edu

OS, BSP and EPICS versions • PAD: – rtems 4. 9. 1 – m 68 k u. C 5282 – epics-R 3. 14. 10 • VME: – rtems 4. 9. 1 – powerpc beatnik (mvme 5500/mvme 6100) – epics-R 3. 14. 8. 2 • PAC: – rtems 4. 9. 1 – m 68 k u. C 5282 – epics-R 3. 14. 10 LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 38 38 Dayle Kotturi dayle@slac. stanford. edu

LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 39 39 Dayle Kotturi dayle@slac. stanford. edu

LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 40 40 Dayle Kotturi dayle@slac. stanford. edu

LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 41 41 Dayle Kotturi dayle@slac. stanford. edu

LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 42 42 Dayle Kotturi dayle@slac. stanford. edu

Measuring Throughput: PAD>VME->PAC LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 43 43 Dayle Kotturi dayle@slac. stanford. edu

Throughput steps: PAD->VME->PAC Event (# matches diagram) Absolute time after trigger (μsec) Description - 0. 0 Trigger delivered to phase/amplitude detector digitizer (PAD) 1 7. 2 ISR: signals “data ready” to wake up DAQ task 2 102. 0 Channel 0 readout and data processing begins (40 Is and 40 Qs) 3 583. 2 Channel 1 readout and data processing begins (40 Is and 40 Qs) 4 1041. 6 Channel 2 readout and data processing begins (40 Is and 40 Qs) 5 1501. 6 Channel 3 readout and data processing begins (40 Is and 40 Qs) 6 1963. 2 PAD starts stream of processed values to VME over private net 7 2349. 6 PAD streaming completed. VME parses, does feedback, sends. 8 2508. 0 Phase/amplitude controller (PAC) receives new setpts from VME 9 2524. 0 PAC writes to FPGA which calcs new WF to send next trigger 10 2529. 2 Data is ready LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 44 44 Dayle Kotturi dayle@slac. stanford. edu

Conclusions • At 120 Hz operation, time budget=8. 333 ms • LLRF PAD->VME->PAC throughput measured=2. 529 ms for 4 channels of 40 points each, with no offsets, – adjust: subtract 2. 5 μsec per pair of IRQ raise/lower calls (8 pairs = 20 μsec) – adjust: one socket sends to multiple PACs; add switching time LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 45 45 Dayle Kotturi dayle@slac. stanford. edu

Acknowledgements • Thanks as always to Ron Akre and Klystron Department for setting up hardware, scopes and signal generators • Thanks to SLAC NAL Controls Group LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 46 46 Dayle Kotturi dayle@slac. stanford. edu

Additional Information • • • Details of the PAD->VME transfer Details of the VME->PAC transfer RF stability measurement PAD Block diagram LCLS LLRF website: http: //www. slac. stanford. edu/grp/lcls/contr ols/global/subsystems/llrf LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 47 47 Dayle Kotturi dayle@slac. stanford. edu

Details of the PAD->VME transfer • http: //www. slac. stanford. edu/grp/lcls/controls/glo bal/sw/epics%20 team%20 meetings/prese ntations/lan. Ip. Basic. pdf • Raw ethernet packets with IP and UDP headers. Similar to BSD sockets. • Solution is for low end CPU on small LAN. • VME initializes, starts and stops PAD streaming • When PAD is streaming, device support for waveform on VME parses out the values and uses them in the feedback calculations of new setpoints. LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 48 48 Dayle Kotturi dayle@slac. stanford. edu

Details of the VME->PAC transfer • On VME, a subroutine record that has calculated new setpoints calls a driver routine that sends the values to the PAC via udp socket • PAC is has thread waiting to receive packet • When packet arrives, it parses out the setpoints and puts them into mem mapped FPGA LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 49 49 Dayle Kotturi dayle@slac. stanford. edu

LCLS Jitter Specification for 2 Seconds is 0. 14% Amplitude and 0. 14 degree Phase Feedback ON 20 Second Plot shows Phase Jitter 0. 043 degrees Amplitude Jitter 0. 022% Amplitude Jitter 0. 024% Short Term RF Jitter Specification for L 0 B are well Exceeded. This is as good as it gets – Don’t tell Physicists or they will expect it. Ron Akre 2007 LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 Dayle Kotturi dayle@slac. stanford. edu

About the PAD • The digitizer used is the Linear Technologies LTC 2208. It was the first 16 bit digitizer chip on the market capable of running at 119 MHz, it is specified to run up to 130 MHz. • At SLAC NAL, PAD digitizer used for RF, beam position monitors, beam charge monitors and bunch length monitors. • Pohang Light Source is also using PAD for new RF system. LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 51 51 Dayle Kotturi dayle@slac. stanford. edu

Phase/amplitude detector (PAD) Block Ron Akre LCLS LLRF Distributed Control System EPICS Collaboration Meeting, Vancouver 1 May 2009 52 52 Dayle Kotturi dayle@slac. stanford. edu