LUSI Experiment Needs Yiping Feng Fluctuations Diagnostic suite

LUSI Experiment Needs Yiping Feng Fluctuations Diagnostic suite Intensity Spatial Temporal Spectral Large 2 -dim detectors Sophisticated DAQ Data storage Real time processing LUSI DOE Review July 23, 2007 Breakout Presentations 1 Yiping Feng yfeng@slac. stanford. edu

Electron Beam Characteristics X-ray Free-Electron Laser (FEL) is fundamentally different from storage-ring based synchrotron sources. For the LCLS photo-injection at 120 Hz at LCLS Each macro electron bunch is different at origination Different timing, length, density After passing through the Linac including acceleration and compression Added difference in timing, length, density Additional difference in energy, orbit Orbit correction less effective due to low repetition rate in contrast to synchrotron sources (revolution frequency at APS = 272 k. Hz) LUSI DOE Review July 23, 2007 Breakout Presentations 2 Yiping Feng yfeng@slac. stanford. edu

X-ray Beam Characteristics X-ray amplification process based on self-seeding SASE* Lasing starts from a random electron density distribution Each X-ray pulse consists of a random time sequence of spikes of varying degrees of saturation è X-ray FEL exhibits inherent Intensity, spatial, temporal, and spectral fluctuations on pulse by pulse basis *Self Amplification of Spontaneous Emission LUSI DOE Review July 23, 2007 Breakout Presentations 3 Yiping Feng yfeng@slac. stanford. edu

Expected Fluctuations of LCLS FEL pulses Parameter Pulse intensity fluctuation Value ~ 30 % Origin* Varying # of FEL producing SASE spikes; 100% intensity fluctuation/perspike; etc. Position & pointing jitter (x, y, ~ 25 % of beam diameter a, b) ~ 25 % of beam divergence Varying trajectory per pulse; Saturation at different locations of b-tron curvature Source point jitter (z) ~5 m SASE process reaching saturation at different z-points in undulator X-ray pulse timing (arrival time) jitter ~ 1 ps FWHM Timing jitter btw injection laser and RF; Varying e-energy per-pulse X-ray pulse width variation ~ 15 % Varying e-energy leading to varying path (compression) in bunch compressors Center wavelength variation ~ 0. 2 % (comparable to FEL bandwidth) Varying e-energy leading to varying FEL fundamental wavelength and higher order *To be discussed in details now LUSI DOE Review July 23, 2007 Breakout Presentations 4 Yiping Feng yfeng@slac. stanford. edu

Expected Fluctuations of LCLS FEL pulses Parameter Pulse intensity fluctuation Value ~ 30 % Origin* Varying # of FEL producing SASE spikes; 100% intensity fluctuation/perspike; etc. Position & pointing jitter (x, y, ~ 25 % of beam diameter a, b) ~ 25 % of beam divergence Varying trajectory per pulse; Saturation at different locations of b-tron curvature Source point jitter (z) ~5 m SASE process reaching saturation at different z-points in undulator X-ray pulse timing (arrival time) jitter ~ 1 ps FWHM Timing jitter btw injection laser and RF; Varying e-energy per-pulse X-ray pulse width variation ~ 15 % Varying e-energy leading to varying path (compression) in bunch compressors Center wavelength variation ~ 0. 2 % (comparable to FEL bandwidth) Varying e-energy leading to varying FEL fundamental wavelength and higher order *To be discussed in details now LUSI DOE Review July 23, 2007 Breakout Presentations 5 Yiping Feng yfeng@slac. stanford. edu

Transverse Jitter Steering coil power supply regulation Quadrupole magnet transverse vibrations Quadrupole magnet power supply regulation in presence of typical 200 -mm transverse misalignment RF structure wakefields with varying charge and typical 200 -mm transverse misalignments CSR in bunch compressor chicanes with varying bunch length LUSI DOE Review July 23, 2007 Breakout Presentations 6 Yiping Feng yfeng@slac. stanford. edu

Spatial Jitter Transverse Stability Orbit-1 a Orbit-1 b e- beam X-ray beam Orbit-2 a Orbit-2 b Pointing Stability LUSI DOE Review July 23, 2007 Breakout Presentations 7 Yiping Feng yfeng@slac. stanford. edu

Expected Fluctuations of LCLS FEL pulses Parameter Pulse intensity fluctuation Value ~ 30 % Origin* Varying # of FEL producing SASE spikes; 100% intensity fluctuation/perspike; etc. Position & pointing jitter (x, y, ~ 25 % of beam diameter a, b) ~ 25 % of beam divergence Varying trajectory per pulse; Saturation at different locations of b-tron curvature Source point jitter (z) ~5 m SASE process reaching saturation at different z-points in undulator X-ray pulse timing (arrival time) jitter ~ 1 ps FWHM Timing jitter btw injection laser and RF; Varying e-energy per-pulse X-ray pulse width variation ~ 15 % Varying e-energy leading to varying path (compression) in bunch compressors Center wavelength variation ~ 0. 2 % (comparable to FEL bandwidth) Varying e-energy leading to varying FEL fundamental wavelength and higher order *To be discussed in details now LUSI DOE Review July 23, 2007 Breakout Presentations 8 Yiping Feng yfeng@slac. stanford. edu

Z-Jitter Dz DR 2 = DZ R 1 R 2 DR 2 = DR 1(R 2/R 1)2 LUSI DOE Review July 23, 2007 Breakout Presentations 9 Yiping Feng yfeng@slac. stanford. edu

Expected Fluctuations of LCLS FEL pulses Parameter Pulse intensity fluctuation Value ~ 30 % Origin* Varying # of FEL producing SASE spikes; 100% intensity fluctuation/perspike; etc. Position & pointing jitter (x, y, ~ 25 % of beam diameter a, b) ~ 25 % of beam divergence Varying trajectory per pulse; Saturation at different locations of b-tron curvature Source point jitter (z) ~5 m SASE process reaching saturation at different z-points in undulator X-ray pulse timing (arrival time) jitter ~ 1 ps FWHM Timing jitter btw injection laser and RF; Varying e-energy per-pulse X-ray pulse width variation ~ 15 % Varying e-energy leading to varying path (compression) in bunch compressors Center wavelength variation ~ 0. 2 % (comparable to FEL bandwidth) Varying e-energy leading to varying FEL fundamental wavelength and higher order *To be discussed in details now LUSI DOE Review July 23, 2007 Breakout Presentations 10 Yiping Feng yfeng@slac. stanford. edu

Expected Fluctuations of LCLS FEL pulses Parameter Pulse intensity fluctuation Value ~ 30 % Origin* Varying # of FEL producing SASE spikes; 100% intensity fluctuation/perspike; etc. Position & pointing jitter (x, y, ~ 25 % of beam diameter a, b) ~ 25 % of beam divergence Varying trajectory per pulse; Saturation at different locations of b-tron curvature Source point jitter (z) ~5 m SASE process reaching saturation at different z-points in undulator X-ray pulse timing (arrival time) jitter ~ 1 ps FWHM Timing jitter btw injection laser and RF; Varying e-energy per-pulse X-ray pulse width variation ~ 15 % Varying e-energy leading to varying path (compression) in bunch compressors Center wavelength variation ~ 0. 2 % (comparable to FEL bandwidth) Varying e-energy leading to varying FEL fundamental wavelength and higher order *To be discussed in details now LUSI DOE Review July 23, 2007 Breakout Presentations 11 Yiping Feng yfeng@slac. stanford. edu

Goals X-ray diagnostics are required to measure these fluctuations since they can’t be eliminated Integral parts of Instruments Timing & intensity measurements for XPP experiments Wave-front characterization for CXI experiments Measurements made on pulse-by-pulse basis Requiring real-time processing by controls/data systems Commonalities in needs & specs Standardized and used for all applicable instruments Modularized for greater flexibility of deployment and placement è Critical diagnostics must be performed and data made available on pulse-by-pulse basis LUSI DOE Review July 23, 2007 Breakout Presentations 12 Yiping Feng yfeng@slac. stanford. edu

Fluctuations, 120 hz pulse rate drive DAQ requirements The 120 Hz per-pulse data collection/reduction, high data rate, large data volume, and sub-ps timing control requirements of LCLS experiments go far beyond those at existing synchrotron sources, requiring considerable complexity and sophistication in controls and data systems’ design, implementation, and integration that are not feasible for individual experimental teams LCLS/LUSI controls and data systems must Provide standard controls to all instruments Support diagnostic measurements Provide standard data acquisition capabilities Provide standard data storage and management capabilities Provide certain standard data analysis capabilities LUSI DOE Review July 23, 2007 Breakout Presentations 13 Yiping Feng yfeng@slac. stanford. edu

Data system requirements Data acquisition Real-time data processing Quick view Data management On-line storage Long term archiving/retrieval Data analysis Volume rendering visualization LUSI DOE Review July 23, 2007 Breakout Presentations 14 Yiping Feng yfeng@slac. stanford. edu

2 D Detectors Cornell (PAD) BNL(XAMPS) Technology diode/ASIC Architecture Bump-bond integrated readout pixel column size 190 x 32 1024 x 1024 Data rate 1. 9 Gb/s 1. 5 Gb/s Resolution 14 bit 12 bit Plug-play with a common interface? LUSI DOE Review July 23, 2007 Breakout Presentations 15 Yiping Feng yfeng@slac. stanford. edu

Data rates - CXI high peak rate & large volume comparable to high-energy physics experiments such as Ba. Bar @ SLAC Data Rate/Volume of CXI Experiment (comparable to other experiments) LCLS Pulse Rep Rate (Hz) 120 Detector Size (Megapixel) 1. 2 Intensity Depth (bit) 14 Success Rate (%) 30% Ave. Data Rate (Gigabit/s) 0. 6 Peak Data Rate (Gigabit/s) 1. 9 Daily Duty Cycle (%) Accu. for 1 station (TB/day) Is it possible to perform real-time data analysis to reduce the data rate? 50% 3. 1 require high performance and high capacity data acquisition and management system LUSI DOE Review July 23, 2007 Breakout Presentations 16 Yiping Feng yfeng@slac. stanford. edu

Long term data storage needs Year 2009 - 2012 - 2015 - Rep Rate (Hz) 120 120 Detector Size (Megapixel) 0. 58 1. 16 Projected 5. 8 Intensity Depth (bit) 14 14 14 Success Rate (%) 10% 30% 50% Ave. Data Rate (Gigabit/s) 0. 1 0. 58 4. 9 Peak Data Rate (Gigabit/s) 0. 97 1. 94 9. 8 Daily Duty Cycle (%) 25% 50% 75% Accu. for 1 station (TB/day) 0. 26 3. 14 39 Accu. for 3 stations (TB/day) 0. 80 9. 4 118 Yearly Uptime (%) 25% 50% 75% Accu. (Petabyte/year) 0. 072 1. 7 32 Duration/Lifetime (year) 3 3 3 Total Accu. (Petabyte) 0. 22 5. 2 97 LUSI DOE Review July 23, 2007 Breakout Presentations 17 Yiping Feng yfeng@slac. stanford. edu

Overall data needs Per pulse data collection Experimental Diagnostic – EO signal, e- and g beam parameters Raw data rate and volume 2 Gb/sec or higher On-line storage capacity - 20 TB/day Timing/Triggering EO timing measurement < 1 ps Detector trigger < 1 ms Real time analysis Frame correction, quality control To the extent possible - binning, sparsification, FFT Quick view Quasi real-time feedback, 5 frame/s Alignment Data Management Unified data model Archiving capacity – 5 PB/year Analysis staging storage capacity – 20 TB Offline Analysis > 1000 node cluster LUSI DOE Review July 23, 2007 Breakout Presentations 18 Yiping Feng yfeng@slac. stanford. edu

Applications needs User programs Endstation operation Calibration Alignment Interface to SW for diffraction/scattering experiments SPEC Interface to instrumentation/analysis SW Mat. Lab. View User tools STRIP tool ALARM Handler LUSI DOE Review July 23, 2007 Breakout Presentations 19 Yiping Feng yfeng@slac. stanford. edu

Pieces of the Puzzle LUSI Control & Data System EO Timing & Triggering Feedback Pulse-by-pulse info exchange Control Subsystem for Operation & Controls LCLS Control System Controls for RF/Undulator High peak rate/ large volume Data Archiving/Retrieval Offline Analysis/Rendering Data Subsystem for Acquisition & Management SLAC Sci. Computing & Computing Srvs. Data Farm (PB Tape Drive) Computer Cluster (2000 processor Node) LUSI DOE Review July 23, 2007 Breakout Presentations 20 Yiping Feng yfeng@slac. stanford. edu

LUSI DOE Review July 23, 2007 Breakout Presentations 21 Yiping Feng yfeng@slac. stanford. edu

LCLS FEL Parameters *Courtesy of Z. Huang LUSI DOE Review July 23, 2007 Breakout Presentations 22 Yiping Feng yfeng@slac. stanford. edu

LCLS Accelerator Schematics* 6 Me. V z 0. 83 mm 0. 05 % 4. 30 Ge. V z 0. 022 mm 0. 71 % 13. 6 Ge. V z 0. 022 mm 0. 01 % Linac-X L =0. 6 m rf= -160 Linac-0 L =6 m rf gun 250 Me. V z 0. 19 mm 1. 6 % 135 Me. V z 0. 83 mm 0. 10 % b , -a L 0 Linac-1 L 9 m rf -25° . . . existing linac 21 -1 b, c, d DL 1 L 12 m R 56 0 Linac-2 L 330 m rf -41° X BC 1 L 6 m R 56 -39 mm Commission in Jan. 2007 21 -3 b 24 -6 d BC 2 L 22 m R 56 -25 mm Linac-3 L 550 m rf 0° 25 -1 a 30 -8 c Commission in Jan. 2008 SLAC linac tunnel undulator L =130 m DL 2 L =275 m R 56 0 research yard *Courtesy of P. Emma LUSI DOE Review July 23, 2007 Breakout Presentations 23 Yiping Feng yfeng@slac. stanford. edu

Micro-bunching & SASE Process Micro-bunched Shot-noise *Courtesy of Z. Huang LUSI DOE Review July 23, 2007 Breakout Presentations 24 Yiping Feng yfeng@slac. stanford. edu

Temporal Characteristic *Courtesy of Z. Huang DT = 100 - 200 fs 200 -300 attosecond tc ~ 200 as M = DT/tc ~ 500 -1000 For ideal e-beam of equal bunch length and same energy <DI/I> ~ 1/√M ~ 5% In reality <DI/I> ~ 1/√M ~ 30% LUSI DOE Review July 23, 2007 Breakout Presentations 25 Yiping Feng yfeng@slac. stanford. edu

Magnetic Bunch Compression DE/E z 0 ‘chirp’ z …or overcompressi on DE/E z E / E V = V 0 sin(wt) RF Accelerating Voltage LUSI DOE Review July 23, 2007 Breakout Presentations 26 undercompressi on z z Dz = R 56 DE/E Path Length-Energy Dependent Beamline Yiping Feng yfeng@slac. stanford. edu

Timing Jitter 6 Me. V z 0. 83 mm 0. 05 % 4. 30 Ge. V z 0. 022 mm 0. 71 % 13. 6 Ge. V z 0. 022 mm 0. 01 % Linac-X L =0. 6 m rf= -160 Linac-0 L =6 m rf gun 250 Me. V z 0. 19 mm 1. 6 % 135 Me. V z 0. 83 mm 0. 10 % b , -a L 0 Linac-1 L 9 m rf -25° . . . existing linac 21 -1 b, c, d DL 1 L 12 m R 56 0 Linac-2 L 330 m rf -41° X BC 1 L 6 m R 56 -39 mm Commission in Jan. 2007 21 -3 b 24 -6 d BC 2 L 22 m R 56 -25 mm Linac-3 L 550 m rf 0° 25 -1 a 30 -8 c Commission in Jan. 2008 SLAC linac tunnel undulator L =130 m DL 2 L =275 m R 56 0 research yard *Courtesy of P. Emma LUSI DOE Review July 23, 2007 Breakout Presentations 27 Yiping Feng yfeng@slac. stanford. edu

X-ray Diagnostics Suite Fluctuation Type Pulse intensity fluctuation Position & pointing jitter Diagnostic Device a) Pop-In Intensity Monitor b) In-Situ BPM/Intensity Monitor c) Pop-In Position/Profile Monitor In-Situ BPM/Intensity Monitor - Pointing determination from multiple BMP’s Source point jitter Focal point jitter w/ focusing optics d) Wave-front Sensor - Back-propagating from radius of curvature measurement X-ray pulse timing jitter e) Electro-Optic Sampling (EOS) Device X-ray pulse width variation EOS Device center wavelength variation LCLS e-energy calibration LUSI DOE Review July 23, 2007 Breakout Presentations 28 - Relative timing btw e-bunch & ref. probe laser - Establishes upper limit - X-ray wavelength cross-calibration is needed Yiping Feng yfeng@slac. stanford. edu

Specifications Purposes Specifications* Coarse beam alignment/monitoring; Destructive; Retractable; Dynamic range 104; Per-pulse operation at 120 Hz; Relative accuracy < 10 -2 Coarse beam alignment/monitoring Destructive; Retractable; At 50 mm resolution - 25 x 25 mm 2 field of view; At 10 mm resolution - 5 x 5 mm 2 field of view (high-resolution) Per-pulse normalization of experimental signals; High-resolution beam position monitoring Transmissive (< 5% loss); Dynamic range 106; Per-pulse operation at 120 Hz; Relative accuracy < 10 -3 In-situ Electro-optic sampling (EOS) device Measure relative timing between electron bunch (thus copropagating x-ray pulse) and a probe optical laser pulse Non-intrusive to e-beam; Non-destructive; Per-pulse operation at 120 Hz; 20 fs resolution; In-situ Wave-front sensor Characterization of wave-front; Locating focal point of focused beam Destructive; Per-pulse operation at 120 Hz; 0. 15 nm < l < 0. 3 nm Pop-in intensity monitor (moderate-resolution) Pop-in position/profile monitor In-situ BPM/Intensity monitor Technically more challenging Diagnostic Item * Must have high damage threshold LUSI DOE Review July 23, 2007 Breakout Presentations 29 Yiping Feng yfeng@slac. stanford. edu

Data Processing Goals System Specifications for Data System Data Acquisition and Management Data Analysis XPP, CXI, and XCS Instruments System Description Gunther Haller’s breakout session LUSI DOE Review July 23, 2007 Breakout Presentations 30 Yiping Feng yfeng@slac. stanford. edu

Scope - XPP Instrument time resolved scattering at < ps time resolution LUSI DOE Review July 23, 2007 Breakout Presentations 31 Yiping Feng yfeng@slac. stanford. edu

Scope - CXI Instrument structures of single molecules at near atomic resolution Molecule injection Pixelated detector (Cornell detector) Intelligent beam-stop (wave-front sensor) beam LCLS d, possibly e (focus ssed) re comp Data Processing potential particle orientation beam To mass spectrometer LUSI DOE Review July 23, 2007 Breakout Presentations 32 Readout & reconstruction Optical & x-ray diagnostics Yiping Feng yfeng@slac. stanford. edu

Scope - PCS Instrument dynamics of disordered systems at < ns time & near atomic resolutions Dat a. P roc ess ing Speckle Pattern (Iron-Aluminum Alloy) LUSI DOE Review July 23, 2007 Breakout Presentations 33 Yiping Feng yfeng@slac. stanford. edu

Diagnostics Control - Hartman Wavefront Sensor Image obtained from Imagine Optics, Ltd Measurement made far from focal plane Single shot operation 120 Hz with CCD modification 1. 5 nm and 0. 15 nm operation with customization LUSI DOE Review July 23, 2007 Breakout Presentations 34 Yiping Feng yfeng@slac. stanford. edu

Timing Control - Temporal Jitter Master Clock Electron Gun Accelerating Elements Coax RF Distribution Network Experimental Pump Laser Sources of Short Term Jitter • Coax RF distribution Network • e-beam phase to RF phase • End Station Laser phase to RF phase Limited to ~ 1 ps ! LUSI DOE Review July 23, 2007 Breakout Presentations 35 Yiping Feng yfeng@slac. stanford. edu

Timing Control - Electro-optic Sampling Stabilized Fiber Optic RF Distribution (10 fs) LBNL Gun Laser Electro-optic Sampling Laser Pump-probe Sector 20 LTU NEH Laser Temporal resolution is now limited by: 1) Our ability to phase lock the lasers to the RF 2) Intra-bunch SASE jitter LUSI DOE Review July 23, 2007 Breakout Presentations 36 Yiping Feng yfeng@slac. stanford. edu

Timing Control - SPPS Laser/X-ray Timing 100 consecutive shots LUSI DOE Review July 23, 2007 Breakout Presentations 37 Single shot, Lorentzian fit Yiping Feng yfeng@slac. stanford. edu

Real-time Processing – Binning in XPP LUSI DOE Review July 23, 2007 Breakout Presentations 38 Yiping Feng yfeng@slac. stanford. edu

Real-time processing – binning in XPP For XPP experiments using 2 D detector, Is it possible to perform real-time data analysis to reduce the data rate? • 10 Hz • Point Detector LUSI DOE Review July 23, 2007 Breakout Presentations 39 Yiping Feng yfeng@slac. stanford. edu

Real-time Processing – sorting in CXI • Diffraction from a single molecule: noisy diffraction pattern of unknown orientation single LCLS pulse unknown orientation • Combine 105 to 107 measurements into 3 D dataset: Classify/sort Real-time? Average Alignment Reconstruct by Oversampling phase retrieval The highest achievable resolution is limited by the ability to group patterns of similar orientation Gösta Huldt, Abraham Szöke, Janos Hajdu (J. Struct Biol, 2003 02 -ERD-047) LUSI DOE Review July 23, 2007 Breakout Presentations 40 Miao, Hodgson, Sayre, PNAS 98 (2001) Yiping Feng yfeng@slac. stanford. edu

Computing hardware requirements Real-time computing Power: 10 Tera-FLOPS 1000 processor cluster Memory: 10 -100 GByte RAM Bandwidth: 100 Gbit/s Integrated w/ detector or immediate downstream of detector output Data Storage/Management 10 – 100 Gbit/s links 10 – 100 on-line capacity: RAID disks, or flash memory 10 – 100 staging capacity: RAID disks, or flash memory 5 Petabyte yearly capacity ESNET connection for transferring to sister institutes Offline Analysis Total FLOPs: 1017 If analysis done in minutes: 2000 – 40000 processor cluster Large volume set rendering: 109 LUSI DOE Review July 23, 2007 Breakout Presentations 41 Yiping Feng yfeng@slac. stanford. edu