FAST Beam Facility Operations Jerry Leibfritz Fermilab July
FAST Beam Facility Operations Jerry Leibfritz Fermilab July 31, 2018
Outline • • 2 Overview Progress/Plans Organization Operations Experimental Program Funding/Budget Path Forward Summary J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
FAST - Overview • The Fermilab Accelerator Science and Technology (FAST) facility is located in the NML building – 140 m accelerator test facility – contains 3 experimental areas (Electron Injector, Proton Injector, and the IOTA Ring) – Will visit during tour CMTF NML Electron Injector Proton Injector (Proton Injector RF Station) IOTA Ring 140 m 3 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
Status at previous review (2013) • • Tunnel (civil) construction complete Most accelerator components in-hand (not installed) Electron gun installed and being commissioned Cryomodule installed FAST Electron Gun FAST Accelerator Tunnel – Location of IOTA Ring 4 J. Leibfritz | 2018 DOE Fermilab GARD review FAST Cryomodule 07/31/2018
Progress since last review (2014 -2018) • Installed electron injector (140 m, including 3 SRF cryomodules) • Commissioned electron beam in stages 50 Me. V to 300 Me. V – Achieved record high beam acceleration gradient in an SRF cryomodule(31. 5 MV/m) – First ever experimental demonstration of an ILC-type beam energy gain (255 Me. V) in a single SRF cryomodule (CM 2) • IOTA Ring installation complete, ready to begin commissioning with 150 Me. V electron beam • Conducted experiments during two beam commissioning periods Low-energy injector 5 J. Leibfritz | 2018 DOE Fermilab GARD review High-energy injector IOTA Ring 07/31/2018
Future plans • 2018 - 2019 FAST Facility – IOTA commissioning with electrons, as well as 3 rd experimental period* – Install (re-locate) IOTA proton injector • 2019 - 2020 – Commission IOTA with protons and begin IOTA research program* • 2021 - beyond – Full IOTA research and FAST experimental program* (6 -months of operations per year) – Establish FAST/IOTA as a National User Facility* *research/experimental program details discussed in other talks at this review 6 J. Leibfritz | 2018 DOE Fermilab GARD review Future IOTA Proton Injector 07/31/2018
Workforce organization • Highly skilled/experienced personnel – SRF accelerator engineers and scientists – Laser, RF scientists – SRF, low-particulate, ultra-high vacuum technicians • IOTA/FAST Department comprised of scientists and research support personnel – Oversees scientific program and accelerator operations • Rely heavily on support departments – FAST Project Engineer matrixed from Mech. Support Department • Plans/organizes infrastructure installation activities and budget • Manages dedicated SRF engineering/technician group • Direct link to other support departments (electrical, controls, RF, instrumentation) – Group of support personnel dedicated to NML/CMTF test facilities • Effort is shifted around to support FAST/IOTA, LCLSII test stand, PIP 2 IT test stand – very efficient and eliminates ”Standing-Army” issues 7 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
Planning • Regular weekly meetings – Project planning meeting • Representatives from all involved groups/departments • High level discussion of progress, issues, overall schedule, etc. • Keeps all involved up to date – IOTA/FAST Department operations meeting • Scientists and Operators • Discuss detailed operations, planning, shift schedule, etc. – Engineering meeting • All engineers involved in installation/fabrication + liaison from IOTA/FAST operations meeting • Discus details of designs, fabrication, technical issues, installation • Web-based technician schedule – Daily priority list for each technician, up to 5 -day outlook, updated daily 8 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
Schedule of IOTA Construction/Commissioning (high-level) 9 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
Schedule of IOTA Construction/Commissioning (high-level) 10 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
FAST e- Injector Startup (mid-level planning) • Injector startup (6 weeks). – Systems startup. • • • Cryo cooldown. Laser tuning and beam from the gun. LLRF turn-on. Cavity conditioning. Instrumentation: Imaging stations, BPMs, MPS. – Beam commissioning (1 week). • • 11 Beam through preinjector to LEA. Phasing of CM-2 cavities. Beam propagation through HE beam line. Lattice tuning – beam matching to IOTA. J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
IOTA Commissioning – 1 (mid-level planning) • Injection device tuning (1 week). – Lambertson septum tuning to observe beam on first BPM. – Kicker timing to send beam directly though injection straight (using synchrotron radiation diagnostics in the first dipole magnet). • First turn (1 week). – Match main bus current to beam energy (using synchrotron radiation diagnostics). – Synchronize BPM electronics with beam signal. • Beam Circulation (1 week). – Starting with ‘soft’ betatron lattice away from major resonances. • Start with uncompensated chromaticity. • Gradually increase number of turns. • Main goal to establish closed orbit and stable betatron tunes. Expect fast convergence due to well-characterized magnets. – Turn on RF and scan cavity phase to synchronize injected beam with bucket. 12 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
IOTA Commissioning – 2 (mid-level planning) • Machine Tuning (2 weeks). – – Tune-up of synchrotron light diagnostics system (CCD cameras). Stable lattice with LOCO. Aperture scans with local orbit bumps to establish ‘golden’ orbit. Scrubbing run at maximum current to achieve maximum beam life time. • Transition to Experimental Operation (1 week). – Commissioning of synchronized turn-by-turn BPM operation, data collection. – Calibration of small-amplitude kicks on circulating beam. • Anticipate the schedule may stretch by up to 2 weeks due to tunnel access and downtime 13 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
Daily Shift Studies Plan (detailed planning) 14 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
Operation Efficiency - Snapshot of 2017 FAST Operations Period – 2 Months (Oct-Nov) • Primary goal – IOTA injector commissioning – – – Overall uptime 85. 6% normalized to planned hours of operation (2 shifts/day) Acceleration in CM-2 to 150 Me. V Instrumentation Machine Protection System Machine stability and reproducibility Beam line tuning for injection to IOTA Downtime contributions • Achievement of 300 Me. V beam in SRF linac – Conditioning and phasing of cavities – Low-Level RF tuning • Collaboration-driven experimental program 15 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
Operations Experience Summary of 2017 FAST Operations Period – 2 months (Oct-Nov) • Revisited and improved startup procedures – 12 days cryo cooldown to 2 K – 9 days cavity conditioning / RF systems startup • 300 Me. V electrons achieved – First demonstration of integrated SRF systems – Cooperation by Fermilab organizations, Technical Division & Accelerator Division • Reliable performance – Almost turn-key operation – 1 -hour startup, 2 8 -hour shifts per day • Build-up of technical expertise – Trained operators (FAST is operated by IOTA Dept. ⇨ AD Ops) • Currently 6 Operators (2 -shifts/day) – Planning and execution of experiments 16 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
Plans to integration AD Ops into FAST Operations • Improve efficiency • Hands-on training – sequencer aggregates – ACNET pages – synoptic displays • Develop an OJT for AD Ops. • Troubleshooting and changing out regulator power supplies (our most common failure) • Diagnosing utility issues like LCW or pneumatics • Moving instrumentation in and out of the beam path • Contacting the appropriate systems personnel • Knowing the study plan ahead of schedule 17 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
Experimental Program • Main operational goal is commissioning/operation of IOTA and the IOTA research program – Operation of facility is not driven by FAST experimental program, experiments are carried out parasitically • Experiment funding and support comes from sources outside of GARD (could be internal FNAL or external organizations) – Minimal technical support provided for experiment installation and support • Comprehensive experiment selection process in place • Experimental program has yielded good results in short period of time (detailed in other talks during this review) 18 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
FAST vs. Experiment Responsibilities • FAST can/will provide the following equipment & services: – – – Fabrication and installation of basic supports stands (floor to device) Installation of experimental device(s) and alignment Particle-Free/Ultra High Vacuum infrastructure and technicians LCW cooling system, hoses and connection to system Basic cabling, cable termination, rack space, and power Controls system, standard instrumentation and basic infrastructure modifications • The Experiment is expected to provide the following: – Any required specialty adjusters (motorized or mechanical) – Any experiment specific water chillers, etc. required beyond LCW system in place at FAST – Experiment specific power supplies, controllers, unique/expensive cabling – Experiment specific instrumentation – Devices designed to meet the FAST vacuum specifications 19 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
Planning and Execution of Experiments • We are developing a rigorous system – Modeled after Tevatron end-of-run studies, BNL/RHIC APEX, and CERN/LHC MD procedures – FAST is not a user facility, but operates similar to one, coordination is paramount • Process before the (2017) run: – Aug. 2016 – One-day retreat after FY 16 run – Sept. -Dec. 2016 – Collection of initial expressions of interest – Jan. -Mar. 2017 – Development of detailed requests for each experiment, mandatory presentation at weekly meetings – April-May 2017 – Committee-developed run plan with rough shift schedule • Process during the run: – Run coordinator keeps up-to-date schedule – Weekly operations and experiment progress reports 20 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
2017 FAST Experiments • Measurements and characterization of wakefields in ILC-type cavities (LANL Ma. RIE/Saclay/FNAL) • Flat Beam generation (NIU) • THz Source (KEK/FNAL) • Phase space tomography (FNAL) • Prototype IOTA SR monitor (FNAL) NIU Collaborators LANL/CEA, Saclay Collaborators Measured spectrum of Higher Order Modes in ILC-type SRF cavity 21 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
FAST Operations Funding - Overview • Mix of funding from GARD and Accelerator Test Facilities (mutually dependent) – GARD – installation of infrastructure (develop, fabricate, install and test accelerator components); and execution of research program • Research/Research Support ~ 7 FTE’s + $400 k M&S (Scientists & Engineering Physicists) • Infrastructure ~ 3 FTE’s + $1, 400 k M&S (includes $1, 100 supplemental funding in FY 18) (Engineers & Technicians) – Test Facilities Ops. - FAST/IOTA accelerator operations as well as its sub-systems (RF, laser, controls, instrumentation, etc. ); and NML/FAST facility maintenance and repairs (water, air, etc. ) • Accelerator Ops. ~ 4 FTE’s + $100 k M&S (Engineering Physicists, Operators, Engineers & Technicians) • Facility Ops. ~ 2 FTE’s + $250 k M&S (Engineers and Technicians) 22 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
FAST Operations Budget Summary FAST Budget ($M) $8, 0 $7, 0 $6, 0 Research $5, 0 Test Faclity Ops $4, 0 Supplemental Infrastructure $3, 0 $2, 0 $1, 0 $0, 0 • • • 23 FY 15 FY 16 FY 17 FY 18 FY 19 FY 20 FY 21 Includes GARD and Test Facility costs for FAST GARD supports (FY 18) ~ 10 FTEs, $1, 800 k M&S Test Facility supports (FY 18) ~ 6 FTE’s, $400 k M&S Total budget has been essentially flat at ~ $6 M (despite $1. 1 M of supplemental funding in FY 18) Flat budget requires significant funds directed towards infrastructure/commissioning, limited funds for research (will shift in future) J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
FAST Operations - Funding Challenges • The installation of IOTA has been accelerated through supplemental (GARD) funding provided by DOE in FY 18 ($1. 1 M) – This resulted in nearly all resources (labor and M&S) being re-directed to focus on the very aggressive schedule of building and commissioning the IOTA Ring in ~1 -year – This schedule acceleration results in additional systems to operate and maintain in parallel to supporting a more aggressive commissioning schedule IOTA Ring – June 2017 24 J. Leibfritz | 2018 DOE Fermilab GARD review IOTA Ring – July 2018 07/31/2018
FAST Operations - Funding Request • In order to support the planned research program in conjunction with continued installation/commissioning of the final stages of IOTA, an increase in the Test Facility Operations budget in FY 19 is required – An additional $594 k to support 2. 2 FTE’s that were previously supported by GARD funds to install the infrastructure of the IOTA ring, who will be redirected to operations of FAST/IOTA in FY 19 in support of the research program – An additional $170 k of M&S to restore the reduced FY 18 M&S budget to the FY 17 level of support – An additional $100 k for M&S to fully maintain and operate the additional systems of FAST/IOTA 25 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
FAST Infrastructure - Challenges • FAST facility has had major upgrades/renovations over past 10 years (a lot of new equipment and infrastructure) • Some old outdated equipment still remains that results in critical vulnerability as well as unique equipment that can cause single point failures – Causes of delay in 2017 Run • Failures of the aging cryogenic system before/during startup – Installed new booster pump – Excellent support by Technical Division Cryo, but system needs investments in the long term (detailed in Jay Theilacker talk) • Failure of the gun laser – New amplifier, different vendor 26 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
Future Goal - transition from current to long-term operation model • Resources – Majority resources (labor, M&S, and schedule) devoted to installation and commissioning - including scientific staff – Limited resources dedicated to research – Shift toward research focused model – most resources support research/experimental program – Establish distinct groups for Research and Operations • Beam Operations – Commissioning dominates operations period (research is parasitic to commissioning) (only operate for 2 -4 months at a time) – 6 months operation per year, 2 shifts/day (use 3 rd shift as contingency) – Increase operations staff by 2. 2 FTE’s (FY 19 funding request) • Up-time Efficiency – ~ 86% operations efficiency (not bad, but can be improved) – 90% efficiency goal 27 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
Summary • FAST e- linac commissioned with 300 Me. V beam – First major SRF accelerator at Fermilab – Opens way to more accelerator science experiments, technology development • IOTA completion moving forward at an aggressive schedule – IOTA Ring installation complete, ready for commissioning – Focused talented team, significant participation by support departments • Experimental Program evolving rapidly – Good success during limited experimental runs to-date – Several high-impact experiments being planned • Facility focus is transitioning from Installation to Research – All the pieces in place to become a unique world-class test facility 28 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
Thank you! 29 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
Backup slides 30 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
FAST Operations Budget Summary (change name FAST table) • • Includes GARD and Test Facility costs for FAST GARD supports (FY 18) ~ 10 FTEs, $1, 800 k M&S Test Facility supports (FY 18) ~ 6 FTE’s, $400 k M&S Total budget has been essentially flat at ~ $6 M (despite $1. 1 M of supplemental funding in FY 18) • Flat budget requires significant funds directed towards infrastructure/construction, limited funds for research (will shift in future) 31 J. Leibfritz | 2018 DOE Fermilab GARD review 07/31/2018
- Slides: 31