Task T CMS at LHC Personnel Professors D

Task T - CMS at LHC Personnel: • Professors: D. Carlsmith, S. Dasu, M. Herndon, D. Reeder, W. Smith • Senior Scientist: R. Loveless ( New) ( Retiring) • Senior Electronics Engineer: T. Gorski • Associate Scientist: P. Chumney, Assistant Scientists: M. Grothe, A. Lanaro • Postdoc: Y. Baek • Engineers: D. Bradley, A. Golyash, M. Jaworski, A. Mohapatra, S. Rader • Graduate Students: M. Anderson, K. Grogg, J. Klukas, K. Lazaridis, J. Leonard, M. Weinberg • Technician: R. Fobes • PSL Engineers: F. Feyzi, J. Hoffman, P. Robl, D. Wahl, D. Wenman • PSL Draft/Tech: B. Dana, G. Gregerson, D. Grim, J. Johnson, D. Northacker, J. Pippin, R. Smith Outline of Talks & Activities: • Overview & Trigger Activity -- this talk • Computing & Software & Physics Activity -- Sridhara Dasu • Common & Muon & Tracking Activity -- Dick Loveless • Endcap Muon Alignment Activity -- Duncan Carlsmith Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 1

Wisconsin Senior Personnel Official Responsibilities D. Reeder • CMS Muon Institutional Board Chair • Founding US CMS Collaboration Board Chair W. Smith • CMS Trigger Project Manager • CMS Steering Committee & Management Board • CMS Electronics Board, Steering Committee & Review Team • US CMS Trigger Level 2 Manager • US CMS Steering Committee, Project Management Group S. Dasu • CMS PRS (Physics Reconstruction & Software) Online Selection Group Co-Convenor • US CMS Calorimeter Trigger Level 3 Manager • US CMS Advisory Software & Computing Board Chair R. Loveless • US CMS Muon Level 2 Construction Project Manager • US CMS Common Projects Level 2 Manager • CMS Endcap Muon System Tech. Coord. , CMS Technical Board • US CMS Steering Committee, Project Management Group • CMS Tracker Integration Team A. Lanaro • US CMS Muon Installation Level 3 Manager Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 2

Calorimeter Trigger Personnel experienced team Physicists (DOE university program support): • Faculty: Wesley Smith (CMS Trigger Project Manager) Sridhara Dasu (US CMS Cal. Trigger L 3 Manager) • Associate Scientist: Pamela Chumney • 6 yrs on CMS Cal Trig, former HERMES Trig. Coor. • Assistant Scientist Monika Grothe • 3 yrs on CMS Cal Trig, 2 years ATLAS higher level trig. , 3. 5 yrs Babar, SVT Commissioner/Run-Coordinator Engineers (US CMS project support): • Tom Gorski -- Lead Engineer & Designer • 3. 5 yrs on CMS Cal Trig, worked at PSL on SDC & Zeus Cal. Triggers • Replaced J. Lackey, who retired but is available for consulting • Matt Jaworski -- Design support, Testing • Worked on Zeus Calorimeter Trigger • Phil Robl -- Copper Link Test/Development • Lead PSL Engineer, Worked on Zeus Trigger, SDC, CMS EMU Technician (project support): Robert Fobes -- Repair, assembly, parts, orders, inventory • Worked on Zeus Trigger Students (work as TAs for now): • Mike Anderson, Kira Grogg, Kristos Lazaridis, Jessica Leonard, Marc Weinberg • Trigger emulator & simulations, Trigger Rate Calculations Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 3

Endcap Muon Personnel Physicists (DOE univ. program support): • Dick Loveless (EMU level 2 manager) • Armando Lanaro (L 3 manager for installation) • Yongwook Baek (cabling, low voltage, installation) • A. Lanaro & Y. Baek based at CERN; responsible for day-to-day work on integration and installation • Duncan Carlsmith (Alignment) - CDF Task • Jim Bellinger, CDF Scientist • Jeff Klukas, Grad Student Engineers(US CMS project support): • Farshid Feyzi, Dan Wenman (UW Physical Sciences Lab) Techs(project support -- paid as needed): • Bart Dana, Mauro Giardoni, Jay Johnson, Dave Northacker, Ron Smith (UW -PSL) Tracker & Common Projects (Endc. Disks): • Dick Loveless • PSL Personnel paid by US CMS Project Wesley Smith, U. Wisconsin, August 22, 2006 L to R: F. Feyzi, D. Wenman, A. Lanaro J. Johnson, M. Giardoni, “Regis”, Y. Baek DOE Review: Task T: CMS at LHC - 4

Computing & Software Personnel Physicists (DOE university program): • Sridhara Dasu • Wisconsin Tier-2 Manager - DISUN • Technical Coordinator, Grid Laboratory of Wisconsin (GLOW) • Chair, USCMS Software Advisory Computing Board System Manager (DOE university program): • Steve Rader Software Engineers (NSF-ITR & US CMS Computing Project): • Dan Bradley • Ajit Mohapatra & Will Maier Collaboration with U. Wisconsin Computer Sciences Dept. & Condor Group • Prof. Miron Livny • Director of Condor Project • Leadership role in Grid Projects: PPDG, Gri. Phy. N, i. VDGL • PI, GLOW Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 5

CMS Trigger & DAQ Systems Level-1 Trigger Event Manager Detector Frontend Readout Systems Builder Networks Run Control Filter Systems Computing Services Level-1 Trigger Requirements: • • Input: 109 events/sec at 40 MHz at full L = 1034 Output: 100 k. Hz (50 k. Hz for initial running) Latency: 3 sec for collection, decision, propagation Output direct to computer filter farm (no physical level-2) Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 6

CMS Level-1 Trigger W. Smith, Project Manager Wisconsin fully responsible US CMS Trigger US CMS fully responsible US CMS partially responsible UNIQUE: All system present trigger objects w/location, ET, quality -topological triggers possible Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 7

Wisconsin CMS Task Support (over past decade) UW 95 -96 Support of Personnel • J. Lackey, S. Dasu, PSL Engineering UW Equipment Support • Teleconference Equipment • Video Room Remodeling • CAD Printer UW CAD support • Purchase of Visula PCB Suite, incl. high speed router • New version of software used for Zeus trigger UW Computing Support • • • > $ 700 K 5 New Servers, 2 TB Disk, 1 GB/s network, Grad. Student RA 70 CPU system for simulation with Condor + network upgrade Two new Computer Rooms with power & AC Matching for “GLOW” shared computing facility Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 8

Calorimeter Trigger Crate Data from calorimeter FE on Cu links @ 1. 2 Gbaud • Into 126* rear Receiver Cards 160 MHz point to point backplane Spares • 18 Clock&Control , 126 Electron ID & Receiver Cards -- all validated not 18 Jet/Summary Cards -- all cards operate @ 160 MHz • Use 5 Custom Gate-Array 160 MHz Ga. As Vitesse Digital ASICs • Phase, Adder, Boundary Scan, Electron Isolation, Sort Wesley Smith, U. Wisconsin, August 22, 2006 included* DOE Review: Task T: CMS at LHC - 9

Regional Cal. Trigger Complete Five full crates at CERN Receiver Card: Front - U. Wisconsin Electron Isolation & Clock: Adder mezz link cards fraction tested (needed) Bar Code PHASE ASICs BSCAN ASICs MLUs Wesley Smith, U. Wisconsin, August 22, 2006 158/164 tested (126) (27/28 Custom Backpl Tested) (18) Receiver Mezz. Card 154/154 Tested (126) SORT EISO DC-DC Jet/Summary: EISO Input Back Rest waiting here EISO ASICs (w/heat sinks) Oscillator Clock Input Clock delay adjust DC-DC Converters Front BSCAN ASICs 23/25 tested (18) Phase ASIC Back 23/25 tested (18) Sort ASICs BSCAN ASICs DOE Review: Task T: CMS at LHC - 10

5 RCT Full Crates Operating at CERN - U. Wisconsin Rear of Full RCT crate fully cabled to HCAL trigger primitive logic Used for integration tests & board checkout after shipment Wesley Smith, U. Wisconsin, August 22, 2006 Front of Full RCT crate with Jet Capture Card that continuously samples output, checks on the fly for errors and provides readout of 256 crossings DOE Review: Task T: CMS at LHC - 11

New: Jet Capture Card -- Tom Gorski Signals Out FPGAs Translators JSC Inputs CCC Signals In Delays Receives 14 Reg. Sums, 8 HF ET’s, 8 e/ , and t and bits on 6 SCSI type cables Captures output from Jet Summary Card going to Global Calorimeter Trigger • Being used in final validation of RCT crates, all cards will go through an extended full crate test with the JCC • Software developed by Pam Chumney & Tom Gorski • Provides calorimeter trigger for CMS Magnet Test & Cosmic Challenge • Provides temporary trigger to Global Trigger while Global Calorimeter Trigger is being prepared Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 12

Trigger Commissioning Operate fully functional trigger electronics at CERN • Employed in myriad tests & preparation activities Tests in Electronics Integration Center • Labs & row of racks for all electronics subsystems • Test interfaces & integration as much as possible before move to USC 55 MTCC in SX 5 • Test multiple trigger components with multiple detector components during the magnet test. • Verify trigger functions & interfaces w/detectors on surface Installation in Underground Counting Room (USC 55) • Started Mid-July ‘ 06 -- “ready for crates” • Racks & Infrastructure installed, cooling operational • RCT crates, power, infrastructure fully installed (next slide) USC 55 Underground Counting Room • Passed CERN Safety Review • Card installation next Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 13

Regional Calorimeter Trigger Crates, Fans, Power Supplies, Monitoring, Controls and other infrastructure installed, validated and passed safety check in July Ready for card installation Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 14

Electronics work in 2006 -7 Finish & Test Master Clock Crate • Develop detailed timing test system • Use 4 -crate test-bed in Madison to produce exact procedures • Validate Clock Fanout Cards Tech’n. R. Fobes (l. ) & Eng’r. T. Gorski (r. ): • Multiple crate tests • Check for data synchronization Repair & Analysis of Board Failures • Spare Parts Inventories Install, Commission & Operate Final System in USC 55 Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 15

Cal Trigger Tasks at CERN - P. Klabbers, M. Grothe & students Retest RCT Crates arriving at CERN • Comprehensive long term tests with Jet Capture Card performed in RCT lab in electronics integration center Install RCT Crates in Underground Counting Room • Connect & test 1000+ E, HCAL cables • Connect & test to Global Cal. Trigger Install & Test Master Clock system Assoc. Scientist P. Klabbers (l. ) & Asst. Scientist M. Grothe (r. ): (both resident at CERN) • Synchronize all within 4 ns to central timing system Diagnostic & Online Software • Downloading detailed test patterns • Trigger Emulator • Bit-level match to actual electronics Integration with Trigger Supervisor • Online Monitoring & Control • Configuration of Memory Lookups • Reprogramming • Function Validation Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 16

Trigger Lab Setup at CERN Staging area - Prevessin 904 • Receive boards, crates, cables from Wisconsin • Inspect and log in using barcodes and database • Assemble crates for Integration/USC 55 • Power up and run system tests • Operate in water-cooled rack for extended tests Integration tests • Racks with cooling on a raised floor nearby Storage and Repair • Spare Crates and cards also available for use in testing and replacement of suspect cards • Soldering station, scope, and tool storage • Spare component storage Responsible: P. Chumney & M. Grothe • When collisions start need added personnel so group members can actively pursue physics analysis (task B redirection). Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 17

Trigger Integration Progress in Electronics Integration Center (Pr. 904) Large scale successful integration tests in central racks using common TTC infrastructure involving trigger primitives, regional & global processing: ECAL RCT GT/ GMT TTC DTTF HCAL CSCTF • Calorimeter Trigger • Muon Trigger • Global Trigger Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 18

ECAL/HCAL/RCT Integration Tests Successful! HCAL HTR to RCT • Used JCC to verify data • 6 cables used - all in sync • Latency measurements w/PMT as source of HCAL TPG RCT ECAL TCC to RCT • Valid link established • Channel number sent as data • 2 channels on 20 m cables sent in sync to RCT HCAL HTR & ECAL TCC to RCT ECAL TCC • 28 cables to RCT from 9 TCC & 18 HCAL HTR outputs • Common clocking: LTC, TTCci • 28 channels in sync - links stable for >10 hours error free running • “Hydra” link cable checked as an option for two SLB’s to one RCT RMC ECAL - HCAL Pattern tests • RCT can “lookup” incoming data • LUTs can be programmed to duplicate incoming data • Have ORCA emulation to help predict outputs Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 19

RCT Integration Tests Successful! RCT to Local Timing Controller (for MTCC) Tests • Receive 56 HCAL links (448 towers) in Green Barracks • Send trigger out with RCT Jet Capture Card • OR of towers in each half using Quality Bit • Validated connection with LTC/GT • GT ANDs top & bottom - reduces rate RCT Jet Capture Card to GT Tests • Backup Global Calorimeter Trigger • JCC in spare VME slot of RCT Crate • Currently sends up to 4 trigger bits • Threshold ET, e/ candidate, MIP • Tests using RCT pattern successful! • ECAL TCC patterns yield JCC output • Synchronized to Bunch crossing 0 RCT- GCT Source Card • First test with new GCT • Completed successfully in July Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 20

Calorimeter Trigger in Cosmic Challenge RCT: • Crate fully operational in Green Barracks at Point 5 • Cable connections made to HCAL • Full modules above and below connected • Allows for top-bottom coincidence HCAL crate to RCT Wesley Smith, U. Wisconsin, August 22, 2006 RCT output trigger RCT input from HCAL DOE Review: Task T: CMS at LHC - 21

Physicist Trigger M&O Tasks (all from base program) Change trigger Configuration • Respond to changing beam/detector conditions & physics priorities Study new trigger configurations • Test runs, Monte Carlo studies, data studies Trigger Physics Analysis • Understand detailed impact of trigger on physics Preparation for luminosity increases • Monte Carlo studies of new conditions, validate with present data Respond to changing apparatus • Changes in material, configuration, etc. changes in simulation Operations - 24 x 7 support during running • • Write, test & maintain electronics test programs Maintain & update bad channel list & run daily checking programs Run Control maintenance Trigger data validation and calibration • Online & Offline analysis of rates & efficiencies • Monte Carlo & data trigger simulation maintenance • Continuous validation of trigger using simulation & readout data Will need to augment UW personnel to perform tasks & do physics • Propose to redirect personnel from Task B (ZEUS) Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 22

The Future - SLHC Luminosity upgrade x 10 – SLHC : L = 1035 cm-2 s-1 • • Extends LHC mass reach by ~ 20 -30% Modest changes to machine Detector upgrades needed -- especially the trigger & tracker Time scale ~ 2015 Trigger: higher luminosity & tracking data • Double operational frequency from 40 MHz to 80 MHz • Design for much higher rejection power for pileup to retain output rate of 100 k. Hz -- more sophistication • Correlation of muon & cal triggers with new tracking trigger Status of planning starting in FY 04 • Workshops: US CMS in Madison Feb ‘ 04, CMS at CERN Feb ‘ 04 & Imperial College Jul ‘ 04, CERN, Jul ‘ 05 • CMS SLHC Level-1 Trigger and Tracker project started. W. Smith & G. Hall (Imperial) chosen to lead effort. • Workshop at Perugia April ‘ 06 proposes plan • CMS management accepts plan and charges W. Smith, G. Hall, J. Nash with writing SLHC EOI for submission to CMS management in Sept. and to LHCC by end of the year. As project ramps up, needs additional personnel • Use redirection of Task B effort Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 23

UW CMS Trigger Concerns Additional scientific staff to permit physics analysis • Present staff will be saturated w/trigger maintenance & operations • Will need additional staff to permit physics analysis, use of Wisconsin computing resources. • Propose to redirect Task B (ZEUS) staff to meet shortfall Project support of Lead Designer/ Expert Engineer: T. Gorski • Vital for commissioning and operations • US CMS project support for M&O supervised by base program • Estimated technical support project needs for M&O: • Based on Zeus Cal. Trigger M&O 1992 -2001 • Technician • Operates, repairs, maintains test facility • Repairs boards & infrastructure under physicist guidance • Total required = 0. 5 FTE • Expert Engineer • ~ 5 trips/year for 2 -3 weeks to make difficult repairs • Complicated/Subtle problems, Modifications to trigger electronics • Designer - available for consultation • ~ 2 trips/year for 2 -3 weeks for review & design issues Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 24

Trigger Summary Calorimeter Trigger • • Production & Testing finished Setup of full testing facility/lab at CERN EIC done Successful Integration tests w/ECAL, HCAL, Global Cal Trigger Final Installation started Very good progress, but… • Major work from now on: • Production Shipping, Retesting, Installation of > 1500 Boards • Software Development: online DQM, controls, offline DQM, integration with Trigger Supervisor, Emulator & Simulation • More Integration Tests • Installation and Commissioning • Operations! • Physics! • SLHC R&D Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 25

Task T Endcap Muon Activities (Talk by R. Loveless) Project Management: Dick Loveless • Construction, test, integrate, install & commission 468 CSCs, electronics & infrastructure (& lower all this in ‘ 06/’ 07!) Chamber Install. , Cabling & Test • UW responsibility -- led by UW scientist A. Lanaro, assisted by UW postdoc Y. Baek Low Voltage System • Wisconsin responsibility for all chambers & crates -- R. Loveless & UW Eng. A. Golyash • Tested and ordered Wiener Maraton system Endcap Alignment • Wisconsin responsibility -- Design & procurement complete • System installation -- A. Lanaro & Y, Baek • Alignment Reconstruction & Corrections -- D. Carlsmith, assisted by J. Bellinger • leverage CDF expertise Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 26

Task T Tracking Activities (Talk by R. Loveless) Started July 2005 with responsibility for design of infrastructure for new Tracking Integration Facility (integrate tracker & operate 25% at any time) • August 2006: TIF Infrastructure is delivered & installation almost complete Additional Engineering: • Mechanics and installation of Tracker Interface Box (PP 1) -- connection point for all pipes, cables, optical fibers inside vacuum cryostat Personnel: • Loveless, PSL engineers Dan Wenman & Farshid Feyzi Tracking Reconstruction & Physics Analysis: • Implement silicon-only tracking algorithms from CDF into CMS. • Use calorimeter & muon systems to identify tracks Personnel: • Prof. Herndon, postdoc, 1 -2 students Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 27

Task T Physics Activities (Talk by S. Dasu) Preparation for CMS Physics TDR (2006) & early data analysis • New energy regime rapid discoveries • Must begin operations (late 2007) with well understood trigger, reconstruction and analysis software Wide-ranging physics interest and expertise in the group • Analysis efforts coming to forefront as detector work finishes • We studied simulated Higgs, SUSY, SM EW and QCD datasets • Prof. Dasu & Dr. Klabbers verified that trigger systems capture all the important physics efficiently while satisfying DAQ bandwidth requirements • Ensure trigger systems operate at full performance at turn-on and beyond • Prof. Dasu led PTDR. v 2 online selection work (L 1 T and HLT) • UW grad student Leonard worked on L 1 rates and efficiencies • Dr. Grothe wrote the diffractive physics sections • Full simulation, trigger design and analysis in new CMS SW • Substantial summer student work supervised by Dasu, Herndon & Smith: • Anderson (H ), Grogg (H W( e )W( jj)) • Lazaridis (H Z( ee)Z( )), Weinberg (pp SUSY Like sign leptons) • Herndon (Z ) for tracking reconstruction studies Creation of physics samples is computationally intensive Tier-2 effort Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 28

UW Computing Activities (Talk by S. Dasu) Rader [DOE Univ. ] & Bradley, Maeir, and Mohapatra [NSF] Support • A CMS Tier-2 computer center from FY 2005 • Data-Intensive Science University Network (DISUN) institute • Leadership in Grid Community • OSG - Open Science Grid (develop tools and provide cycles) • GLOW - Grid Laboratory of Wisconsin (Lead role in development and implementation with CS Condor group to benefit entire UW campus: Ice. Cube, CS, Genomics, Chem. Eng. , Med. Phys. , etc. ) • Benefits both CMS and ATLAS + Computing support for all HEP tasks • Simulation production using local UW resources • The single largest institute producing CMS events 2002 -2006 • Over 60 M events produced, simulated, reconstructed & analyzed with ORCA in 05 -06 • Over 17 M events produced with new CMSSW since commissioning 2 weeks ago Rader: UW Tier-2 Systems Lead Mohapatra : Production czar Bradley: Software expert 547 k. SI 2000 CPU 428 batch slots 90 TB raw storage 10 Gbps bandwidth 24 h/7 d operation ~200 user accouts Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 29

UW Software Activities (Talk by S. Dasu) Trigger Controls Software • Prof. Dasu: Core VME software and framework • Dr. Klabbers: Testing and MTCC operations software • Dr. Grothe: Integration with CMS Trigger Supervisor Trigger Emulator Software • UW Student Leonard: RCT emulator in CMSSW Trigger Monitoring Software • New effort needed now to develop under supervision of Klabbers Trigger Physics Studies • New effort needed now to ensure good trigger for first physics Computing Resources Management Software • SW Engineer Bradley: New scheduler, Computing-On-Demand • Contributions to Condor, OSG and ROOT projects New Effort: Postdoc redirection from Task B and student support (Leonard full year, others from summer 07) Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 30

Wisconsin Concerns DOE Core Program Manpower at Wisconsin • Major effort on trigger software required • Tasks: board testing, monitoring/controls, diagnostics, configuration downloads & documentation, modeling, physics simulation, etc. • In US CMS Plan as core program manpower • Major effort on testing, installation, integration tests & commissioning • Planned as activity of base program manpower • For Trigger, Computing, , Muon Projects (and now Tracker) • Project-supported personnel need core program supervision • Wisconsin responsibilities are growing with time! • Long term trigger M&O + Physics Analysis + SLHC • Increase in personnel needed • Particularly so physicists can participate in the physics harvest Needs continued support & eventual growth • Provide growth from Task B redirection Wesley Smith, U. Wisconsin, August 22, 2006 DOE Review: Task T: CMS at LHC - 31