GLAST Ground Software Gammaray Large Area Space Telescope

GLAST Ground Software Gamma-ray Large Area Space Telescope The GLAST Data Handling Pipeline “Pipeline II” Daniel Flath for the GLAST Collaboration Stanford Linear Accelerator Center CHEP 2007, Victoria, Daniel Flath 1

GLAST Ground Software Outline Introduction to GLAST Computing and GLAST Pipeline Components Processing Task Examples Conclusion CHEP 2007, Victoria, Daniel Flath 2

GLAST Ground Software GLAST Key Features • Huge field of view – LAT: 20% of the sky at any instant; in sky survey mode, expose all parts of sky for ~30 minutes every 3 hours. GBM: whole unocculted sky at any time. • • Huge energy range, including band 10 Ge. V - 100 Ge. V Will transform the HE gamma-ray catalog: – by > order of magnitude in # point sources – spatially extended sources – sub-arcmin localizations (source-dependent) Two GLAST instruments: LAT: 20 Me. V – >300 Ge. V GBM: 10 ke. V – 25 Me. V Feb 5, 2008. Cape Kennedy 565 km, circular orbit 5 -year mission (10 -year goal) Large Area Telescope (LAT) spacecraft partner: General Dynamics Launch: CHEP 2007, Victoria, Daniel Flath GLAST Burst Monitor (GBM) 3

GLAST Ground Software GLAST MISSION ELEMENTS GPS Large Area Telescope & GBM Telemetry 1 kbps GLAST Spacecraft TDRSS SN S & Ku DELTA 7920 H GN Schedules Mission Operations Center (MOC) GRB Coordinates Network LAT Instrument Science Operations Center GLAST Science Support Center Schedules Alerts White Sands HEASARC GSFC GBM Instrument Operations Center Data, Command Loads CHEP 2007, Victoria, Daniel Flath 4

GLAST Ground Software Overview of LAT • Precision Si-strip Tracker (TKR) 18 XY tracking planes. Single-sided silicon strip detectors (228 mm pitch) Measure the photon direction; gamma ID. • Hodoscopic Cs. I Calorimeter (CAL) Array of 1536 Cs. I(Tl) crystals in 8 layers. Measure the photon energy; image the shower. • Segmented Anticoincidence Detector (ACD) Tracker 89 plastic scintillator tiles. Reject background of charged cosmic rays; segmentation removes self-veto effects at high energy. • Electronics System Includes flexible, robust hardware trigger and software filters. ACD [surrounds 4 x 4 array of TKR towers] e+ e– Calorimeter Systems work together to identify and measure the flux of cosmic gamma rays with energy 20 Me. V - >300 Ge. V. CHEP 2007, Victoria, Daniel Flath 5

GLAST Ground Software Motivation: General Computing Requirements • Data rate: – 400 Hz total (5 Hz Signal) – 12 GB raw data, total, over 8 daily downlinks • Prompt Processing: – Reconstruct downlink within 1 hr of receipt – Provide results to collaboration, other missions for: • Analysis, Monitoring • Flaring, bursting source detection • Follow-up observation, confirmation – Result: 500 GB reconstructed data per day • Monte Carlo: – Ongoing throughout mission, ~100 TB/yr CHEP 2007, Victoria, Daniel Flath 7

GLAST Ground Software Pipeline Components SLAC nly d-o rea Oracle Data Catalog Web Interface Client SLAC Client Line Mode Client Pipeline Web Interface Pipeline Server read-only CHEP 2007, Victoria, Daniel Flath Oracle Command Client Data Client Portal SLAC Job Control Batch Farm INFN Job Control Batch Farm IN 2 P 3 Job Control Batch Farm 8

GLAST Ground Software Example: A “Simple” Task DRP_monitoring Execution get. Interval. Data Success, Failure, (all)-Completed Conditiondiffuse. Responses live. Time. Cube launch. Roi. Analysis Script Process Jython; Runs as thread inside Server Process. Launches Subtasks, sets variables, registers roi. Analysis datasets with Data Catalog get. Roi. Data Sub. Task Provides parallelism exposure. Map exposure. Sub. Map Batch Processdrp. Exp. Map Compute intensive: Dispatched to farm. drp. Exp. Map Results collected via email. combine. Exp. Maps source. Analysis Sub-Sub. Task launch. Energy. Band. Analysis energy. Band. Analysis fit. Energy. Band CHEP 2007, Victoria, Daniel Flath fit. Energy. Band 9

GLAST Ground Software The Real Deal: Downlink Prompt Processing Calibration Merge Register Merge Monitoring Verify Clean Digitization Reconstruction CHEP 2007, Victoria, Daniel Flath 10

GLAST Ground Software Usage Plots: Activity Summary CHEP 2007, Victoria, Daniel Flath 11

GLAST Ground Software Usage Plots: Task Activity Detail CHEP 2007, Victoria, Daniel Flath 12

GLAST Ground Software Processing Plots: Single Process Statistics Maintain a record of: • Total time (Wall Clock) • Wait (Pending) time • CPU time used • Node name (machine class) CPU Factors show up CHEP 2007, Victoria, Daniel Flath Expect this to be ~1. 0 if our process is not I/O bound 13

GLAST Ground Software Pipeline Front End: Stream View CHEP 2007, Victoria, Daniel Flath 14

GLAST Ground Software Technologies Used • Database – Oracle – Java (and PL/SQL) Stored Procedures • Servers and Client Libraries (Pipeline, Data Catalog) – Java • Web: – Apache/Tomcat servers – JSP for web pages • Display. Tag for tabular data • AIDA TLD for plotting • Custom TLDs expose Pipeline client methods – Java Servlets • Serve Graph. Viz State diagrams CHEP 2007, Victoria, Daniel Flath 15

GLAST Ground Software Conclusion • Pipeline is currently in extensive use: – Monte Carlo Production – Integration and Test • raw data processing • monitoring • Software not coupled specifically to GLAST or SLAC – Could be used elsewhere – Additional batch farms incorporated by implementing simple Job Control (Java)-Interface – Contact <pii-devlist@slac. stanford. edu> if you are interested • To see GLAST Pipeline II, Data Catalog (and much more!) in action: – http: //glast-ground. slac. stanford. edu • For more on Display. Tag (plotting in web pages): – Victor Serbo: “JAIDA, JAS 3, WIRED 4 and the AIDA tag library” -EP 7, Carson Hall A, Thursday August 6 17: 30 CHEP 2007, Victoria, Daniel Flath 16

GLAST Ground Software Acknowledgments Software Development Team: Daniel Flath Charlotte Hee Karen Heidenreich Tony Johnson Igor Pavlin Max Turri Brave Users: Warren Focke Tom Glanzman CHEP 2007, Victoria, Daniel Flath 17

GLAST Ground Software Supplemental: GLAST Data Catalog CHEP 2007, Victoria, Daniel Flath 18