AMS02 Computing and Ground Data Handling CHEP 2004

  • Slides: 38
Download presentation
AMS-02 Computing and Ground Data Handling CHEP 2004 September 29, 2004. Interlaken Alexei Klimentov

AMS-02 Computing and Ground Data Handling CHEP 2004 September 29, 2004. Interlaken Alexei Klimentov — Alexei. Klimentov@cern. ch ETH Zurich and MIT September 29. 2004. CHEP 04. Alexei Klimentov.

Outline u AMS – particle physics experiment STS 91 precursor flight AMS-02 ISS mission

Outline u AMS – particle physics experiment STS 91 precursor flight AMS-02 ISS mission u Classes of AMS data u Data Flow u Ground centers u Data Transmission SW u AMS-02 distributed Monte-Carlo Production September 29. 2004. CHEP 04. Alexei Klimentov. 2

AMS : a particle physics experiment in space AMS, the Alpha Magnetic Spectrometer, scheduled

AMS : a particle physics experiment in space AMS, the Alpha Magnetic Spectrometer, scheduled for a three years mission on the International Space Station (ISS). PHYSICS GOALS : Accurate, high statistics measurements of charged, cosmic ray spectra in space > 0. 1 GV • The study of dark matter (90% ? ) Nuclei and e-/e+ spectra measurement • Determination of the existence or absence of antimatter in the Universe Look for negative nuclei as anti-Helium, anti-Carbon • The study of the origin and composition of cosmic rays Measure isotopes D, He, Li, Be… September 29. 2004. CHEP 04. Alexei Klimentov. 3

AMS-01 The operation principles of the apparatus have been tested in space during a

AMS-01 The operation principles of the apparatus have been tested in space during a precursor flight : 100 M events recorded Trigger rates 0. 1 -1 k. Hz DAQ lifetime 90% Results : v_ Anti-matter search : He / He = 1. 1 x 10 -6 v. Charged Cosmic Ray spectra Pr, D, e-/e+, He, N v. Geomagnetic effects on CR under/over geomagnetic cutoff components Magnet : Nd 2 Fe 14 B TOF : trigger, velocity and Z Si Tracker : charge sign, rigidity, Z Aerogel Threshold Cerenkov : velocity Anticounters : reject multi particle events September 29. 2004. CHEP 04. Alexei Klimentov. 4

September 29. 2004. CHEP 04. Alexei Klimentov. 5

September 29. 2004. CHEP 04. Alexei Klimentov. 5

AMS-02. u u Super Conducting Magnet (B = 1 Tesla). Transition Radiation Detector (TRD)

AMS-02. u u Super Conducting Magnet (B = 1 Tesla). Transition Radiation Detector (TRD) , rejects protons better than 10 -2, lepton identification up to 300 Ge. V. u Time Of Flight Counters (TOF), time of flight measurement to an accuracy of 100 ps. u Silicon Tracker, 3 D particle trajectory measurement with coordinate resolution 10 um, and energy loss measurement. u Anti-Coincidence Veto Counters (ACC) reject particles that leave or enter via the shell of the magnet. u Ring Image Cherenkov Counter (RICH), measures velocity and charge of particles and nucleis. u Electromagnetic Calorimeter (ECAL). Measures energy of gamma-rays, e-, e+ , distinguishes e-/e+ from hadrons. September 29. 2004. CHEP 04. Alexei Klimentov. 6

DAQ Numbers Sub-detector Channels Total Raw KBits Tracker 196, 608 3, 146 To. F+ACC

DAQ Numbers Sub-detector Channels Total Raw KBits Tracker 196, 608 3, 146 To. F+ACC 384 49 TRD 5, 248 84 RICH 27, 760 348 ECAL 2, 592 47 Total 232. 6 K ~3700 Raw data rate : 3. 7 Mbit x 200 -2000 Hz = 0. 7 -7 GBit/s Data reduction, filtering : 2 Mbit/s AMS Power budget : 2 KW September 29. 2004. CHEP 04. Alexei Klimentov. 7

AMS September 29. 2004. CHEP 04. Alexei Klimentov. 8

AMS September 29. 2004. CHEP 04. Alexei Klimentov. 8

AMS 02 Ground Support Centers Payload Operations Control Center (POCC) at CERN (first 2

AMS 02 Ground Support Centers Payload Operations Control Center (POCC) at CERN (first 2 -3 months in Houston TX). “counting room”, usual source of commands. receives Health & Status (H&S), monitoring and science data in real-time receives NASA video. voice communication with NASA flight operations. Science Operations Center (SOC) at CERN (first 2 -3 months in Houston TX). receives the complete copy of ALL data processing and science analysis. data archiving and distribution to Universities and Laboratories. Ground Support Computers (GSC) at Marshall Space Flight Center Huntsville AL. receives data from NASA -> buffer -> retransmit to Science Center. Regional Centers. Aachen, ITEP, Karlsruhe, Lyon, Madrid, Milan, MIT, Nanjing, Shanghai, Taipei, Yale -> 19 centers. analysis facilities to support geographically close Universities. September 29. 2004. CHEP 04. Alexei Klimentov. 9

Classes of AMS Data (Health & Status data) u Critical Health and Status Data.

Classes of AMS Data (Health & Status data) u Critical Health and Status Data. – Status of detector Magnet State (charging, persistent, quenched…) Input Power (1999 W) Temperature (low, high) DAQ state (active, stuck) Rate < 1 Kbit/sec need in Real-Time (RT) to AMS Payload Operation and Control Center (POCC), to ISS crew and NASA ground September 29. 2004. CHEP 04. Alexei Klimentov. 10

Classes of AMS Data (Monitoring data) u Monitoring (House-Keeping, Slow Control) Data All slow

Classes of AMS Data (Monitoring data) u Monitoring (House-Keeping, Slow Control) Data All slow control data from all slow control sensors Data rate ~ 10 Kbit/sec needed in Near Real Time (NRT) to AMS POCC visible to ISS crew complete copy “later” (close to NRT) for science analysis September 29. 2004. CHEP 04. Alexei Klimentov. 11

Classes of AMS Data (Science data) u Science Data events, sub-detectors calibrations samples approximately

Classes of AMS Data (Science data) u Science Data events, sub-detectors calibrations samples approximately 10% to POCC to monitor detector performance in RT complete copy “later” to SOC for event reconstruction and physics analysis. 2 Mbit/sec orbit/average September 29. 2004. CHEP 04. Alexei Klimentov. 12

Classes of AMS Data (Flight Ancillary Data) u Flight Ancillary Data ISS latitude, attitude,

Classes of AMS Data (Flight Ancillary Data) u Flight Ancillary Data ISS latitude, attitude, speed, etc needed in Near Real Time (NRT) to AMS POCC complete copy “later” (close to NRT) for science analysis 2 Kbit/sec September 29. 2004. CHEP 04. Alexei Klimentov. 13

Commands u Command Types simple, fixed (few bytes – require H&S data visibility) short,

Commands u Command Types simple, fixed (few bytes – require H&S data visibility) short, variable length (<1 KByte – require monitoring data) files, variable length (K to Mbytes – requires science data) In the beginning we may need to command intensively, over the long haul anticipate : a few simple or short commands per orbit occasional (daily-weeks) periods of heavy commanding very occasional (weekly-monthly) file loading u Command Sources Ground : one source of commands – POCC Crew via ACOP – contingency use only of simple or short commands September 29. 2004. CHEP 04. Alexei Klimentov. 14

September 29. 2004. CHEP 04. Alexei Klimentov. 15

September 29. 2004. CHEP 04. Alexei Klimentov. 15

AMS Crew Operations Post (ACOP) u u u ACOP is a general purpose computer,

AMS Crew Operations Post (ACOP) u u u ACOP is a general purpose computer, the main duties of ACOP are to : Serves as internal recording device to preserve data Allows for burst mode playback operations up to 20 times original speed to assist in data management Allows access to MRDL link (another path to ground), and will enable AMS to take advantage of future ISS upgrades such as 100 baset MRDL Potential for additional data compression/ triggering functions to minimize data downlink Serves as additional command interface – Upload of files to AMS (adjust main triggering) – Direct commanding to AMS September 29. 2004. CHEP 04. Alexei Klimentov. 16

AMS Ground Data Handling How much of (and how soon) AMS data gets to

AMS Ground Data Handling How much of (and how soon) AMS data gets to the ground centers determines how well : u Detector performance can be monitored u Detector performance can be optimized u Detector performance can be tuned into physics September 29. 2004. CHEP 04. Alexei Klimentov. 17

AMS Ground Data Centers September 29. 2004. CHEP 04. Alexei Klimentov. 18

AMS Ground Data Centers September 29. 2004. CHEP 04. Alexei Klimentov. 18

Ground Support Computers u At Marshall Space Flight Center (MSFC), Huntsville Al u Receives

Ground Support Computers u At Marshall Space Flight Center (MSFC), Huntsville Al u Receives data from NASA Payload Operation and Integration Center (POIC) u Buffers data until retransmission to the AMS Science Operation Center (SOC) and if necessary to AMS Payload Operations and Control Center (POCC) u Runs unattended 24 h/day, 7 days/week u Must buffer about 600 GB (data for 2 weeks) September 29. 2004. CHEP 04. Alexei Klimentov. 19

Payload Operation and Control Center u AMS “counting room” u Usual source of AMS

Payload Operation and Control Center u AMS “counting room” u Usual source of AMS commands u Receives H&S, monitoring, science and NASA data in real-time mode u Monitor the detector state and performance u Process about 10% of data in near real time mode to provide fast information to the shift taker u Video distribution “box” u Voice loops with NASA September 29. 2004. CHEP 04. Alexei Klimentov. 20

Science Operation Center u Receives the complete copy of ALL data u Data reconstruction

Science Operation Center u Receives the complete copy of ALL data u Data reconstruction and processing, generates event summary data and does event classification u Science analysis u Archive and record ALL raw, reconstructed and H&S data u Data distribution to AMS Universities and Laboratories September 29. 2004. CHEP 04. Alexei Klimentov. 21

Regional Centers u Analysis facility to support local AMS Universities and Laboratories u Monte-Carlo

Regional Centers u Analysis facility to support local AMS Universities and Laboratories u Monte-Carlo Production u Mirroring DST (ESD) u Provide access to SOC data storage (event visualization, detector and data production status, samples of data , video distribution) September 29. 2004. CHEP 04. Alexei Klimentov. 22

Telescience Resource Kit (TRe. K) u TRe. K is a suite of software applications

Telescience Resource Kit (TRe. K) u TRe. K is a suite of software applications that provide: – Local ground support system functions. – An interface with the POIC to utilize POIC remote ground support system services. u TRe. K is suitable for individuals or payload teams that need to monitor and control low/medium data rate payloads. u The initial cost of a TRe. K system is less than $5, 000. M. Schneider MSFC/NASA September 29. 2004. CHEP 04. Alexei Klimentov. 23

ISS Payload Telemetry and Command Flow Space Shuttle International Space Station TDRS White Sands

ISS Payload Telemetry and Command Flow Space Shuttle International Space Station TDRS White Sands Complex IP’s P/L Uplinks SSCC/ MCC-H P/L User Data Telescience Support Centers (TSC’s) P/L User Data P/L Uplinks POIC (EHS, PDSS, PPS) P/L Uplinks Processed P/L Data US. Investigator Sites P/L User Data P/L Uplinks September 29. 2004. CHEP 04. Alexei Klimentov. M. Schneider MSFC/NASA 24

Telemetry Services TRe. K Telemetry Capabilities: * Receive, Process, Record, Forward, and Playback Telemetry

Telemetry Services TRe. K Telemetry Capabilities: * Receive, Process, Record, Forward, and Playback Telemetry Packets. * Display, Record, and Monitor Telemetry Parameters * View Incoming Telemetry Packets (Hex/Text Format) * Telemetry Processing Statistics M. Schneider MSFC/NASA September 29. 2004. CHEP 04. Alexei Klimentov. 25

Command Services TRe. K Command Capabilities: * Command System Status & Configuration Information *

Command Services TRe. K Command Capabilities: * Command System Status & Configuration Information * Remotely Initiated Command (Cmd Built from POIC DB) * Remotely Generated Command (Cmd Built at Remote Site) * Command Updates * Command Responses * Command Session Recording/Viewing * Command Track * Command Statistics M. Schneider MSFC/NASA September 29. 2004. CHEP 04. Alexei Klimentov. 26

Internet Voice Distribution System (IVo. DS) MSFC Payload Operations and Integration Center Remote Sites

Internet Voice Distribution System (IVo. DS) MSFC Payload Operations and Integration Center Remote Sites Conference Servers EVo. DS Voice Switch Voice Loops VOIP Telephony Gateways 1 Virtual Private Network Server LAN Administrator Server EVo. DS Keysets PAYCOM Client PC NASA, Research, and Public IP Networks IVo. DS User Client PC’s Encrypted IP Voice Packets Administrator Client PC • Windows NT/2000 PC with COTS sound card and headset • Web-based for easy installation and use • PC location very mobile – anywhere on LAN • Challenge: minor variations in PC hardware and software configurations at remote sites K. Nichols MSFC/NASA September 29. 2004. CHEP 04. Alexei Klimentov. 27

IVo. DS User Client Capabilities u u u u u Monitor 8 conferences simultaneously

IVo. DS User Client Capabilities u u u u u Monitor 8 conferences simultaneously Talk on one of these eight conferences using spacebar, ‘Click to Talk’ button, or ‘Mic Lock’ User selects from authorized subset of available voice conferences Volume control/mute for individual conferences Assign talk and monitor privileges per user and conference Show lighted talk traffic per conference Talk to crew on Space (Air) to Ground if enabled by PAYCOM Save and Load conference configuration Set password September 29. 2004. CHEP 04. Alexei Klimentov. K. Nichols MSFC/NASA 28

Data Transmission u Facing the long running period (3+ years) and the way how

Data Transmission u Facing the long running period (3+ years) and the way how data will be transmitted from the detector to the ground centers. High Rate Data Transfer between MSFC Al and AMS centers (POCC, SOC) will become a paramount importance September 29. 2004. CHEP 04. Alexei Klimentov. 29

Data Transmission SW to speed up data transfer to encrypt sensitive data and not

Data Transmission SW to speed up data transfer to encrypt sensitive data and not encrypt bulk data to run in batch mode with automatic retry in case of failure u … starting to look around and came up with bbftp (still looking for a good network monitoring tools) (bbftp developed in Ba. Bar and used to transmit data from SLAC to IN 2 P 3@Lyon) adapted it for AMS, wrote service and control programs September 29. 2004. CHEP 04. Alexei Klimentov. 30

Data Transmission SW (the inside details) u Server Ø Ø Ø u Client copy

Data Transmission SW (the inside details) u Server Ø Ø Ø u Client copy data files between directories (optional) scan data directories and make list of files to be transmitted purge successfully transmitted files and do book-keeping of transmission sessions September 29. 2004. CHEP 04. Ø Ø periodically connect to server and check if new data available bbftp new data and update transmission status in the catalogues. Alexei Klimentov. 31

Data Transmission Tests September 29. 2004. CHEP 04. Alexei Klimentov. 32

Data Transmission Tests September 29. 2004. CHEP 04. Alexei Klimentov. 32

AMS Distributed Data Production u Computer simulation of detector response is a good possibility

AMS Distributed Data Production u Computer simulation of detector response is a good possibility to study not only detector performance, but also to test HW and SW solutions that will be used for AMS-02 data processing u Data are generated in 19 Universities and Laboratories, transmitted to CERN and then available for the analysis September 29. 2004. CHEP 04. Alexei Klimentov. 33

Year 2004 MC Production u u u Started Jan 15, 2004 Central MC Database

Year 2004 MC Production u u u Started Jan 15, 2004 Central MC Database Distributed MC Production Central MC storage and archiving Distributed access September 29. 2004. CHEP 04. Alexei Klimentov. 34

AMS Distributed Data Production u u CORBA client/server for inter-process communication Central relational database

AMS Distributed Data Production u u CORBA client/server for inter-process communication Central relational database (ORACLE) to store regional centers description, list of authorized users, list of known hosts, jobs parameters, files catalogues, version of programs and executables files etc. u Automated and Standalone mode for processing jobs - automated job description file is generated by remote user request (via Web) user submits job file to a local batch system job requests from central server : calibration constants slow control corrections service info (e. g. path to store DSTs) central server keeps the table of active clients, number of processed events, handle all interactions with database and data transmission - standalone job description file is generated by remote user request (via Web) user receives a stripped database version, submits the job client doesn’t communicate with central server during job execution DSTs and log files are bbftp’ed to CERN by user September 29. 2004. CHEP 04. Alexei Klimentov. 35

MC Production Statistics 185 days, 1196 computers 8. 4 TB, 250 PIII 1 GHz/day

MC Production Statistics 185 days, 1196 computers 8. 4 TB, 250 PIII 1 GHz/day URL: pcamss 0. cern. ch/mm. html September 29. 2004. CHEP 04. Particle Million Events % of Total protons 7630 99. 9 helium 3750 99. 6 electrons 1280 99. 7 positrons 1280 100 deuterons 250 100 anti-protons 352. 5 100 carbon 291. 5 97. 2 photons 128 100 Nuclei (Z 3… 28) 856. 2 85 Alexei Klimentov. 36

Y 2004 MC Production Highlights u u u u Data are generated at remote

Y 2004 MC Production Highlights u u u u Data are generated at remote sites, transmitted to AMS@CERN and available for the analysis (only 20% of data was generated at CERN) Transmission, process communication and book-keeping programs have been debugged, the same approach will be used for AMS-02 data handling 185 days of running (~97% stability) 18 Universities & Labs 8. 4 Tbytes of data produced, stored and archived Peak rate 130 GB/day (12 Mbit/sec), average 55 GB/day (AMS-02 raw data transfer ~24 GB/day) 1196 computers Daily CPU equiv 250 1 GHz CPUs running 184 days/24 h Good simulation of AMS-02 Data Processing and Analysis September 29. 2004. CHEP 04. Alexei Klimentov. 37

List of Acronyms ACOP AMS Crew Operation Post Al Alabama AMS amsbbftp Alpha Magnetic

List of Acronyms ACOP AMS Crew Operation Post Al Alabama AMS amsbbftp Alpha Magnetic Spectrometer AMS ftp (see ftp) bbftp Ba. Bar ftp (see ftp) bps Ground Support Computers CA Bit per second CERN California CET DLT European Laboratory for Particle Physics, Geneva, CH DST Central Europe Time ESD Digital Linear Tape FTP Data Summary Tape GB Event Summary Data GSC H&S HW Hz ISS File Transfer Health and Status data Protocol Hardware Giga. Byte Hertz International Space Station September 29. 2004. LAN LOR LTO MRDL MSFC Local Area Network Lost Of Record Linear Tape Open Middle Rate Data Link Marshall Space Flight Center, Huntsville Alabama NASA NCFTP NM NRT PDSS National Aeronautics and Space Agency See FTP New Mexico Near Real Time Payload Data Service System POCC Payload Operations Control Center POIC RAID RT RTDS SMP SOC STS SW TB, Tbyte TDRSS TRe. K Payload Operation and Integration Center, MSFC CHEP 04. Redundant Array of Inexpensive Disks Real Time Data System Symmetric Multi-Processor Science Operations Center Space Shuttle Software Tera. Byte Tracking & Data Relay Satellite System Telescience Resource Kit Alexei Klimentov. 38