SOWG11 Solar Orbiter Archive SOAR Pedro Osuna Solar

SOWG#11 Solar Orbiter Archive (SOAR) Pedro Osuna Solar Orbiter Archive Scientist ESA UNCLASSIFIED - For Official Use

What was presented in SOWG#10 (July 2017) • ESAC as the Centre that hosts all ESA Space based missions’ data archives • Summary of Science Archives at ESAC • Brief description of Heliophysics archives at ESAC: Cluster, Double-Star, Ulysses, Proba-2, SOHO, ISS-Solaces, Solar Orbiter • Implementation strategy of Solar Orbiter archive (SOAR) • Use Cases for SOAR and URD (User Requirements Document) • SOAR current and future plans • SOAR current status • Data related issues and open questions Solar Orbiter Science Operations Working Group Meeting – SOWG#11 ESAC | 17/01/2018 | Slide 2

Activities since last SOWG • New Archive Plan (version 2. 5) released after GSIR (Ground Segment Implementation Review) • Investigation on how to implement direct access to timesieries within the database (“data-as-metadata” paradigm – see later) • Metadata checkers implemented in ingestion of data into the archive – very useful information for current data production efforts (LL 01 files sent to us by PI teams) • New functionalities: • SAMP access provided for CDF and FITS files for helper applications able to handle them • Back-end move to use TAP (Table Access Protocol) to make it ready for further interoperability • TAP access provided for metadata Solar Orbiter Science Operations Working Group Meeting – SOWG#11 ESAC | 17/01/2018 | Slide 3

Data currently in the archive • Telemetry test files produced by SOC • Fake Planning files (except one real test one) produced by SOC • LL 01 files produced by Pis: Solar Orbiter Science Operations Working Group Meeting – SOWG#11 ESAC | 17/01/2018 | Slide 4

LL 01 file metadata checks • Files are received at the SOC from the PIs • Currently, SOC pre-views the files, and sends them to the ESDC (ESAC Science Data Centre) Solar Orbiter development team for further ingestion and storage in the SOAR systems. • ESDC has developed metadata checks at ingestion time; failed files are not ingested. Useful reports to polish some of the current files e. g. : solo_20170921. report. txt Solar Orbiter Science Operations Working Group Meeting – SOWG#11 ESAC | 17/01/2018 | Slide 5

Functionalities • Download of available files (any versions) • Details of metadata in files (FITs and CDF) in details window • Capability to send files via SAMp to other applications (currently working with Autoplot, JHelioviewer (but see later), TOPCAT, Aladin) • Command line (programmatic access to the archive) available for Meatadata searches in synchronous mode Solar Orbiter Science Operations Working Group Meeting – SOWG#11 ESAC | 17/01/2018 | Slide 6

Snapshots Solar Orbiter Science Operations Working Group Meeting – SOWG#11 ESAC | 17/01/2018 | Slide 7

Time Series access and TN • ESDC Team has investigated whether the approach to keep all data as metadata in DB is feasible • Their results can be found in a technical note under review currently (“Feasibility Study on storage in Database of Solar Orbiter Time Series data”, ESDC -SOAR-TN-005) • Main conclusion from the document: • […] The main conclusion that we can extract from the analysis performed is that it is feasible to store the Solar Orbiter time series data in a relational database. [. . . ] • Still work ongoing to find out how to improve performance for big future queries • Limits will have to be set to not overflow the system Solar Orbiter Science Operations Working Group Meeting – SOWG#11 ESAC | 17/01/2018 | Slide 8

Data Related Issues, open questions, etc. From last SOWG#10: - Plan to have “data as metadata” in the DB (apart from regular files, like in the current LL data in the archive) - SAMP interaction (with Jhelioviewer, Autoplot, …) - Future visualisation of time-series within the archive What parameters to display? How to allow users to build time-series/spectrograms, etc. ? Related to current LL data visualisation discussions - Data Product Description Documents for Science – start thinking about it • Now, we really need you (Pis) to produce the Data Product Description Documents for the (future) Science Data • Remember: DPDD templates available at: https: //issues. cosmos. esa. int/solarorbiterwiki/display/SOL/DPDD+Template Solar Orbiter Science Operations Working Group Meeting – SOWG#11 ESAC | 17/01/2018 | Slide 9

Small archive demo • Current version (0. 3. 1): http: //soardev. esac. esa. int/soar-beta/ • Internal version (with access to TAP): http: //soardev. esac. esa. int/soar-int/ • Jhelioviewer (development versin with SAMP): http: //swhv. oma. be/download_test/ • Autoplot: http: //autoplot. org/autoplot. jnlp (remember to run the following line in the Autoplot screen before running any SAMP Interaction: script: http: //autoplot. org/data/tools/start. Samp. Hub. jy) • Archive help: http: //soardev. esac. esa. int/soar-beta/#help • Machine interface help (only internal version): http: //soardev. esac. esa. int/soar-int/#aio Solar Orbiter Science Operations Working Group Meeting – SOWG#11 ESAC | 17/01/2018 | Slide 10

Thank you! Pedro. Osuna@esa. int Solar Orbiter Science Operations Working Group Meeting – SOWG#11 ESAC | 17/01/2018 | Slide 11