Using PDS 4 The MAVEN Experience Raymond J

Using PDS 4 – The MAVEN Experience Raymond J. Walker The People Who Really Did the Work Joe Mafi, Todd King, Lyle Huber and the PPI and ATMOS Teams

Mars Atmosphere and Volatile Evolution mission MAVEN • The mission’s goal is to explore the planet’s upper atmosphere, ionosphere, and interactions with the Sun and solar wind. Scientists will use MAVEN data to explore the loss of volatile compounds—such as CO 2, N 2, and H 2 O—from the Martian atmosphere to space. – – – – – Imaging and Ultraviolet Spectrometer (IUVS) Neutral Gas and Ion Mass Spectrometer (INGIMS) Extreme Ultraviolet Monitor (EUV) Langmuir Probe and Waves (LPW) Magnetometer (MAG) Solar Energetic Particle (SEP) Suprathermal and Thermal Ion Composition (STATIC) Solar Wind Electron Analyzer (SWEA) Solar Wind Ion Analyzer (SWIA)

MAVEN • MAVEN along with LADEE were the first missions to archive data by using PDS 4. • PDS 4 was being developed in parallel with MAVEN. • Mission personnel were not all familiar with archiving in PDS. • It was decided that PPI personnel would create the metadata for the particles and fields instruments (MAG excepted). • MAVEN science data are delivered to PDS, the PPI writes the metadata and then the combined metadata and science data are peer reviewed. • MAVEN particles and fields data (MAG excepted) were provided in the Common Data Format (CDF).

The CDF Challenge • Define a version of CDF that is compatible with both the Heliophysics Data Environment (HPDE) and Planetary Data System standards. • About four years ago a small group from HPDE described a set of minimum requirements for an archive quality CDF file called CDF-A. • Earlier this year a group from both HPDE and PDS were chartered to define a version of CDF-A that could be completely described by PDS 4 metadata and thereby enable the archived data to be read by using tools that are compatible with PDS 4.

CDF-A Team Community members from a range of stakeholders. • Team included: • • Ed Grayzeck (NASA + GSFC + PDS) Aaron Roberts (NASA + HPDE + GSFC) Jim Thieman (HPDE + SPASE + GSFC) Bob Mc. Guire (GSFC + CDF) Bobby Candey (GSFC + CDF) Ray Walker (PDS/PPI + NSF) Todd King (PDS/PPI + SPASE) • Reviewed by: • Dave Heather (PSA) • Baptiste Cecconi (IPDA + LESIA)

CDF-A Specification • Defines a core set of minimal features of a CDF file to be considered of archival quality. • Includes extensions for Heliophysics Data Environment (HPDE) and Planetary Data System (PDS). • Is the guiding document PPI uses when considering archiving of data in the CDF format. Available at: http: //ppi. pds. nasa. gov/doc/CDF-A-Specification-v 1. 0. pdf

CDF-A Requirements A CDF file meets a minimum set of requirements for data archiving when: • The file contents comply with a released version of the CDF format specification. • The file includes at least a minimum set of metadata, as defined by the ISTP metadata guidelines with SPDF extensions. • All data variables which are part of the CDF file are included in the same file.

PDS/CDF-A PDS 4 metadata can describe data that are stored as contiguous arrays. For a CDF file to meet PDS 4 requirements it must adhere to all the CDF-A requirements and must also meet the following conditions: 1. The CDF file must be CDF version 3. 4 or later. 2. No compression (file or variable). 3. No unused records. (No superfluous, non-decodable records) 4. No fragmented variables (all data for a variable must be contiguous in the file). 5. No sparse variables. All data values are physical (data for all dimensions in a variable are written) 6. No virtual (calculated) variables. 7. Variables are stored in row majority order. 8. Use only "z. Variables" for data.

Working with MAVEN Data in PDS/CDF-A • In a PDS 4 label both the data and structures in a CDF-A file are described. – Data are described as objects in the core (base) PDS 4 information model. – Structures are described with a discipline specific information model (local data dictionary)

CDF-A Data in a PDS 4 Label • In a PDS 4 label the data in a CDFA file are described as Array objects • Element_Array are the observation values • Axis_Array define the dimensions of each axis and assign a name to the axis. <Array> <name>diff_en_fluxes</name> <local_identifier>diff_en_fluxes</local_identifier> <offset unit="byte">108211189</offset> <axes>4</axes> <axis_index_order>Last Index Fastest</axis_index_order> <description>Calibrated Differential Energy Flux</description> <Element_Array> <data_type>IEEE 754 MSBSingle</data_type> <unit>ev/[e. V cm^2 sr s]</unit> </Element_Array> <Axis_Array> <axis_name>epoch</axis_name> <elements>8784</elements> <sequence_number>1</sequence_number> </Axis_Array> <axis_name>phi_coarse</axis_name> <elements>16</elements> <sequence_number>2</sequence_number> </Axis_Array> <axis_name>dindex</axis_name> <elements>4</elements> <sequence_number>3</sequence_number> </Axis_Array> <axis_name>energy_coarse</axis_name> <elements>48</elements> <sequence_number>4</sequence_number> </Axis_Array> </Array>

CDF-A Structures in a PDS 4 Label • Structures have a classification. In this case “Particle_Observation” • The primary value use the local internal reference to point to the Array. • The axis values use the local internal reference to point to the Array <Particle_Observation xmlns="http: //pds. nasa. gov/pds 4/particle/v 0"> <name>diff_en_fluxes</name> <description>Calibrated Differential Energy Flux</description> <Primary_Values> <Local_Internal_Reference> <local_identifier_reference>diff_en_fluxes</local_identifier_reference> <local_reference_type>particle_observation_to_observation_values</local_reference_type> </Local_Internal_Reference> </Primary_Values> <Axis_Values> <Local_Internal_Reference> <local_identifier_reference>Epoch</local_identifier_reference> <local_reference_type>particle_observation_to_axis_values</local_reference_type> </Local_Internal_Reference> <axis_number>1</axis_number> </Axis_Values> <Local_Internal_Reference> <local_identifier_reference>phi_coarse</local_identifier_reference> <local_reference_type>particle_observation_to_axis_values</local_reference_type> </Local_Internal_Reference> <axis_number>2</axis_number> </Axis_Values> <Axis_Values> and so on…. .

Results! • The PDS 4 label can be parsed to generate a plot. • Generated with a Phython script which parses the PDS 4 label and reads the data based on offset, type and array size. • No CDF library needed. • More details at: http: //ppi. pds. nasa. gov/review/pds 4/

MAVEN Release #1 Status Node ATM PPI Instrument Release #1 Status ACC Review complete. PDS staging data. target Jul 17 IUVS Review complete. IUVS processing lien-resolved data. target Jul 20 IUVS KP Sample data delivered. Updated SIS delivered. Review to begin after 7/15 (waiting for PDS to schedule). Remainder of data to be generated pending review. target Aug 1 NGIMS Review complete. Liens resolved. Errata delivered. Certified. RELEASED Jun 3 EUV L 2 review complete; in lien resolution. Certified with liens. L 3 delivered to SDC, sample L 3 delivered to PDS. Waiting for PDS to schedule review. (Target to be ready for 2 nd release. ) RELEASED Jul 1 LPW Review begun 7/15. Comments due Aug 5. MAG Review complete; in lien resolution. Certified with liens. SEP NAIF 15 July 2015 Est. Release Data delivered. Awaiting updated SIS. Awaiting metadata generation. Review TBD. target Aug 7 RELEASED Jun 3 TBD STATIC Review complete; in lien resolution. Certified with liens. RELEASED Jul 8 SWEA Review complete; in lien resolution. Certified with liens. RELEASED Jun 5 SWIA Review complete. Liens resolved. Certified. RELEASED Jun 3 Ancillary Review complete; in lien resolution. Certified with liens. RELEASED Jun 3 Insitu KP Review complete; in lien resolution. Certified with liens. RELEASED Jul 13 Review unnecessary. Certified. RELEASED Jun 3 SPICE kernels

MAVEN Release 1 • First MAVEN data released: June 3 • Release notification: • First confirmed data use: 5: 10 PM – 3 Jun 2015 5: 52 PM – 3 Jun 2015

What we Leaned from MAVEN Source: Mitchell, Huber & Mafi presentations, PDSMC F 2 F, Berkeley, CA, 18 -19 Nov. 2014. 1. Initiate contact between PDS and the Mission early. a. PDS 4 is new to everyone. b. Face-to-face discussion is particularly useful. 2. Provide better early estimates of archiving costs. 3. Design archive schedule with mission activities in mind. 4. Identify archive conventions which the mission should establish early: a. Archive data products (raw, calibrated, high level) b. Archive organization (defining bundles/collections) c. File and archive product versioning scheme d. Directory and file naming conventions e. Time format(s) f. Data format(s)