PDS REGISTRY AND SEARCH PDS MANAGEMENT COUNCIL FACETOFACE

PDS REGISTRY AND SEARCH PDS MANAGEMENT COUNCIL FACE-TO-FACE SANTA BARBARA, CA AUGUST 13 -14, 2019 © 2019 California Institute of Technology. Government sponsorship acknowledged.

What is the “Registry”

What is the Registry? • Registry component provides capability for tracking, auditing, locating, and maintaining artifacts within the system. • Registry Search component provides functionality to search for and retrieve registered products through defined search APIs • Artifacts - can range from data files and label files, schemas, dictionary definitions for objects and elements, services, etc. A searchable inventory of archive products. pds. nasa. gov

pds. nasa. gov

Lifecycle for archiving a product in PDS 4 pds. nasa. gov

Lifecycle for archiving a product in PDS 4 pds. nasa. gov

Enabling Data Discovery PDS Engineering PDS Archive PDS Node Archive PSA Archive JAXA Archive Stewardship Services 1. High Level Search; Content Products Data Access Portals Search Services (1, 2, 9) 2. Product Level Search 3. Retrieve Product by LIDVID 1. High Level Search 2. Product Level 3. Retrieve Product Access Services (3) Transformation Services (4) Visualization (7) On-demand Algorithms (6, 8) Data Analysis Tools (5) Computational Services (6) User Services pds. nasa. gov

CURRENT STATE OF THE REGISTRY pds. nasa. gov

Registry/Search Upgrade • Build 9 b included a major upgrade to Registry and Search components – Simplified technology stack and installation – Improved performance • Full product labels will get stored under a Registry Index in Solr – Old registry only stored product metadata – Harvest Tool was updated to support this feature – Will improve re-indexing performance • Improved performance over the old Registry • Docker for streamlined installation and deployment – https: //www. docker. com/ • Has not yet been deployed operationally at discipline nodes https: //pds-engineering. jpl. nasa. gov/development/pds 4/9. 1. 0/search/index. html pds. nasa. gov

Discipline Node Installations Node Version Installed Notes ATM Build 7 b • Registry and Search Services installed • Used as the basis for product-level search GEO Build 7 a • Registry installed • Separate from product-level search engines IMG Build 9 a • Registry and Search Services installed • Metadata from search service is extracted into product-level search NAIF N/A RMS N/A • Currently no PDS 4 data archived SBN Build 9 b • Beta testing latest Registry and Search • Intend to use for context and product-level search pds. nasa. gov

Example of User Interface using Registry: LADEE NMS Data Search 11 pds. nasa. gov

EN Registry Integration • EN maintains central registry containing only bundles, collections, and context information • EN and DN registries are currently not integrated automatically • Instead, DN notifies EN when new PDS 4 data is released • Data Users can then search for this data at https: //pds. nasa. gov/datasearch/keyword-search/ pds. nasa. gov

Search for ”insight” and facet on PDS 4: pds. nasa. gov

Registry Search APIs • Registry Search component can be adapted to work with various search protocols • Implemented – PDS Search Protocol • Protocol Docs: https: //planetarydata. org/projects/activeprojects/registration-and-search/pds-search-protocol/pds-searchprotocol/view • REST endpoint: https: //pds. nasa. gov/services/search/ – PDAP Search Protocol • Protocol Docs: https: //planetarydata. org/projects/activeprojects/registration-and-search/pds-search-protocol/pds-pdapsearch-protocol/view • REST endpoint: https: //pds. nasa. gov/services/search/pdap/ pds. nasa. gov

PDS Search Protocol Attribute Description Example identifier Search for products with a specific logical identifier (LID). identifier: urn: nasa: pds: maven. iuvs. raw: echelle instrument Search for products with a specified instrument name. instrument: instrument_name: IMAGING ULTRAVIOLET SPECTROGRAPH instrument-host-type instrument-type investigation observing-system product-class target-type title Search for products with a specified instrument host type. Search for products with a specified instrument type. Search for products with a specified investigation (mission) name. Search for products with a specified observing system name. instrument-host: MAVEN instrument-host-type: Spacecraft instrument-type: Imager investigation: Galileo observing-system: Pathfinder-IMP Search for products with a specific PDS 4 Product Class name (Product_Bundle, Product_Collection, product-class: Product_Collection etc). Search for products with a specified target name. target: MARS Search for products with a specified target type. target-type: Planet Search for products with a specified value within title: Bundle the title field of a product. start-time Search for products within a specified start-time: [2008 -05 -01 T 00: 00. 000 z to 2008 -06 datetime range. Datetime format should be in ISO 01 T 00: 00. 000 Z] 8601 format. stop-time Search for products within a specified stop-time: [2008 -05 -01 T 00: 00. 000 z 15 to 2008 -06 datetime range. Datetime format should be in ISOjpl. nasa. gov 01 T 00: 00. 000 Z] 8601 format.

NEAR-TERM PLANS (BY JANUARY 2020) pds. nasa. gov

Form a cohesive knowledge-base of all PDS 4 products archived in the PDS Tasks – Complete enhancements of Registry documentation – Deploy Registry at all Discipline Nodes – Update PDS 4 release processes to include registration of all data products Benefits / Rationale – Better understanding of PDS archive as a whole • Metrics, Analytics, etc. – Enables easier / faster re-indexing capability – Enables integrated search and discovery capabilities pds. nasa. gov

Form a cohesive knowledge-base of all PDS 4 products archived in the PDS Node Impact – Installation • Effort dependent upon operating system and systems administration – Configuration of Harvest to ingest data to Registry • Needs to be completed once per bundle • Minimum configuration for Harvest can be completed with minimal effort • Depends upon how node will use it (e. g. product search, registry-only) – Registration upon release • This effort should be minimal but could vary if any issues arise pds. nasa. gov

Prototype re-imagining of PDS central search user interface Tasks – Rapid prototyping of enhanced search interface – Complete extensive user testing to formulate and refine search user stories (requirements) – Leverage user stories to better understand user needs in developing data services task plan Benefits / Rationale – Current data/keyword search is not really useful, need a stronger foot to stand on for user testing – Gain extensive insight into our user needs and goals – Allows for development of more refined, accurate data services task plan – Begin to unravel necessary search / registry index refinements • E. g. Understanding what results should be at the top pds. nasa. gov

Prototype re-imagining of PDS central search user interface Node Impacts – Integration with Discipline Node product-level search • Assist with understanding node search APIs – Identifying potential users for user testing • Assist with users that would be will to participate in testing – May be able to leverage PDS User Alliance pds. nasa. gov

LONG-TERM PLANS (2021 AND ON) pds. nasa. gov

Define common search terms (APIs) and query language leveraging Information Model Tasks – Form working group to define: • Base set of search terms applicable across international planetary community • Common query language that should be adopted by all PDS search tools – EN implements in Registry – DNs implements in product-level search tools Benefits / Rationale – Allows for seamless integration between search tools – Enables the community to develop software that can integrate with any PDS search tool pds. nasa. gov

Define common search terms (APIs) and query language leveraging Information Model Node Impacts – Working group participation • Technical expertise and understanding of discipline node metadata – Implementing API and query language • Effort dependent upon node search implementation, but could be significant pds. nasa. gov

pds. nasa. gov

Other possible tasks dependent upon Data Services Project Plan tasks and priorities… • Develop method for dynamically tagging/augmenting of products with additional non-archival metadata (e. g. content tags, keywords, etc. ) • Develop a minimum set of best practices, criteria for node search implementations and expected results • Implement the passing of search parameters / query models across node boundaries • Improve search engine optimization with Google • Develop best practices and templates for common look-andfeel across websites / search engines pds. nasa. gov

Questions/Comments

Backup