NOAA Metadata Update Ted Habermann NOAA EDMC Documentation

NOAA Metadata Update Ted Habermann

NOAA EDMC Documentation Directive This Procedural Directive establishes Roles and Responsibilities 1) a metadata content standard (International Standards Organization [ISO] 19115 Parts 1 and 2) a recommended representation standard (ISO 19139) Background Information for documenting NOAA’s environmental data and information.

Building the Foundation NOAA Line/Staff Offices and Programs must build on Develop and implement common metadata management tools Use rubrics to establish a baseline and monitor progress Promote and highlight good examples a strong foundation of support across all of NOAA Support training specifically targeted at improving NOAA’s data documentation Initiate teams to work on “special documentation problems” that cross Line and Staff Offices Encourage and support participation in the ISO and Open Geospatial Consortium (OGC)

Line / Staff Office & Program Responsibilities 1. Identify Documentation Expertise Data Collectors/Providers Data Users Data Stewards 2. Assess current state of documentation Standards Experts 3. Create and Improve Metadata for Discovery Metadata for Use Metadata and Documentation for Understanding Documentation of Collections Documentation of Datasets Documentation of Services 4. Publish/Preserve Metadata NOAA Data Centers / Archives

Building the Foundation The GOES-R Program recently expanded the scope of their metadata model from just ISO to ISO plus net. CDF. This creates an integrated metadata framework that includes metadata in granules (files) as well as the ISO standards. This framework is important across NOAA and for many of our partners (NASA). Initiate teams to work on “special documentation problems” that cross Line and Staff Offices

Building the Foundation The ISO 19115 metadata standard adopted by NOAA is currently undergoing a significant revision that addresses many requirements from NOAA and the net. CDF-CF community that includes many NOAA data providers and partners. These changes will be adopted as part of the Draft International Standard which is about to be approved by ISO (Nov. 2011) Encourage and support participation in the ISO and Open Geospatial Consortium (OGC)

Building the Foundation ISO metadata standard trainings were held during June in Silver Spring as part of the annual EDMC Data Management Workshop and during October as part of the FGDC ISO Metadata Summit The GEO-IDE Wiki includes over 200 pages of information about the ISO Standards. Support training specifically targeted at improving NOAA’s data documentation

Building the Foundation A set of tools for translating metadata collections from FGDC to ISO have been developed and tested by NODC and NGDC. Tools for evaluating the completeness of ISO metadata that include links to the GEO-IDE wiki have been developed and tested at NGDC. Develop and implement common metadata management tools Use rubrics to establish a baseline and monitor progress

What Goes Where?

Spiral Development / Training Check Back With Data Collectors/ Providers Check Back With Users Spiral 2 -N: Scientific Questions New Requirements New Use Cases Standard Guidance / Implementation Metadata Content Independent of standard Spiral 1: Initial Content

Potential Spirals Discovery Identification Id Title Abstract Resource Date Topic Category Theme Keyword Metadata Contact Science Contact Understanding Extent Geospatial Bounding Box Temporal Start/End Vertical Min/Max Place Keywords Connection Online. Resource: Name Description Function Application Schema Acquisition Information Instrument Platform Instrument Keywords Platform Keywords Distribution Distributor Contact Online Resource Distribution Format Data Center Keywords Browse Graphic Content Information Attribute Type Attribute Names Attribute Definitions Attribute Units Text Searches Purpose Extent Description Lineage Statement Project Keywords Quality/Lineage Sources Process Steps Quality Reports / Coverages

Spirals and Rubrics Spiral None 0 -33% 34 -66% 67 -99% All Identification Connection Extent G Distribution O A Description L S Content Progress Lineage Acquisition Information Metadata Evaluation for Dataset X Date: 2010/10/24 Each record has a score that can be calculated using a consistent, standard tool

Spiral Development / Training: Rubrics

Questions?

Identify documentation expertise 1. Establish Data Stewardship Teams to facilitate documentation creation and improvement for appropriate organizational units or around programmatic needs. 2. Data Stewardship Teams should include the following expertise/skills: • Data Collectors/Providers • Data Users • Data Stewards • Standards Experts Line/Staff Office Responsibilities

Assess current state of documentation 1. Identify existing sources of documentation (Data Collectors/Providers) 2. Classify existing documentation into following categories (Data Stewards and Standards Experts): § Metadata for Discovery § Metadata for Use § Metadata and Documentation for Understanding § Documentation of Collections § Documentation of Datasets § Documentation of Services 3. Identify high-priority targets for improvement (All members of Data Stewardship Team) 4. Highlight best practices, successful teams and individuals (Line and Staff Office Management) Line/Staff Office Responsibilities

Create and Improve Metadata 1. Translate/transform existing metadata into the recommended representation (ISO 19139) 2. Create metadata for undocumented data and information 3. Use spiral approach for improving metadata Line/Staff Office Responsibilities

Publish/Preserve Metadata (Data Stewards) 1. Publish metadata record in new or existing Web-Accessible Folders or using a standard catalog service. This will make it possible to connect metadata records to various discovery portals using standard services 2. Work with NOAA Archives to ensure that documentation and metadata will be preserved for the long-term Line/Staff Office Responsibilities

Cross-NOAA Responsibilities 1. Encourage and support participation in the ISO and Open Geospatial Consortium (OGC) standards development and evolution processes 2. Develop and implement common metadata management tools including mechanisms for evaluating the completeness and quality of data documentation • Utilize rubrics to establish the baseline and monitor progress. • Engage users in providing feedback on data documentation efforts and opportunities. 3. Promote and highlight good examples of documentation and the individuals involved in their creation. 4. Support training specifically targeted at improving NOAA’s data documentation. 5. Initiate teams to work on “special documentation problems” that cross Line and Staff Offices

EDMC Responsibilities 1. Review this Procedural Directive twice a year to evaluate effectiveness and monitor progress. 2. Work with the CIO Council and the NOAA Observing Systems Council to implement and monitor progress on the Cross-NOAA responsibilities listed above. 3. Encourage and support partnerships with external organizations in the process of migration of metadata from FGDC to ISO Standards.

Measure Progress 1. An increase in the amount of NOAA environmental data and information that is well documented and discoverable via national and international discovery portals. 2. Improvement in the quality of Line and Staff Office and Program data documentation processes.