NASAs Earth Science Data Systems Standards Endorsement Process

NASA’s Earth Science Data Systems Standards Endorsement Process July 03, 2006 Richard Ullman Ming-Hsiang Tsou Co-chairs ES-DSWG Standards Process

Motivation • One initiative after another has stressed the need for interoperability standards. • Many standards initiatives, both formal and grass roots have put forward specifications or demonstrated various ways to enable access to data. • NASA, or NASA funded projects are often in the forefront of these activities. • However, NASA participation in a standards development activity does not imply that NASA projects endorse the results of that activity. • Need a way to identify the “standards that work” in the context of NASA’s research and applications data systems under the operational loads that NASA typically experience. ES-DSWG Standards Process 1

Standards’ Role in Achieving NASA’s Earth Science Data Systems Needs. • Future Data Systems Features (January 2004) – Selection and management will emphasize flexibility and accountability over centralization. – – More distributed geographically, functionally and managerially. – Responsiveness to defined communities: Innovation to serve new community needs encouraged. – – Services to broad community. Ability to add new data system components, independently developed and independently managed without unduly perturbing existing systems Diversity in implementation will be encouraged - with coordination at the interfaces. ES-DSWG Standards Process 2

The Standards Process Group (SPG) • History: Starting in January 2004, NASA instituted a set of internal working groups to develop ongoing recommendations for evolution of Earth Science Data Systems development and management within NASA. One of these Data Systems Working Groups is called the Standards Process Group. • Goal: Facilitate broader use of data standards that have proven implementation and operational benefit to NASA Earth science. • This is a new strategy for standards at NASA: – – Grass-roots rather than top-down. Only after practices have been shown to (1) have demonstrated implementation and (2) benefit to operation will they be endorsed for preferential use. ES-DSWG Standards Process 3

Insights from “SEEDS” Analysis • Interoperability does not require homogeneous systems, but rather coordination at the interfaces. • Management can judge success based upon program goals rather than dictate solutions. • • – example: degree of interoperability rather than use of particular data format. Communities of practice have solutions. Published practices that demonstrate benefit can grow … – – – successful practice in specific community broader community adoption community-recognized “standards” ES-DSWG Standards Process 4

The Request For Comment Process • • • Modeled after example of Internet “IETF RFC”. • The Standards Process Group forms a Technical Working Group (TWG) to coordinate evaluation. Tailored for responsiveness to NASA. Proposed standards are documented as specifications acording to SPG guidelines and submitted by practitioners within the NASA community. – What does “implementation” of this specification mean in the context of NASA Earth Science Data Systems? – What constitutes successful “operational” experience? • The community is invited by means of email announcement to comment on the specification and particularly to address questions formulated by the TWG. • The TWG also identifies key stakeholders that are likely to have particular experience with the technology and solicits their opinion. • The TWG reports to the SPG and the SPG makes recommendations for final status of the RFC. ES-DSWG Standards Process 5

The Three Step Standards Process Initial Screening RFC Community Core üInitial review of the RFC üProvide RFC submission support üForm TWG; set schedule Proposed Standard Community Stakeholders Evaluate Implementations TWG Evaluate Implementations and Community Response Core Review of Implementation SPG Recommendation Draft Standard Community Review of Operation Stakeholders Evaluate Implementations TWG Evaluate Implementations and Community Response Core SPG Recommendation ES-DSWG Standards Process Standard Community Core 6

Responsibilities • • Community Leader – Identify someone in their community who will document standard according to SPG guidelines. – Work with the community to get an extended review of the proposed standard. SPG – – – Assign “RFC editor” to advise on RFC document. – Recommend action to NASA HQ. Publish and publicize RFC Assign “TWG”, technical working group to organize community review and evaluate responses. ES-DSWG Standards Process 7

Kinds of Practices Suitable for SPG • Any data system practice that increases interoperability or interuse of data within a community or among communities. – – • Standard - Documents Operational Use Tech Note - Builds community awareness; sometimes a precursor to a standard Examples: – Describe science content (e. g. Content standard for a level-2 precipitation product, surface reflectance product content) – – Describe interface (e. g. Data Access Protocol, Web Map Server) Describe metadata (e. g. DIF, ECHO) Describe File Format (e. g. HDF, Geo. TIFF) Best Practices (e. g. File naming conventions, data management procedures) ES-DSWG Standards Process 8

Successful RFCs will have • • • At least two implementers. Demonstrated operational benefit. Strong community leadership to support and use standard – – • Leadership in generating the RFC. Community willing/able to review Potential for “impact” and spillover to other communities ES-DSWG Standards Process 9

Benefit of publishing through the SPG • • • Benefit to NASA data systems of community endorsement: – NASA Earth science data management can rely on standards to achieve highest priority interoperability. – Science investigators are assured that standards contribute to science success in their discipline. Benefit to community/project that proposes “RFC” – – Encourage consensus within the community. Grows use of common practices among related activities. Wider discipline community learns from successful practice. Lowers barriers to entry and use of NASA data by external discipline communities within NASA and outside NASA. Accelerate “evolution” of practices through better communication. – – – From: successful practice in specific community To: broader community adoption To: community-recognized “standards” ES-DSWG Standards Process 10

Words from Op. ENDAP Group • • Excerpt from James Gallagher’s (Op. ENDAP Group): • By choosing to vet and publish standards for established technologies, the SPG standards carry extra weight. It is likely they will (continue to) be implemented and adopted. • Having a high quality document benefits development teams (reducing costs due to errors, miscommunication, et cetera). • The nature of the standards increase the likelihood of more independent implementation, which will strengthen the individual implementations and lead to high quality (demand-based) improvements. • • Standards are notorious on several levels; this process has gone very smoothly. A small group cannot develop a high quality specification without external review. The SPG provides a critical service because the process is too expensive for such a group. The OPe. NDAP Board or Directors singled this activity out as one of the most important for the past year. They felt that the benefits were well worth the (low) costs. ES-DSWG Standards Process 11

RFCs Status • RFC-004 Data Access Protocol (DAP) 2. 0: The SPG reviewed community use of DAP (a. k. a. OPe. NDAP) and found that the protocol has multiple interoperable implementations and has demonstrated operational effectiveness. The SPG has recommended that NASA endorse DAP as a community standard. • RFC-005 Web Map Service Implementation Specification (WMS) v 1. 3: The WMS is an Open Geospatial Consortium standard. This RFC makes a case for WMS use by NASA data systems. Comments received on implementation indicated that NASA data systems did not have sufficient experience with Version 1. 3 implementation to make a recommendation for WMS 1. 3 to be considered a NASA community standard. The RFC was therefore published as a “Technical note” • RFC-006 WMS V 1. 1. 1: NASA stakeholders have good experience with v 1. 1. 1 implementation of WMS. The RFC is now out for comment on use of WMS 1. 1. 1 in an operational environment. • RFC-007 HDF 5. 0: The underlying library and format of the EOS Aura standard products. This RFC is now out for comment on implementation experience. • RFC-008 HDF 5 -EOS: The particular profile of HDF 5 used by Aura standard products. This RFC is now out for comment on implementation experience. ES-DSWG Standards Process 12

“In work” Candidate RFCs Status • Geo. TIFF: Members of the geo. TIFF community have expressed a desire to use the NASA process for publication of a standard geo. TIFF specification. Author identified but progress slow because of lack of funding. • Aura DSWG data product standards: The data systems working group for the Aura mission specification for mission standard products. Publication of this as an RFC will enhance public and scientific use of these products. Author identified, but presently stalled. • ECHO: NASA’s Earth science data metadata clearinghouse has a series of APIs for data providers, client developers and service providers. Publication of these APIs as a specification is proposed as technical notes. Not a present priority of ECHO development. ES-DSWG Standards Process 13

Potental RFCs • To be discussed at the July 2006 meeting: – – – ESMF (Earth Systems Modeling Framework) SWEET (Semantic Web for Earth and Environmental Terminology) Marine Geoscience Data Management System International Institute for Climate and Society net. CDF (network Common Data Format) ES-DSWG Standards Process 14

Earth Science DSWG Standards Process Facilitate Data Systems Interoperability and Data Interuse. • Benefit to NASA data systems of community endorsement: – – • • Grows use of common practices among related activities. Wider discipline community learns from successful practice. Lowers barriers to entry and use of NASA data by external discipline communities within NASA and outside NASA. – – – From: successful practice in specific community To: broader community adoption To: community-recognized “standards” Status of Requests for Comments Modeled after example of Intern et “IETF RFC”. July 03, 2006 Tailored for responsiveness to NASA. Proposed standards are documented as specifications acording to SPG guidelines and submitted by practitioners within the NASA community. The Standards Process Group forms a Technical Working Group (TWG) to coordinate evaluation. – • Encourage consensus within the community. Accelerate adoption of practices through better communication. The Process • Science investigators assured that standards contribute to success in their discipline. Benefit to community/project that proposes “RFC” • • NASA Earth science data management can rely on standards to achieve highest priority interoperability. – What does “implementation” of this specification mean in the context of NASA Earth Science Data Systems? What constitutes successful “operational” experience? The community is invited by means of email announcement to comment on the specification and particularly to address questions formulated by the TWG. The TWG also identifies key stakeholders that are likely to have particular experience with the technology and solicits their opinion. • • • RFC-004”OPe. NDAP” recommended for endorsement - 2005 RFC-005 “WMS 1. 4” NOT recommended for endorsement RFC-006 “WMS 1. 1. 1” Open for comment on operational experience. RFC-007 “HDF 5” Open for comment on implementation experience. RFC-008 “HDF 5 -EOS” Open for comment on implementation experience. “geo. TIFF” Interest from community, waiting for RFC “Aura DSWG” Interest from community, no one available for RFC “ECHO APIs” Some interest from project, not priority. “ESMF”, “SWEET”, “net. CDF” In conversation with communities. The TWG reports to the SPG and the SPG makes recommendations for final status of the RFC. ES-DSWG Standards Process 15