Science Data System Reference Architecture Concepts and Principles

Science Data System Reference Architecture Concepts and Principles Dan Crichton/Steve Hughes/Emily Law (JPL) 2010 ESIP Federation Summer Meeting 1

Topics Architecture Reference Architecture Domain Specific Software Architecture Science Data System Domains Architecture Principles Conclusion SPG 7/22/10 2

Architecture: what is it? Conceptual design and decisions Define structure and behavior of a system Frameworks and standards • ANSI/IEEE Std. 1471 -2000 • RM-ODP (Reference Model of Open Distributed Processing) • TOGAF (The Open Group Architecture Framework) SPG 7/22/10 3

Communicating an architecture A good architecture is one that can be communicated to the stakeholders A good architecture presents viewpoints of the system that address stakeholder concerns A good architecture uses models and descriptions that are relevant to the stakeholders • Different models may be used to present different viewpoints (e. g. , A UML model of the system may be appropriate for some but not all stakeholders) SPG 7/22/10 4

Viewpoints and views • A viewpoint is a template for constructing a view • Enterprise, Functional, Informational, Logical, Physical, etc • A view is a description of the system from the perspective of a set of related concerns. The view is what you see The viewpoint is where you look from SPG 7/22/10 (Project Managers, Engineers, Scientists, Business Analysts, …) 5

ESDS functional view Data Acquisition Spacecraft Tracking & Data Relay Satellite (TDRS) Flight Ops Data Capture, Data Initial Processing, Transport Backup Archive to DAACs Science Data Processing, Data Mgmt. , Interoperable Data Archive & Distribution and Data Access, Research W W Data Processing & Mission Control Ground Stations NASA Integrated Services Network (NISN) Mission Services EOSDIS Data Centers* ECHO* ACCESS REASo. Ns/ MEa. SUREs Credit: H. K. “Rama” Ramapriyan SPG 7/22/10 Value-Added Providers Interagency Data Centers Earth System Models ACCESS Polar Ground Stations W Education Science Teams Measurement (SIPS)* Teams International Partners Decision Support Systems *EOSDIS Elements 6

ESDS organization view NCAR, U of Col. HIRDLS, MOPITT, SORCE GSFC GLAS, MODIS, OMI, OCDPS NSIDC DAAC SEDAC LP DAAC ASF DAAC CDDIS 1 JPL MLS, TES 1 12 1 161 9 3 11 1 1 12 1 1 1 1 GES DISC 1 1 182 7 11 112 14 7 11 2 OBPG LAADS 1 San Diego ACRIM 1 La. RC CERES, SAGE III GHRC PO. DAAC PPS ASDC 1 GHRC AMSR-E, LIS ORNL DAAC KEY EOSDIS Data Centers Related Data Providers Measurementbased Systems Science Investigator -led Processing Systems (SIPSs) 41 REASo. N 17 29 MEa. SUREs ACCESS Credit: H. K. “Rama” Ramapriyan SPG 7/22/10 7

ESDS interoperable management view Science Teams Mission Ops Coordination & Config Meetings External Science team Meetings, Ground systems meetings SIPS Regular Telecons, Managers Meetings, EOSDIS communicates on several levels to manage interfaces Level 0 Process (other agencies, international) UWGs, User services, Outreach, publications Data Centers Users Overall: Program meetings, Discipline meetings; System Testing; System Req & Config, … Credit: J. Behnke SPG 7/22/10 8

Reference Architectures (RAs) Provide a proven template solution for an architecture for a particular domain Provide a vocabulary useful for implementations Generalized and structured based on a set of patterns observed in a number of successful implementations Show components, functions, and interfaces at a high level of abstractions Consist of information model at a sufficient abstract level Satisfy an abstracted set of functions from the reference requirements Are engineered for the “ilities” (reusability, extensibility, etc) Implementation neutral SPG 7/22/10 9

Domain Specific Software Architectures (DSSAs) An assemble of software components, specialized for a specific domain Generalized for effective use across that domain Context for patterns of common problem, solution elements Domain model • By experts who have the “holistic” view and can drive the need for product lines Critical to map domain models to reference requirements Credit: Tracz, Will, Domain-Specific Software Architecture, ACM SIGSOFT, 1995 SPG 7/22/10 10

RAs vs DSSAs in science In science data systems, construction of multiple architecture viewpoints of a system is critical • Process/Enterprise • Information/Data • Technology We find the “viewpoints” are similar, but models can be domain specific • Opportunity to develop a reusable reference architecture if the “patterns” can be extracted SPG 7/22/10 11

Scientific data systems Covers a wide variety of domains Solar system exploration Astrophysics Earth science Biomedicine etc Each has its own communities, standards and systems But, there is an underlying reference architecture and domain specific software architecture in each! SPG 7/22/10 12

Space science data systems Relay Satellite Simple Information Object Spacecraft / lander Spacecraft and Scientific Instruments Science Data Archive Primitive Information Object Science Information Package Primitive Information Object Science Data Processing Telemetry Information Package External Science Community Science Information Package Science Products Information Objects Science Information Package Data Analysis and Modeling Science Information Package Planning Information Object Instrument Planning Information Object Data Acquisition and Command SPG 7/22/10 Science Team Mission Operations Instrument /Sensor Operations • Common Meta Models for Describing Space Information Objects • Common Data Dictionary end-to-end 13

Earth science data systems PO. DAAC DS Mission #1 Science Processing Center 1 Archive & Distribution (DAAC 1) Science Processing Center 2 Archive & Distribution (DAAC 2) DS Mission #2 SMAP, Desdyni SPG 7/22/10 Distributed Data Analysis (Subsetting, Gridding, Transformation, Modeling) Users Other Data Sources (e. g. NOAA) Infrastructure to support Analysis of Distributed Data 14

Patterns in science data systems Generation of Engineering and Science Data Products Data Processing Data Analysis Data Archival Data Distribution Distributed Facilities Data Movement SPG 7/22/10 15

“Ilities” in science data systems Usability Scalability Reliability Reuability Sustainability Configurability SPG 7/22/10 16

Specialization within domains Domain information models • Planetary Science Ontology • Earth Science Ontology • Etc Specific services and domain implementations are derived from the reference architecture • Reference Architecture->Domain Specific Software Architecture-> Domain Implementations SPG 7/22/10 17

Architectural instantiation RA DSSA Multi-Disciplinary Science Data System (SDS) Reference Architecture Planetary SDS Implementation Instance (PDS) Process Data Technology SPG 7/22/10 Earth SDS Architecture Health SDS Architecture Earth SDS Instance (ESDS) Health SDS Instance (EDRN) Process Data Technology … … Process Data Technology 18

Sample reference architecture Reference Architecture Process Architecture Data Standards Information Model System Process Repository Structure System Management User Management Data Dictionary System Architecture Client Presentation Application Service Data Processing Data & Resource Management System Service SPG 7/22/10 19

General SDS architectural principles Define a model for describing systems and their resources Separate the technology and the information architecture Use ontology to drive both the development and long term management of the information model Provide data system location independence Require that communication between distributed systems use metadata Provide scalability in system components and data size Allow systems using different data dictionaries and metadata implementations to be integrated Leverage existing software, where possible (e. g. , open source, etc) SPG 7/22/10 20

Conclusion A reference architecture is critical for driving a strategy and support Science Data Systems • How detailed to make the reference architecture is an art! • Take time and both domain and information modeling experts • Useful ways to represent the architecture, they need to address multiple views for different stakeholders: • Process/Enterprise • Information/Data • Technology • Agree on architecture principles that guide the design and evolution • Exploit common Science Data Systems patterns • Take advantage of available frameworks and standards, and existing implementations • Architecture development might be incremental, but having a roadmap is important SPG 7/22/10 21

Thank You! Contacts Dan Crichton Daniel. J. Crichton@jpl. nasa. gov Steve Hughes John. S. Hughes@jpl. nasa. gov Emily Law Emily. S. Law@jpl. nasa. gov SPG 7/22/10 22