OSTP Phase II Assessment Climate Record Generation Briefing

OSTP Phase II Assessment: Climate Record Generation Briefing to Panel on Options to Ensure the Climate Record from the NPOESS and GOES-R Spacecraft Jeffrey L. Privette NOAA’s National Climatic Data Center 19 June 2007

Outline • • Background NOAA’s Climate Data Record Framework OSTP Climate Record Generation Assessment Next Steps 2

Climate Data Record Generation Background • National Academy has emphasized the need for Climate Data Records – Current and past NASA and NOAA efforts have been ad hoc, need-driven and experimental • NOAA initiated the Scientific Data Stewardship project to develop and implement a long-term, systematic, and comprehensive approach – Rooted in NRC dialog – Planning and project is joint with NASA © 2004 Other key NRC reports: • Global Environmental Change: Research Pathways for the Next Decade (1999) • Ensuring the Climate Record from the NPP and NPOESS Meteorological Satellites (2000) • Climate Change Science: An Analysis of Some Key Questions (2001) • Planning Climate and Global Change Research (2003) 3

NRC Identified Key Elements of a Successful CDR Program CDR - A time series of measurements of sufficient length, consistency, and continuity to determine climate variability and change CDR Organizational Elements • High-level leadership council • Advisory council to represent climate research community and other stakeholders • Fundamental Climate Data Record (FCDR) Teams • Thematic Climate Data Record (TCDR) Teams CDR Generation Elements Sustaining CDR Elements • High accuracy and stability of FCDRs • Pre-launch characterization of sensors and lifetime monitoring • Thorough calibration of sensors • Well-defined criteria for TCDR selection • Stakeholder involvement and feedback for TCDRs • Well-defined criteria for TCDR validation • Use of in-situ data for validation • Available resources for reprocessing CDRs as new information becomes available • Provisions for feedback from scientific community • Long-term commitment of resources for generation and archiving of CDRs and associated data Fundamental Climate Data Record (FCDR): Time series of calibrated signals for a family of sensors together with the ancillary data used to calibrate them. Thematic Climate Data Record (TCDR): Geophysical variables derived from FCDRs, often generated by blending satellite observations, in-situ data, and model output. 4

Observational Requirements Weather vs. Climate • Weather – Best instant snapshot of an individual product – Weather variations are large compared to time-dependent observation biases – Always marches forward using the present to predict the near future • Climate – Best blend of all snapshots from a family of records – Climate variations are small compared to time-dependent observation biases – Revisits the past to provide a richer context for the present and future 5

What is Needed for EDRs vs. CDRs? Data (Direct & Remotely Sensed) Time-tagged Geo-Referenced Sensor Data Records (SDRs) Converted to Bio-Geophysical Variables Environmental Data Records (EDRs) Homogenization and Calibration Fundamental Climate Data Records (FCDRs) Converted to Bio-Geophysical Variables Climate Data Records or Homogenized Time Series Thematic Climate Data Records (TCDRs) 6

What is Needed for EDRs vs. CDRs? Data (Direct & Remotely Sensed) Time-tagged Geo-Referenced Developing Sensor Data Records (SDRs) Converted to Bio-Geophysical Variables Environmental Data Records (EDRs) Mature Homogenization and Calibration Fundamental Climate Data Records (FCDRs) Converted to Bio-Geophysical Variables Climate Data Records or Homogenized Time Series Thematic Climate Data Records (TCDRs) 7

CDRs Exploit Accumulated Sensor and Algorithm Knowledge Sensor Knowledge (~ Product Quality) Peak CDR knowledge (applied to complete record) Peak EDR knowledge (applied to final products only) Mission Duration Checkout & Operational Lifetime Prelaunch Characterization Launch Long-term Climate End of Operations Time (years) 8

AVHRR Surface Reflectance EDR Shows Spurious Jumps and False Trends Result: Incorrect Climate Conclusions 9

Error Source Problem: Orbital drift caused change in observation time EDR depicts erratic trends and variability due to observatory rather than environment. Solution: Reprocess with calibration and orbital corrections CDR provides proper record of environmental variability. 10

End-to-End CDR Planning Landscape: Prioritization, Production, & Knowledge Knowledg e 11

Global Essential Climate Variables with Heritage Records * * With Certified NPOESS Program (Post Nunn-Mc. Curdy)

Example Key Uncertainties, Gaps in Knowledge, and Research Needs Identified in the IPCC 4 th Assessment Report • Changes in Human and Natural Drivers of Climate • • • Solar changes - TSIS Modification of clouds by aerosols – VIIRS, APS, ERBE Radiative forcing due to stratospheric water vapor – Cr. IS Atmospheric methane – Cr. IS Tropospheric ozone – Cr. IS Land surface properties – VIIRS 13

Production Example - Sea Surface Temperature Linkages & processes needed to create daily SST product: Climate-quality Algorithm Complexity Far Exceeds That for Weather Higher Quality, Known Uncertainties, Connection to heritage record 14

Climate Product Hierarchy Designed to Address IPCC 4 Key Questions Product Taxonomy ECVs Example Surface Winds, Cloud Properties Climate Data Records Ocean Vector Winds, Cloud Top Temperatures Climate Information Product Trends in Atlantic Hurricane Intensity Major Climate Uncertainties IPCC 4: Limitations on hurricane frequency, intensity, and data interpretation 15

Phase II Assessment and Background • Phase I Assessment (January 2007): “Impacts of NPOESS Nunn-Mc. Curdy Certification on Joint NASA-NOAA Climate Goals” • Phase II Study: “Identifying Alternatives in Response to Joint NASA-NOAA Climate Goals” • Terms of Reference (21 February 2007) – Joint NASA-NOAA Study – NASA Primary Role • Analyze and cost options for space measurement restoration – NOAA Primary Role • Analyze and cost processes to (re-)produce, archive, and steward climate quality data records 16

Phase II Climate Product Considerations • Key Issues 1. Climate Product Generation is an essential element for development/continuity/stewardship of global climate record 2. Priorities are to minimize measurement gaps and provide climate products 3. Starting point: assume raw instrument data are available in archive • • • Data may come from non-NPOESS programs, incl. foreign partners Address all Certified / Demanifested / Alternative instruments CDRs merge current and relevant heritage satellite and in-situ data 4. Lifecycle Cost Estimates: Through FY 26 (19 years) 5. Roles and Responsibilities currently under discussion 17

ECVs Can Require Data Fusion Altimetry (4) Plus derived NOAA-Unique Information Products: e. g. , Hurricane trends, trends in extreme temperatures and rainfall • SEA STATE • SEA LEVEL • LAKES MIS (and OVW CAPABILITY) (10) • GLACIERS/ICE CAPS • ALBEDO (SURFACE) • LAND SURFACE TEMP • LAND COVER • OCEAN COLOR • VEGETATION LEAF AREA • VEGETATION FAPAR • FIRES DISTURBANCE VIIRS (14) • CLOUD PROPERTIES • SEA ICE • SNOW COVER • SEA SURFACE TEMP • UPPER AIR WIND (POLES) • PRECIPITATION • SURFACE WIND SPD & DIRECTION • ATMOS. WATER VAPOR • UPPER AIR TEMP • RADIATION Cr. IS/ATMS (4) BUDGET (SFC & TOA) ERBS/CERES) (1) • OZONE • METHANE, CO 2, GHG • AEROSOL PROPERTIES APS (1) • SOLAR IRRADIANCE TSIS (1) OMPS (2) 18

Demanifested Partially Certified ECV, CDR and Product Counts by NPOESS Sensor Essential Climate Variables (#) Climate Data Records (#) Subproducts* (#) VIIRS 11 32 180 Cr. IMMS 3 14 105 MIS 5 8 76 OMPS (Nadir Sensor Only) 1 4 5 CERES (C 1 Platform Only) 1 2 12 TSIS 1 2 6 Altimeter 2 2 16 APS 1 3 30 Totals: 25 67 430 *Subproducts include data in different assemblages, resolutions, formats, etc.

Key CDR Activities § Climate Algorithm Development and Refinement (Leveraged) § Characterization, Time-varying Bias Adjustment, Seamless Records (Leveraged) § Data (Re-)Processing (New for climate) § Archiving and Distribution (New for climate) § Data Stewardship (New for climate) § CDR Science and Information Product Development (New for Climate) 20

Climate Record Generation Key Activity Relationships Instrument and Data Characterization Algorithm Development, Refinement & Validation Data (Re-)Processing Data Stewardship (Ensures interoperability of subsystems, integrity of complete system, and long-term data preservation and extendability) Archiving & Distribution CDR Science & Information Product Development Joint agency development to provide long-term, systematic and comprehensive CDR generation 21

What Is Data Stewardship? • Ensures interoperability of all CDR subsystems, integrity of complete system, and long-term data preservation and extendability • Begins pre-launch (e. g. , guidance and co-development with system segments) • Required throughout lifecycle (e. g. , CM, documentation, tools, product support) • Examples of Pre-launch and Early Mission Stewardship Requirements – – – Develop Policies and Processes to Ensure Traceability, Provenance and Reproducibility • Metadata – Developing system-level standards – Sufficient for cataloging and user access – Developing product-by-product information and formats (pixel, granule, and collection levels) • Auxilliary, Ancillary and Intermediate Product Archiving and Distribution Develop CDR Configuration Control System • Versioning, Filename structures, Data Retention/Deletion Process Document and Configuration Manage: Algorithms, Subsystems, and Products • Algorithm Theoretical Basis Documents, Algorithm Design Docs, Detailed Design Docs, Hardware Description – – – Develop CDR Uncertainty Definition and Reporting Process • e. g. , MODIS Stage 1, 2, 3 validation definitions Develop Characterization/Cal/Val data Ingest/Archive/Distribution Framework • Metadata, Documentation and Configuration Control • Intermediate Product archiving/distribution requirements Develop Guidelines for CLASS Product Assembling: Aggregation, Bundling, Compositing 22

Hypothetical Transitioning of Agency Roles & Responsibilities 23

Next Steps • Listen closely to discussions/recommendations from this Workshop • Refine CDR Science Support budget and risk estimation • Continue to work with OSTP • Continue dialogue regarding potential international and/or domestic partnerships 24

Backups

Total ECVs & CDRs MIS 26

Prioritizing – Key Uncertainties, Gaps in Knowledge, and Research Needs Identified in the IPCC 4 th Assessment Report • Societal benefits of the Global Earth Observations System of Systems (GEOSS) emphasize the tradition of applied climatology but using 21 st century systems • Key Science Intergovernmental Panel on Climate Change (IPCC) Issues address the policyrelevant scientific research topics of highest priority for Climate and Global Change studies 27

Production “Climate Central” Concept NOAA/CLASS – For reprocessing, NOAA’s Scientific Data Stewardship (SDS) requires an IDPS-like system (Climate Central) to process: RDR SDR Multi-satellite FCDRs & TCDRs ACQUIRE RDRs IMPROVED SDRs MULTISATELLITE FCDRs Users NOAA SDS CLIMATE CENTRAL PROCESS TCDRs – SDS interdependent with CLASS, e. g. , large data I/O – Reprocessing – Requires full production engineering PROVIDE ASSESSMENTS IMPROVEMENTS Products and Services Provide FCDRs and TCDRs 28

Assumptions for “Cost to Nation” Scenario • Climate Record Generation – All CCSP/GCOS satellite-based ECVs plus NOAA-unique ECVs (e. g. , hurricane metrics) – NASA budgeting used for demanifested sensors • Traditional sensor and science teams • Includes funding of field validation networks/campaigns – NOAA budgeting used for Certified sensors • Team size linked to CDRs assigned to sensor • Validation based on operational field networks (e. g. , buoys, sondes, Climate Ref. Net) – NOAA Stewardship Augmentation • Climate Central activities beyond traditional Science Team responsibilities – Agency roles and responsibilities not yet defined • Research-to-Operations transition expected 29

Science Support Budget Development Approach: 1. CDR-based budgeting spreadsheet provided Rough Order of Magnitude (ROM) Budget for May deliverable 2. Comprehensive “Bottom-Up” cost model in development Cost C E ov CV er ag e ■ Cost_to_Nation (complete) ■ Detailed_Model (Summer 2007) Cost/Schedule Reliability 30

Characterization: Inventory of Tasks Example Pre Launch & Checkout Operational Life Long-term Climate • Design Review/Analy. /Accept. • Trade Study Support • Diagnostic Telemetry Analy. • Onboard Cal System Checkout • System/Algorithm Coefficient Table Revision (on -orbit data) • Sensor Diagnostic Operation and Data Analysis • Calibration Field Campaigns • Inter-satellite Calibration Evaluation • Lunar Calibration Analysis (if applicable) • Diagnostic Telemetry Analy. • Cal Data Monitoring/Trending • Heritage Data Calibration/Rectification –Inter-satellite –Vicarious –Lunar –EDR Impact Analyses • On-site T/V Testing Support • EDU Subsystem, System. Test Data Analysis • FU Subsystem, System Test Data Analysis • System/Algorithm Coefficient Table Generation –ISO Reference Traceability • Operational Data Monitoring System Design& Build • In-situ Cal/Val Network Gap Analysis/Augmentation –Protocols for C/V Data Collection, Metadata, Archiving and Distribution • Inter-satellite Cal/Val System Design/Build –Intra-orbit Seasonal Analy. –Long Term Stability Analy. • Cal/Val Field Campaigns • Cal/Val Network Operations and Maintenance • SDR/EDR Product Validation • System/Algorithm Coefficient Table Updates and Tuning • Anomaly Detection, Analysis & Resolution • Graceful Degradation Support • Algorithm Modifications to Accommodate Degradation • s/c Maneuver Support and Analysis (if applicable) –Develop Dynamic System Coefficient Tables • Multisatellite Long Term Stability Analysis and Correction • CDR Product Validation and Error Characterization • Heritage Data Anomaly Detection, Analysis & Resolution • Heritage System Graceful Degradation Development • Algorithm Modifications to Accommodate Heritage System Anomalies/Artifacts (e. g. , orbital drift) –Acquisition Coordination 31

Next Steps: Roles and Responsibilities Space Is Complex Considerations Activity • Activity set (coarse scale) is readily developed • Contributor set must be developed, evaluated • Agency, Interagency, Industry, Academia • Timing must address multiple challenging factors • “Research-to-Operations” hand-offs • Existing/Funded Projects and Capabilities • Possible extended EOS / POES lifetimes • Proposed Projects and Capabilities • Mission roadmap decisions • NRA responses (proposals) difficult to predict Ti m e Contributors 32

Proposed Path Forward Defining Tasks, Roles & Responsibilities 1. Develop task inventory for each major Climate Support activity § Tasks, subtasks may vary with activity, sensor and/or product 2. Determine agencies/organizations with heritage capabilities 3. Determine projects that can cover needs with existing funds § e. g. , NPOESS operational program, NASA SDS, NOAA STAR/GSICS 4. Identify gaps in current capabilities and funding 5. Develop plan for allocating uncovered tasks among agencies § Time varying § Justify overlaps where needed 6. Develop plan for allocating uncovered tasks to organizations within agency § Justify overlaps where needed 7. Roll up into master budget profile 33

Team Organization • • Steering: Mary Kicza and Mike Freilich Leads: – NOAA: Jeff Privette and Jim O’Neal – NASA: Bryant Cramer and Steve Neeck – IPO’s PEO/SUAG Representative: Mike Bonadonna Ex Officio (NOAA): Dave Young, Mike Tanner, Chet Koblinsky, Tom Karl Working Group (NOAA): – NOAA Chief Financial Office: Darrell Robertson and Mahendra Shrestha – Cost Modeling • Consultants: John Bates (NOAA) and Bruce Barkstrom (NOAA) – Sensor Expert Teams • • • TSIS: Rossow (City University of New York [CUNY]), Barkstrom (NOAA) ERBS: Rossow (CUNY), Ackerman (Wisconsin) ALT: Wilson (NOAA), Miller (NOAA) OMPS: Flynn (NOAA), Ravishankara (NOAA) APS: Murphy (NOAA), Laszlo (NOAA) CMIS: Janowiak (NOAA), Chang (NOAA), Chelton (Oregon), Arkin (Maryland) Cr. IMMS: Barnett (NOAA), Goldberg (NOAA) VIIRS: Menzel (Wisconsin), Reynolds (NOAA), Privette (NOAA) Support: Goldberg (NOAA), Menzel (Wisconsin), Justice (UMD), Weng (JCSDA) 34