The Integrated Surface Database Partnerships and Progress Neal

The Integrated Surface Database: Partnerships and Progress Neal Lott, Stephen Del Greco, Russell Vose, Tom Ross NOAA’s National Climatic Data Center, Asheville, NC Steven Worley, Joey Comeaux National Center for Atmospheric Research, Boulder CO 1 88 th Annual American Meteorological Society Meeting New Orleans, LA January 20 -25, 2008

Integrated Surface Data (ISD) – What is it? ? • ISD—formerly called ISH (Integrated Surface Hourly)—an integration of over 100 original datasets (NCDC, USAF, Navy, NCAR) • Data format • Data model • Relational Database table structure (data and metadata) • Archived dataset, with over 20, 000 worldwide stations for 1901 – present (< 100 sites in early years, > 10, 000 stations now active, pre 1900 data being added) • Quality control and data processing system • Data access software • Product suite • Both mature and evolving 2 88 th Annual American Meteorological Society Meeting New Orleans, LA January 20 -25, 2008

Partners and Stakeholders • Air Force 14 th Weather Squadron • US Navy Fleet Numerical Meteorology and Oceanography Detachment • National Center for Atmospheric Research • Regional Climate Centers • National Weather Service • Environmental Protection Agency • Climate Database Modernization Program • Many others 3 88 th Annual American Meteorological Society Meeting New Orleans, LA January 20 -25, 2008

Climate Database Modernization Program (CDMP) • • • Transforming NOAA’s paper and microform archives into an electronic database CDMP has placed online over 52 million weather and environmental images Keying U. S. hourly and daily data for early climate locations that recorded data during the 1800’s to early 1900’s Also partnering with several countries to digitize various datasets Data are being fully integrated into ISD and made accessible online 4 88 th Annual American Meteorological Society Meeting New Orleans, LA January 20 -25, 2008

National Center for Atmospheric Research (NCAR) • • • NCAR is now a key partner in the ISD effort Numerous datasets from NCAR being made available for integration Datasets from Brazil, Australia, Greenland, and Mexico now integrated into ISD Approximately fifty additional datasets have been identified for potential integration into ISD Spatial and temporal coverage of data are enhanced 5 88 th Annual American Meteorological Society Meeting New Orleans, LA January 20 -25, 2008

Other Partners and Players • • Other datastreams such as the Climate Reference Network (CRN) and the US Surface Radiation Network (SURFRAD) are being operationally integrated Others such as the US Cooperative Network (over 8000 active stations) are planned Cooperation with our ISD partners within and outside of the Federal Climate Complex (USAF, Navy) will provide continued daily flow of data USAF 14 th Weather Squadron has been and is a key contributor of historical and near real-time global synoptic and hourly surface data 6 88 th Annual American Meteorological Society Meeting New Orleans, LA January 20 -25, 2008

Integrated Surface Data (ISD) Process Flow Various historical National/International data forms Numerous historical datasets, WMO GTS, etc Historical US data, NOAAPORT, etc Country-specific datasets, etc CDMP/ NCDC USAF/ Navy NCDC NCAR Convert to ISD standards Merge and quality-control into final ISD database Access Services— FTP, GIS, WWW Interface, etc 7 NOAA and external customers—climate monitoring, engineering 88 th Annual American Meteorological standards, etc. Society Meeting Products—data summaries, “global summary of day, ” etc New Orleans, LA January 20 -25, 2008

8 88 th Annual American Meteorological Society Meeting New Orleans, LA January 20 -25, 2008

ISD Spatial Coverage 9 88 th Annual American Meteorological Society Meeting New Orleans, LA January 20 -25, 2008

10 88 th Annual American Meteorological Society Meeting New Orleans, LA January 20 -25, 2008

Mexico -- Stations with Additional Data Integrated into ISD 11 88 th Annual American Meteorological Society Meeting New Orleans, LA January 20 -25, 2008

Brazil -- Stations with Additional Data Integrated into ISD 12 88 th Annual American Meteorological Society Meeting New Orleans, LA January 20 -25, 2008

Greenland -- Stations with Additional Data Integrated into ISD 13 88 th Annual American Meteorological Society Meeting New Orleans, LA January 20 -25, 2008

Australia -- Stations with Additional Data Integrated into ISD 14 88 th Annual American Meteorological Society Meeting New Orleans, LA January 20 -25, 2008

Data and Product Access • Climate Data Online cdo. ncdc. noaa. gov • FTP ftp. ncdc. noaa. gov/pub/data/noaa • GIS Interface gis. ncdc. noaa. gov 15 88 th Annual American Meteorological Society Meeting New Orleans, LA January 20 -25, 2008

The ISD Product Suite via the NOAA Virtual Data System (NVDS) • Climate Data Online services provide several format options • Local Climatological Data product • Global Summary of Day product • CDs and DVDs • 12 data summaries/frequency distributions by station – such as ceiling-visibility, temperature, dew point, sky cover • Graphing capabilities, wind rose • GIS services, FTP services • Updated daily with latest data 16 88 th Annual American Meteorological Society Meeting New Orleans, LA January 20 -25, 2008

The Product Suite……. 17 88 th Annual American Meteorological Society Meeting New Orleans, LA January 20 -25, 2008

18 88 th Annual American Meteorological Society Meeting New Orleans, LA January 20 -25, 2008

Data Usage—A Few Samples • Engineering design: Ice loads for towers, cables, wires, etc • Wind loads for buildings, etc • Heating/cooling requirements • Drainage/runoff extremes (pipes, culverts) • Aircraft operations: Crosswinds (runway design), instrument landing systems, etc • Space shuttle accident investigation: Nexrad data • Ship routing and oil rig placement • Global re-analyses for climate trends assessment, etc • HAZMAT operations and studies: oil spills, toxic release, etc • Weather risk management industry (now > $40 billion per year) • Insurance investigations and verification • Court cases and criminal investigations • Aircraft accident investigations • Wind energy studies: Wind farms, US and overseas • Commercial innovation and design: Typical and extreme conditions for a new market 19 88 th Annual American Meteorological Society Meeting New Orleans, LA January 20 -25, 2008

Benefits of Integration • Reduction of subjectivity and inconsistencies among datasets that span multiple observing networks and platforms • Standardized QA/QC based on reporting time resolution (e. g. , QC methodology for hourly temperature data independent of network) • Products which are more easily developed, consistent, and improved by collective experience and expertise • Modular software which is much more extensible • Data documentation which is consistent and applies to the full period of record for the data • Simplified portability of data into reanalysis efforts and climate change studies 20 88 th Annual American Meteorological Society Meeting New Orleans, LA January 20 -25, 2008