NDBC Data Management of IOOS Data Dawn Petraitis
NDBC Data Management of IOOS Data Dawn Petraitis, Kevin Kern, and Shannon Mc. Arthur National Data Buoy Center Stennis Space Center, MS May 24, 2017
Summary of Topics �Data flow from IOOS data provider to NDBC and GTS �Measurements accepted by NDBC �Station IDs �NDBC XML format �Quality Control overview �GTS format �Historical data �Future work
How does data flow from IOOS to NDBC?
Measurements accepted by NDBC �Meteorological �Winds (direction, speed, gust), air temperature, dew point, relative humidity, barometer, sea surface temperature, wave height, dominant period, mean wave direction, short wave and long wave radiation �Oceanographic �Sub-surface water temperature and salinity (30 depths) �Ocean currents (70 depths/bins) �Dissolved oxygen (near surface and near bottom), turbidity, chlorophyll, tides, p. H, EH �Waves �Directional and non-directional (bulk and spectral)
What is needed to send data to NDBC? �Location, station name, and list of measurements �Station ID assigned by NDBC �Moored buoys and coastal stations require different IDs �FTP account assigned by NDBC �Data will be automatically pushed to comms. ndbc. noaa. gov �XML format message �XML tags to be included will depend on data sent to NDBC
Assignment of Station IDs �Moored buoys require a 5 -digit WMO ID �ID is region-specific �Land-based fixed stations require a 5 -character alphanumeric ID �ID is from the NWS Location Identifier system �Each state has a specific code �All station IDs are assigned by NDBC when we are notified of a new station by the RA/operator �Unique IDs are important for distribution on the GTS
XML Format Tags - Met
XML Format Tags - Ocean
XML Format Tags - TESAC
XML Format Tags - Supplemental
XML Format Tags - Missing
XML Format Tags – Spectral Wave
XML Format Tags – Spectral Wave Missing
XML Message Example Met and Ocean
XML Message Example– Spectral Wave Non-directional
XML Message Example – Spectral Wave Directional
Quality Control of IOOS Data �All data is run through automated QC upon receipt �NDBC runs QARTOD checks �Automated QC Handbook available on NDBC website �http: //www. ndbc. noaa. gov/NDBCHandbookof. Automat ed. Data. Quality. Control 2009. pdf �Lists the algorithms used and all flags �GTS bulletins are generated after auto QC is complete
Auto QC Flags �Hard flags stop data from being distributed on GTS and web
Auto QC Flags �Soft flags are an advisory to the data analyst �Do not stop data from being distributed
Manual Quality Control �NDBC Mission Control Center performs manual QC on a daily basis �Reviews all met, wave, and ocean data �Report on all daily QC activities sent to partners �Available on NDBC website: http: //www. ndbc. noaa. gov/pstat. shtml
GTS Data �Three types of GTS messages �Met, Wave, and Temp/Salinity/Ocean Currents (TESAC) �No limit on frequency to the GTS �Some NDBC stations now sending data every 10 minutes �Bio/geo/chemical data (oxygen, chlorophyll, p. H, etc. ) not sent out via GTS �GTS distributes data for use by weather and ocean models and forecasters
Other Methods of Data Distribution �All data sent to NDBC is displayed on website in real time �Tabular ASCII text and graphical format �Net. CDF format available on our DODS server �http: //dods. ndbc. noaa. gov/
Historical Data �NDBC does not archive IOOS data with NCEI �Maintain historical data records on our website and on DODS server
IOOS Data Availability �NDBC sends report to RAs each month �Represents the percent of hours each month that an IOOS station reports data to the GTS �Internal reports for message counts and data availability
Future Work �Change from 5 -digit WMO IDs to 7 -digit WMO IDs for GTS BUFR format �Will not require new WMO IDs for existing stations �Distribute data in BUFR format on GTS �BUFR format allows for more metadata �Will not change what variables are distributed on GTS unless a new template is released �Data Quality Improvement Project �Review our auto and manual QC procedures �Work with IOOS Program Office to better quantify the volume of IOOS observations going to the GTS
- Slides: 25