The Dos and Donts of Tabular Data Distribution

The Do's and Don'ts of Tabular Data Distribution Bob Simons <bob. simons@noaa. gov> IT Specialist at NOAA NMFS SWFSC ERD

Tabular Data index id owner type latitude longitude time wtemp atmp 1 46088 NDBC 3 m Discus 48. 336 -123. 159 1993 -06 -01 T 14: 20: 00 Z 16. 4 18. 0 2 46088 NDBC 3 m Discus 48. 336 -123. 159 1993 -06 -01 T 14: 50: 00 Z 16. 5 18. 2 137522 BP 114 BP 3 m Discus 36. 905 -75. 713 2003 -02 -09 T 02: 00 Z 16. 7 12. 2 137523 BP 114 BP 3 m discus 36. 905 -75. 713 2003 -02 -09 T 04: 00 Z 16. 6 12. 0 1732156 NC 312 NCSU C-MAN 24. 456 -81. 877 1968 -10 -14 T 16: 00 Z 15. 8 14. 9 1732157 NC 312 NCSU C-MAN 24. 456 -81. 877 1968 -10 -14 T 17: 00 Z 15. 8 14. 8 3282459 41005 NDBC 6 m Discus 32. 501 -79. 090 1984 -08 -22 T 14: 20: 00 Z 14. 6 26. 8 3282460 41005 NDBC 6 m Discus 32. 501 -79. 090 1984 -08 -22 T 14: 50: 00 Z 14. 7 26. 2

Tabular / In situ Data

Don't treat tabular data like gridded data. Logic: - Socrates is a man. - All men are mortal. - Therefore, Socrates is mortal.

Don't treat tabular data like gridded data. Logic? - Our tools for gridded data are great! - We can treat tabular data as grids. - Therefore, using gridded tools for tabular data will work great!

Don't treat tabular data like gridded data. It is flawed logic. Grid dimensions: physical continuum. Tabular station. ID "dimension": a list.

Don't treat tabular data like gridded data. So the rows of data that you want are scattered. index id owner type latitude longitude time wtemp atmp 1 46088 NDBC 3 m Discus 48. 336 -123. 159 1993 -06 -01 T 14: 20: 00 Z 16. 4 18. 0 2 46088 NDBC 3 m Discus 48. 336 -123. 159 1993 -06 -01 T 14: 50: 00 Z 16. 5 18. 2 137522 BP 114 BP 3 m Discus 36. 905 -75. 713 2003 -02 -09 T 02: 00 Z 16. 7 12. 2 137523 BP 114 BP 3 m discus 36. 905 -75. 713 2003 -02 -09 T 04: 00 Z 16. 6 12. 0 1732156 NC 312 NCSU C-MAN 24. 456 -81. 877 1968 -10 -14 T 16: 00 Z 15. 8 14. 9 1732157 NC 312 NCSU C-MAN 24. 456 -81. 877 1968 -10 -14 T 17: 00 Z 15. 8 14. 8 3282459 41005 NDBC 6 m Discus 32. 501 -79. 090 1984 -08 -22 T 14: 20: 00 Z 14. 6 26. 8 3282460 41005 NDBC 6 m Discus 32. 501 -79. 090 1984 -08 -22 T 14: 50: 00 Z 14. 7 26. 2

Don't just keep the data for yourself and a few colleagues.

Don't just serve data files via FTP.

Don't make custom web applications.

Do offer web applications and web services.

Do use re-usable software.

Do use Free and Open Source Software (FOSS).

Do follow standards.

Do serve tabular data as OPe. NDAP sequences. SQL: SELECT latitude, longitude, time, temperature WHERE station=40119 DAP: ? latitude, longitude, time, temperature&station=40119

SQL is great.

SQL/OPe. NDAP queries are "declarative" languages. ? latitude, longitude, time, temperature&station=40119

Do try ERDDAP. • • • Treats gridded and tabular data differently Has web applications built on web services Re-usable Free and Open Source Follows standards (OPe. NDAP)

Do try ERDDAP. • • Gets data from local and remote sources Catalog services Offers users a choice of response file formats Standardizes time formats

Thank you!