Sea Data Net PanEuropean infrastructure for ocean and

Sea. Data. Net - Pan-European infrastructure for ocean and marine data management Present state, relations to EMODnet, and future developments By Dick M. A. Schaap – MARIS (Sea. Data. Net Technical Coordinator) Copenhagen – Denmark, May 2011, EEA – WISE-MARINE Meeting Supported by the European Commission 1

EU FP 6 2006 - 2011 Sea. Data. Net has set up and is operating a pan-European infrastructure for managing marine and ocean data by connecting 40 National Oceanographic Data Centres (NODC’s), national oceanographic focal points, and ocean satellite data centres, in Europe 2

Interoperability is the key to distributed data management system success. This is achieved in Sea. Data. Net via: Adopting the ISO 19115 metadata standard for all metadata directories Providing standard metadata entry tools: MIKADO, and online CMS Providing XML Validation Services to quality control the metadata maintenance Using common and controlled vocabularies, including international content governance Using harmonised Data Transport Formats for data sets delivery and standard tools format conversion, such as NEMO, . . Using common quality control protocols and flag scale Adopting of OGC standards for mapping and viewing services Using SOAP Web Services in the Sea. Data. Net architecture Organising training and instruction workshops for transfer of expertise and hands-on training 3

Portal with harmonised services, data products and tools for users and data centres http: //www. seadatanet. org 4

Pan-European Directory Services The Sea. Data. Net portal provides overviews of marine organisations in Europe and their engagement in research, scientific cruises, monitoring and data management for European waters and global oceans: EDMO: European Directory of Marine Organisations (>2200 entries) EDMED: European Directory of Marine Environmental Data sets (>3000 entries) EDMERP: European Directory of Marine Environmental Research Projects (>2500 entries) CSR: Cruise Summary Reports (>31500 entries) EDIOS: European Directory of Ocean-observing Systems (>270 programmes for the UK alone and many underway for other European countries) These Directories are maintained by NODCs for their country and published at pan-European level 5

Pan-European Directory Services All these Directories have been harmonised and mutually tuned in format, syntax and semantics (common vocabularies), a common XML editor (MIKADO), online CMS and online user interfaces. EDMERP CSR EDMO EDIOS EDMED 6

Relations between Directory Services From the EDMO interface one can oversee and retrieve cross related entries in each of the Directories. 7

Sea. Data. Net from data discovery to unified access to analysis 8

Common Data Index (CDI) Data Discovery and Access Service CDI is a fine-grained index to individual data measurements. The CDI Service provides users a highly detailed insight and unified access to the large volumes of marine and oceanographic data sets An intelligent middle tier connection is configured between the Sea. Data. Net portal and the local data management systems at each of the data centres. A shopping basket allows users to submit a shopping request for multiple data providers in one go and to follow its processing by each of the providers via an online transaction register. 9

Common Data Index (CDI) Data Discovery and Access Service Distributed Data Management 10

Data Policy and implementation Metadata free and open access, no registration required each data centre is obliged to provide the meta-data in standardized format to populate the catalogue services Data and products services the general case is free and open access however users must register once in order to get a personal login password Web form to provide necessary information User agrees with “Sea. Data. Net User Licence” After processing, login/password sent by email (email check) Licence is part of the Sea. Data. Net Data Policy, that is intended to be fully compatible with the Directive of the European Parliament and of the Council on public access to environmental information, the INSPIRE Directive, IOC, ICES, WMO, GCOS, GEOSS and CLIVAR data principles. Sea. Data. Net Data Policy is an overarching policy, respecting any local policy Each user gets a Sea. Data. Net Role; while each CDI record has a ‘Data access restriction’ label. The combination of role and access label determines ‘access OK’, ‘access to be discussed’ or ‘access denied’ 11

Sea. Data. Net CDI – Data retrieval and downloading Check Status In RSM Search Request Confirmed Include in Basket Results Ready at DC x Shopping list Submit + Authentication Download Data SDN format 12

Sea. Data. Net CDI Quick Search interface Drilling down and very intuitive for any user 13

Sea. Data. Net CDI Extended Search interface Searching for nutrients 14

Sea. Data. Net CDI Extended Search interface Details of a selected CDI record 15

Sea. Data. Net CDI Search interface Searching for nutrients – results map zoom in and also with Blue Marble 16

Sea. Data. Net CDI Extended Search interface Shop for nutrient data around Danmark 17

Sea. Data. Net CDI Search interface Submit Shopping list for nutrient data around Danmark 18

CDI extra functionalities Numerous options for searching and drilling down of search results CDI XML format can handle points, detailed tracks and polygons as GML CDI Summary function for search results with statistics and drill-down function CDI Mapping supports OGC WMS services with option to add external WMS layers and vice versa to serve CDI as WMS – WFS layer to other portals A query and GIS layer configuration can be stored by users as a favourite and can be forwarded to other users as a URL Extra attribute included for linking to (pre)viewing services 19

Sea. Data. Net CDI Search interface Searching for nutrients – results Summary 20

Sea. Data. Net CDI Search interface Searching for nutrients – results Summary – Data centres - clickable 21

Sea. Data. Net CDI Search interface Add OGC WMS layers – EMODNet Hydrography 22

Sea. Data. Net CDI Search interface OGC WMS layers added – EMODNet Hydrography 23

OGC WMS – WFS support CDI entries from the CDI portal are available as a series of OGC WMS and WFS services, e. g. for inclusion in the EMODNet portals. 24

CDI - INSPIRE compliance CS-W service on top of the CDI portal to deliver CDI XML entries in the INSPIRE compliant ISO 19139 format (at present aggregated by discipline and by data centre) 25

Sea. Data. Net Viewing services Ocean Data View (ODV) General data analysis and visualization software >10, 000 registered users ODV software upgraded to ODV 4 for: Extending ODV‘s graphical display capabilities and interactive controls for automatic and visual quality control and data quality flagging Seamless connection to Sea. Data. Net output: SDN ODV 4 data formats, quality flag scale Integration of DIVA gridding software Important tool for the Sea. Data. Net regional Data Products DIVA software (Data-Interpolating Variational Analysis) allows to spatially interpolate observations on a regular grid in an optimal way. The analysis is performed on a finite element grid allowing for a spatial variable resolution and a good representation of the coastline and isobaths. 26

Sea. Data. Net Viewing services – ODV + DIVA 27

Regional Products services Climatologies and interpolated data products for the Arctic seas, Atlantic Ocean, Baltic sea, Mediteranean sea and the Black Sea regions. Statistical output in Net. CDF files, that are visible via WMS through Ocean Browser 28

Cross sector involvement Sea. Data. Net cooperates and provides services for many other European and international groups and projects, contributing and safeguarding good data management. Examples: Euro. GOOS, maintaining the European Directory of Ocean observing systems (EDIOS) and quality controlling and archiving long-term data series. See www. edios. org GMES Marine Core Services for streamlining the provision of long-term archives for optimising marine forecast services; there is an Mo. U formulating the cooperation between My. Ocean and Sea. Data. Net POGO, collecting and providing information on ocean-going research vessels, and their operators, planned and completed cruises. See www. pogo-oceancruises. org Services for the international oil & gas industry for metocean data (SIMORC). See www. simorc. org 29

Sea. Data. Net Expansion Sea. Data. Net standards and tools are adopted by other EU-funded projects: Up-Grade Black Sea SCENE project, (2009 – 2011) involving 6 NODCs and 35 other data holding institutes from the 6 Black Sea countries, to provide metadata and data access and to strengthen their national NODC networks. EU-funding 3. 4 Million Euro. CASPINFO project, (2009 – 2011) involving 12 institutes and private industry from the Caspian Sea region, to provide metadata and data access and to build their capacity for data management and user services. EU-funding 0. 8 Million Euro. Geo-Seas project, (2009 -2012) involving 24 geological and geophysical data centres from 16 European countries (Euro. Geo. Surveys), to provide metadata and data access. EU-funding 4. 9 Million Euro. Fleets project, (2009 -2012) involving 24 research institutes and data centres from 17 European countries, to optimise the management of research vessels and to streamline the flow of data from the research cruises to the data centre infrastructure. EU-funding 7. 2 Million Euro. In each project several Sea. Data. Net partners are participating for transfer of knowledge & expertise, to secure interoperability and to achieve crossfertilisation leading to upgrading of the Sea. Data. Net standards and tools as well as enriching of the services. 30

Sea. Data. Net and EMODnet EU has adopted a new Marine Strategy Framework Directive (MSFD) which includes a requirement for an overarching European Marine Observation and Data Network (EMODnet) EMODnet can be considered as a network of existing and developing European observation systems, linked by a data management structure covering all European coastal waters, shelf seas and surrounding ocean basins’. 31

Sea. Data. Net and EMODnet 32

Sea. Data. Net and EMODnet Sea. Data. Net has qualified itself as leading infrastructure for the EMODnet data management component and is undertaking several preparatory EMODnet projects: Chemistry – focus on the groups of chemicals required for monitoring the MSFD for establishing Good Environmental Status – cooperation with EEA, ICES and Regional Conventions Hydrography – to produce and deliver a higher resolution digital bathymetry for European seas Physics – together with My. Ocean and Euro. GOOS Regions to give wide access to operational metocean data, real-time and archives The Geology lot is producing harmonised seabed maps (WMS). The Sea. Data. Net infrastructure is adopted via the Geo-Seas project to give overview and access to the underlying geological and geophysical data sets. Sea. Data. Net is data partner in the Biology lot and a closer technical relation is foreseen in the Sea. Data. Net II project. 33

EMODnet geographic areas Hydrography North Sea Celtic Seas, Channel, Mediterranean sea, NW Atlantic Geology North Sea Baltic and Celtic Seas Biology North Sea Bay of Biscay and the Iberian Coast Chemistry North Sea Black Sea + Med spots Physics North Sea All other European seas Habitats North Sea Baltic, Celtic Seas and the Western Mediterranean 34

Sea. Data. Net and EMODnet The EMODnet pilots and associated research projects stimulate a wider uptake of Sea. Data. Net standards and contribute to enriching the standards and services Sea. Data. Net is a project with a fixed partnership. The EMODnet pilots are data driven and bring on board many more data providers that adopt the standards to connect to the infrastructure and to populate the data resources. To illustrate this: Sea. Data. Net comprises 40 Data Centres, but already 60 Data Centres are now connected and we are making good progress towards ca 90 connected Data Centres EMODnet is management and policy driven which also provides an excellent opportunity for Sea. Data. Net to upgrade its status from project to sustained operational infrastructure. 35

CDI Service as common service www. emodnet-chemistry. eu www. blackseascene. net www. geo-seas. eu Sea. Data. Net CDI Service www. emodnet-hydrography. eu (www. emodnet-physics. eu) www. caspinfo. net 36

CDI – Data Coverage CDI coverage at end May 2011: > 1. 000 CDI entries from 29 countries and >60 data centres for physics, chemistry, geology, geophysics, bathymetry and biology; connecting another 30 data centres and increase in CDIs is well underway 37

Sea. Data. Net II proposal – Recently accepted There are many challenges and recent innovations which will be taken up in Sea. Data. Net II that will run from 1 st October 2011 – 30 th September 2015 with a funding budget of 6 Million Euro. A number of objectives: Achieving full INSPIRE compliance and contributing to the INSPIRE process for developing implementing rules for oceanography Achieving data access and data products services that meet requirements of end-users and intermediate user communities, such as GMES Marine Core Service, establishing Sea. Data. Net as the core data management component of the EMODNet infrastructure, and contributing on behalf of Europe to global portal initiatives, such as the IOC-IODE - Ocean Data Portal (ODP), and GEOSS. Achieving an improved capability for handling also marine biological data and interoperability with the emerging biodiversity data infrastructure in Europe 38

Sea. Data. Net II project – selected technical aims Establishing machine-to-machine interfacing next to present user portal for technically serving specific user communities, such as GMES Marine Core Service and WISE-Marine (EEA) Developing online visualisation (previewing, viewing) services Extension of the CDI metadata format with Observations & Measurements module and Sensor. ML module, where applicable for specific types of data, e. g. for operational oceanography Formulation of Net. CDF core data model including Sea. Data. Net metadata header in cooperation with Uni. Data Tackling the duplicates problem 39

Conclusions Sea. Data. Net is a strong partnership, well established in the Member States and in the ocean and marine community Sea. Data. Net operates and further develops a data management infrastructure that is adopted as core data management component in the EMODnet pilots and also adopted by many research projects and monitoring initiatives Sea. Data. Net has established and is further developing marine data management standards that are adopted on a wide scale Sea. Data. Net has established and is further developing interoperability solutions, such as machine-to-machine interfaces, for serving specific user communities Associated projects and EMODnet pilots stimulate expansion of the number of connected data centres and increase of data population, which is very relevant for MSFD and WISE-Marine. 40

EMODnet opportunities Additional funding (22. 5 MEuro) is being arranged for 2012 – 2014 for more data pilots and extending and operating existing pilots. Moreover EMODnet has the promise of a substantial and structural funding from 2014 onwards for the data management component and the observation systems. Already the preparatory actions are accelerating the convergence between different data sectors (geology, hydrography, chemistry, …. ). There is an increasing willingness to harmonise, to become interoperable and to enable sharing of data in sectors and between sectors. These opportunities and the Sea. Data. Net – EMODnet momentum should be used by DG Environment and EEA. 41

MSFD brings EMODnet and WISE-Marine together Policy relevant data Country data Research data Marine Core Service data Reference layers (maps) E M O D NET MSFD uropean arine bservation ata work WISE-Marine 42