Sea Data Net PanEuropean infrastructure for ocean and

Sea. Data. Net - Pan-European infrastructure for ocean and marine data management A key component of EMODNET By Dick M. A. Schaap – MARIS (Sea. Data. Net Technical Coordinator) Brussels, 4 th May 2010, Expert Group on Marine Research Infrastructure Supported by the European Commission 1

Unified access to distributed data centres Sea. Data. Net is a 5 year project, funded by EU via FP 6, and has succeeded Sea-Search Sea. Data. Net is setting up and operating an efficient Pan-European distributed infrastructure for managing marine and ocean data by connecting: 40 National Oceanographic Data Centres (NODC’s), national oceanographic focal points, and ocean satellite data centres, in Europe these Data Centres are mostly divisions of major national marine research institutes and based in 35 countries, surrounding the European seas Data sets are made easily accessible for downloading through a unique portal, while the data sets are stored and managed at the individual Data Centres. 2

49 Partners = 40 Data Centres + ICES + IOC-IODE + JRC + specialists in statistics, products and informatics 3

Interoperability is the key to distributed data management system success. This is achieved in Sea. Data. Net via: Adopting the ISO 19115 metadata standard for all metadata directories Providing XML Validation Services to quality control the metadata maintenance Using common and controlled vocabularies, including international content governance Providing standard metadata entry tools Using harmonised Data Transport Formats for data sets delivery Using common quality control protocols and flag scale Adopting of OGC standards for mapping and viewing services Using SOAP Web Services in the Sea. Data. Net architecture Organising training and instruction workshops for transfer of expertise and hands-on training 4

Sea. Data. Net portal www. seadatanet. org 5

Sea. Data. Net Discovery Services EDMED - Data Sets EDMERP – Research Projects CSR – Cruise Summary Reports EDIOS – Monitoring systems EDMO – Marine Organisations CDI – Common Data Index Common tools for entry and maintenance, including Controlled Vocabularies, and common user interfaces 6

Sea. Data. Net unified User Interfaces EDMERP CSR EDIOS EDMO 7

Sea. Data. Net from data discovery to data download to analysis 8

Access to Data – Common Data Index (CDI) The Common Data Index (CDI) is a fine-grained index to individual data measurements (such as a CTD cast or moored instrument record). CDI XML Schema is based upon the ISO 19115 standard Common Data Index (CDI) system includes a Shopping and Transaction mechanism for direct online data access through file downloads Use of harmonised Data Transport Formats for data sets delivery Authorisation and Authentication Services, based upon a Central User Register and vocabulary of Data Access Restrictions 9

Data Policy and implementation Metadata free and open access, no registration required each data centre is obliged to provide the meta-data in standardized format to populate the catalogue services Data and products services the general case is free and open access however users must register once in order to get a personal login password Web form to provide necessary information User agrees with “Sea. Data. Net User Licence” After processing, login/password sent by email (email check) Licence is part of the Sea. Data. Net Data Policy, that is intended to be fully compatible with the Directive of the European Parliament and of the Council on public access to environmental information, the INSPIRE Directive, IOC, ICES, WMO, GCOS, GEOSS and CLIVAR data principles. Sea. Data. Net Data Policy is an overarching policy, respecting any local policy Each user gets a Sea. Data. Net Role; while each CDI record has a ‘Data access restriction’ label. The combination of role and access label determines ‘access OK’, ‘access to be discussed’ or ‘access denied’ 10

Sea. Data. Net CDI – Data retrieval and downloading Check Status In RSM Search Request Confirmed Include in Basket Results Ready at DC x Shopping list Submit + Authentication Download Data SDN format 11

Delivery Services – system set-up and process 12

Delivery Services – downloading 13

Sea. Data. Net CDI – Quick Search interface Using an intuitive approach which is easy for users 14

Sea. Data. Net CDI – Quick Search interface With Show / Hide Map option for display of search results 15

CDI – Data Coverage CDI V 1 coverage at end April 2010: > 760. 000 CDI entries from 25 countries and 33 data centres; connecting the remaining 10 countries and extending data coverage is well underway 16

Sea. Data. Net presentation and analysis tools Ocean Data View (ODV) General data analysis and visualization software >10, 000 registered users ODV software upgraded to ODV 4 for: Extending ODV‘s graphical display capabilities and interactive controls for automatic and visual quality control and data quality flagging Seamless connection to Sea. Data. Net output: SDN ODV 4 data formats, quality flag scale Integration of DIVA gridding software Important tool for the Sea. Data. Net regional Data Products DIVA software (Data-Interpolating Variational Analysis) allows to spatially interpolate observations on a regular grid in an optimal way. The analysis is performed on a finite element grid allowing for a spatial variable resolution and a good representation of the coastline and isobaths. 17

Sea. Data. Net presentation and analysis tools 18

Sea. Data. Net Regional Products services Temperature and salinity climatologies for the Arctic seas, Atlantic Ocean, Baltic sea, Mediteranean sea and the Black Sea regions Statistical output in Net. CDF files, that are visible via WMS 19

Cross sector involvement Sea. Data. Net cooperates and provides services for many other European and international groups and projects, contributing and safeguarding good data management. Examples: Euro. GOOS, maintaining the European Directory of Ocean observing systems (EDIOS) and quality controlling and archiving long-term data series. See www. edios. org POGO, collecting and providing information on ocean-going research vessels, and their operators, planned and completed cruises. See www. pogo-oceancruises. org GMES Marine Core Services for streamlining the provision of long-term archives for optimising marine forecast services Partners in the Humboldt project (GMES) contributing to a European Spatial Data Infrastructure Services for the international oil & gas industry for metocean data (SIMORC). See www. simorc. org 20

Sea. Data. Net Expansion Sea. Data. Net standards and tools are being adopted by new EU-funded projects, that will have a dedicated Portal, but also will be fully integrated in the Sea. Data. Net infrastructure: Up-Grade Black Sea SCENE project, (2009 – 2011) involving 6 NODCs and 35 other data holding institutes from the 6 Black Sea countries, to provide metadata and data access and to strengthen their national NODC networks. EU-funding 3. 4 Million Euro. CASPINFO project, (2009 – 2011) involving 12 institutes and private industry from the Caspian Sea region, to provide metadata and data access and to build their capacity for data management and user services. EU-funding 0. 8 Million Euro. Geo-Seas project, (2009 -2012) involving 24 geological and geophysical data centres from 16 European countries (Euro. Geo. Surveys), to provide metadata and data access. EU-funding 4. 9 Million Euro. Fleets project, (2009 -2012) involving 24 research institutes and data centres from 17 European countries, to optimise the management of research vessels and to streamline the flow of data from the research cruises to the data centre infrastructure. EU-funding 7. 2 Million Euro. In each project several Sea. Data. Net NODCs are participating for transfer of knowledge & expertise, to secure interoperability and to learn from new developments 21

Sea. Data. Net and EMODNET EU has adopted a new Marine Directive and Maritime Policy. In its Blue Book the Commission proposed to take steps in 2008 towards an overarching European Marine Observation and Data Network (EMODNET) In the accompanying Action Plan the Commission proposed to prepare by 2009 a Road map for EMODNET. EU Taskforce is lead by DG-MARE and comprises also representatives of DG Research, DG Enterprise (GMES), DG Environment, and EEA. The Roadmap was released in April 2009 as a Commission staff Working Document. 22

Sea. Data. Net and EMODNET The Roadmap provides an overview of the main data and information service categories to be covered, some of their sources and uses, as well as examples of benefits and added value of better integration, and it clarifies how this initiative relates to other initiatives. The Marine Strategy Framework Directive aims to achieve environmentally healthy marine waters by 2020. EMODNET will provide data for that purpose. Euro. GOOS and ESF Marine Board together prepared an EMODNET Vision document as input for the Roadmap. It states: ‘EMODNET will be a network of existing and developing European observation systems, linked by a data management structure covering all European coastal waters, shelf seas and surrounding ocean basins, accessible to everyone’. 23

Sea. Data. Net and EMODNET 24

Sea. Data. Net and EMODNET is also considered as a significant observation and monitoring data system for the part of the Water Information System for Europe (WISE) that is being developed for dealing with marine information (WISE-Marine) and supporting the data and indicator needs for the initial assessments required by member States in 2012 by the Marine Strategy Framework Directive. 25

Sea. Data. Net and EMODNET To further the aims set out, the Commission launched preparatory actions. These aim at setting up portals to grant access to certain types of data over a number of maritime basins. 3 year data pilots started in 2009, harvesting from DG Research developments: Geology – undertaken by Euro. Geo. Surveys consortium Chemistry – undertaken by Sea. Data. Net consortium Biology – undertaken by Eur. OBIS – MARBEF consortium Hydrography – undertaken by sub Sea. Data. Net consortium Marine Habitats – undertaken by JNCC consortium Additional funding (22. 5 MEuro) arranged for more data pilots and extending and operating existing pilots in the coming years. Overall objectives are to assemble fragmented and inaccessible marine data into interoperable, contiguous and publicly available data streams for complete maritime basins. 26

Sea. Data. Net and EMODNET All pilots adopt Sea. Data. Net standards and interconnect to the Sea. Data. Net CDI Directory and Access services, adopting the Sea. Data. Net philosophy and strategy to open data resources: Organisations, managing data for the focus regions, are invited to contribute their data for internal project use in the data products. The public data products reference to the source data via the CDI metadata. This provides a shop window for data providers. Data providers can regulate access to their data sets by indicating their ‘data access restrictions’ in the CDI metadata. Users can follow the references to the data owners and request access to the data using the Sea. Data. Net shopping mechanism. Sea. Data. Net has been adopted as core element in EMODNET for the data management component and this way is enabled to upgrade its status from project based to sustained operational infrastructure. 27

Sea. Data. Net and EMODNET has the promise of a substantial and structural funding from 2014 onwards for the data management component and the observation systems. It can be seen from the preparatory actions that this has accelerated the convergence between different data sectors (geology, hydrography, chemistry, …. ). There is an increasing willingness to harmonise, to become interoperable and to enable sharing of data in sectors and between sectors. One explanation might be the fact that organisations want to qualify for structural funding; another might be the INSPIRE Directive. By converging and cooperating both conditions can be fulfilled in a joint effort and more influential. 28

Sea. Data. Net and EMODNET So far EMODNET has a focus on improved management and distribution of existing data. Still to come is analysing and stimulating the interconnecting of observation systems and analyses which data are structurally missing and should be provided to support user applications in an optimal way. This will be done in synergy with GMES, Euro. GOOS and EEA (in-situ coordination). 29