2019 Improving data availability INSPIRE Geoportal INSPIRE MIGT

2019. Improving data availability INSPIRE Geoportal INSPIRE MIG-T 56 th Meeting Ispra, 3 -4 April 2019 Joint Research Centre The European Commission’s science and knowledge service

2019. 2 Improving data availability Session Agenda: • • Geoportal Workshop 23 -24. 1. 2019 INSPIRE Geoportal • • • Status of thematic/priority data sets availability New harvesting console - demo INSPIRE Geoportal V 1. 1 – new features New Resource linkage checker - demo Simplification of data/service linking INSPIRE Geoportal V 1. 2 – V 2. 0 new features

INSPIRE Geoportal Workshop 23 -24. 1. 2019 Together improving accessibility of MS data sets Main objectives of the workshop were: • Improving MS data sets accessibility via INSPIRE Geoportal by: • Presenting & clarifying Geoportal back-end system • Sharing our assessment of the MS feedback (31 x) related to the description of the MS infrastructures in use • Presenting a new system for harvesting MS resources • Improving MS Data sets usability • Future steps (operational aspects) and the way of collaborating with MS technical experts (Helpdesk, Forum, Virtual meetings. . )

INSPIRE Geoportal Workshop 23 -24. 1. 2019 Together improving accessibility of MS data sets Geoportal WS participants: • 45 Experts from 28 EU Countries + EC & EEA INSPIRE Coordination Team Expectations summary: • • • Clarifications - data - service linkages Understand better the Geoportal / harvesting checks logic New Geoportal features / functionality More Priority datasets / downloadable datasets Understand better what is expected from MS INSPIRATION for MS level (also MS Country ICT overview)

MS Survey on the INSPIRE Geoportal • • Launched in December 2018 31 answers received (except for Bulgaria) 1 reply per country except for Belgium (2 ) Divided into three main sections: • 1. Technical description of your system use • 2. Organizational description of your system use • 3. Improvement of the Geoportal • Background documents provided • Geoportal workflow for establishing links between data sets and network services (PDF) • Individual report per cuntry with resource linking issues

Data/service linking Additional information: Data-service linking - New national projects to improve (4) 30 - National workshops/webinars upcoming (3) 26 - JRC doc. on Geoportal data/service linking not clear only for (1) 25 20 - Correct links only for harmonized INSPIRE datasets (2) 15 - There is still a value in providing only MD for datasets without services (2) 11 10 7 4 5 3 0 1 - Clear - done 2 - Clear - working progress 3 - Different solution 4 - Not MS priority/no resources to improve 5 - Do not how to do it. - Due to the WFS problems – ATOM feed only (2) - + Technical issues (missing extended capability for WFS, WFS 2. 0 generation, WFS Stored queries. . ) related to the use of various SW packages The majority of MSs INSPIRE coordinators know - how to set up the correct links between data and services. The controlling and updating of MD of National “resources” is on-going activity.

MS Metadata organisation – harvesting mode Do you harvest automatically metadata of data sets and services from your data providers or from regional catalogues? 25 20 20 15 11 10 5 0 Yes No

Metadata organisation – For those who do harvest automatically How often do you harvest metadata from your regional and data providers’ catalogues? 14 Do you use pull or push method for harvesting metadata? 16 13 14 12 12 10 10 8 6 4 3 4 8 6 2 4 0 2 Daily or mostly daily Monthly or mostly monthly According to data providers needs 11 Pull Push Provide a description of your implemented harvesting solution 14 9 10 • CSW • Webdav (WAF) 3 0 Have you implemented incremental updates to harvest metadata? 12 • Different catalogues interfaces are often supported: 17 18 12 12 10 8 8 6 6 6 4 4 2 2 2 0 0 Yes No Many to 1 node 1 to 1 node No data • Reporting and approval system for publication often put in place.

Software used for the management of metadata Software used for management of national metadata of data sets and services 30 25 25 20 15 10 6 5 0 Main software Combination Some countries are planning: • to move to Geonetwork • upgrade their current Geonetwork’s version

Metadata software versions used & main issues Geonetwork • Index issues - duplicated metadata Geoportal Server MDE Server • Download errors with stored query • Requires still too much manual operation on raw XML (Tagging of priority datasets, resource linking) • Doesn’t drop off decimals of coordinates without rounding up • Opensearch document is not complete (missing information) • Metadata editor doesn't produce 100% INSPIRE compliant metadata. • Not possible to import new thesaurus, code lists • Metadata anchor element not supported (manually corrected metadata, is not valid for ESRI validator the file cannot be uploaed • Have to remain on an older version of Geoportal software as newer versions of CSW not allowed under INSPIRE technical guidelines • Mix of issues with MD editor and harvesting process

View services software versions used & main issues Esri • Sometimes, different versions of software used by the same organisation depending on the resource targeted • INSPIRE extensions and plugins are often used with Geoserver & ESRI • WMS service configuration environment allows to add only one Layer Identifier authority attribute per service • Bad performance of service Deegree • view services render slow

Download services software versions used & main issues Geoserver • • • Esri Arc. GIS server • • • Impossible to provide a single INSPIRE stored query Geoserver lists all stored queries, also for other datasets. Impossible to provide many INSPIRE themes dataset on one Geoserver because in a feature type you can’t refer to the same namespace. Impossible to establish different security policies for different WFS services Bugs in the App Schema extension Includes invalid parameter in the Get. Map request: EXCEPTION= application/vnd. ogc. se_xml Limitation on the maximum number of records that can be downloaded via a REST Feature Access service Sometimes a WFS service works on QGIS but at the same time is unresponsive in Arc. GIS Impossible to configure INSPIRE extended properties for metadata in Get. Capabilities Impossible to export SLD legend of dataset Limit on the number of features loaded Slow reload for complex datasets and rectified grid in GML Deegree • • • Mapserver • Doesn’t have an app-schema plugin. Therefore we can't deliver harmonised data on-the-fly Geonetwork • ATOM/Open. Search passes 39 out of 40 test cases on the etf-webapp HALE connect • Application Schema for geology theme has issues with cardinality of Geofeature object

General issues with implementing INSPIRE Additional information: Generic INSPIRE implementation issues 9 8 8 7 6 5 4 4 4 3 3 2 1 - Competition with other Regulations e. g. e. Reporting - Semantic Interoperability – heavy load - Complex “business” calculations - Data at “zero cost” = not priority for publishing 0 Data sharing, harmonisation of licensing, open data strategy Data/service creation and maintanence Lack of resources, skills, bennefts National coordination / - low priority Involvement of data providers Any suggestions on how to raise the priority at MS (Data providers) for INSPIRE implementation? Could data/service creation and maintenance issue be solved/partially by not using INSPIRE infrastructure exclusively?

INSPIRE Geoportal Workshop 23 -24. 1. 2019 Together improving accessibility of MS data sets Main conclusions: 1) Geoportal back-end & data-service linking • Back-end better understood • Data-service linking harvesting checker is already more flexible Actions: JRC: Update the data-service linking doc. and communicate widely (MIG-T, MIG) JRC/MS: Start testing of simplified approach for data-service linking JRC/MS to develop Resource Linking Testing Tool JRC will investigate integration with Reference Validator 2) Improving data/service organisation for publishing Focus on organization of data for download/view and authentication Open data – see the results from the Art. 17 DG ENV Study follow-ups Actions: JRC to summarise good practices and technical solutions in the Workshop report MS to propose additional good practices

INSPIRE Geoportal Workshop 23 -24. 1. 2019 Together improving accessibility of MS data sets Main conclusions: 3) Harvesting - new system • Proposed harvesting console appreciated, more feedback after testing • Regular harvesting should be offered (monthly) Actions: • Priority improvements: metadata 2. 0 support, incremental harvesting, and new simplified dataservice linking approach • For both issues, JRC will contact MS for suggestions, examples 4) Geoportal - new functionalities New Geoportal UI - appreciated Actions: • JRC to work on priority improvements: INSPIRE analytics, language support, indexing by search engines (link to MIG-T activity) • JRC to request suggestions for API and language support

INSPIRE Geoportal Workshop 23 -24. 1. 2019 Main conclusions: 5) Geoportal Helpdesk § Implementation / harvesting knowledge base –very useful Actions: JRC to reorganize the Helpdesk – FAQ’s, closing issues etc. 6) Future meetings / group organisation etc. • Physical meeting is useful (~ 1/year) also include presentations from MS (success stories, issues) • Ad-hoc virtual meetings for dedicated issues, e. g. harvesting development of new features data-service linking success stories from MS Actions: JRC: All documents to be published on the MIG collaboration space https: //webgate. ec. europa. eu/fpfis/wikis/display/Inspire. MIG/INSPIRE+Geoportal+Workshop JRC: to keep the mailing list of MS catalogue Po. Cs – will be developed based on the MS expert nominations for Harvesting console.

INSPIRE Geoportal Workshop 23 -24. 1. 2019 Geoportal WS participants after meeting

INSPIRE GEOPORTAL – Data sets availability

2018 -09 -20 -> 2019 -01 -21 INSPIRE GEOPORTAL – Priority data sets Country Austria Belgium Bulgaria Croatia Cyprus Czech Republic Denmark Estonia Finland France Germany Greece Hungary Iceland Ireland Italy Latvia Liechtenstein Lithuania Luxembourg Malta Netherlands Norway Poland Portugal Romania Slovakia Slovenia Spain Sweden Switzerland United Kingdom Sum Ratio 86. 84% 31. 73% 0. 00% 100. 00% 10. 26% 2. 63% 4. 76% 36. 84% 0. 00% 61. 54% 0. 00% 100. 00% 86. 57% 92. 59% 29. 27% 11. 11% 0. 00% 9. 42% 42. 86% 16. 67% 25. 00% 97. 53% 0. 00% 11/12/2018 # Metadata # Downloadable 76 66 104 33 0 0 6 0 5 5 39 4 38 1 21 1 19 7 105 0 65 40 0 0 12 0 2 2 9 0 12 0 0 0 0 67 58 54 50 41 12 9 1 0 0 223 21 21 9 12 2 4 1 81 79 24 0 0 0 1049 392 10/23/2018 Ratio # Metadata # Downloadable 88. 00% 75 66 32. 71% 107 35 0. 00% 0 0 0. 00% 6 0 100. 00% 5 5 10. 26% 39 4 2. 63% 38 1 4. 76% 21 1 42. 11% 19 8 0. 00% 26 0 59. 26% 54 32 0. 00% 0 0 100. 00% 1 1 0. 00% 9 0 0. 00% 12 0 0. 00% 0 0 83. 58% 67 56 92. 59% 54 50 29. 27% 41 12 11. 11% 9 1 0. 00% 0 0 2. 82% 213 6 40. 91% 22 9 16. 67% 12 2 100. 00% 1 1 98. 77% 81 80 0. 00% 24 0 0. 00% 0 0 936 370 9/20/2018 Ratio # Metadata # Downloadable 86. 49% 74 64 33. 33% 105 35 0. 00% 0 0 0. 00% 6 0 100. 00% 5 5 10. 26% 39 4 13. 51% 37 5 4. 76% 21 1 0. 00% 19 0 0. 00% 0 0 56. 92% 65 37 0. 00% 0 0 0. 00% 9 0 0. 00% 1 0 0. 00% 0 0 84. 62% 65 55 92. 59% 54 50 34. 04% 47 16 0. 00% 7 0 0. 00% 0 0 2. 86% 210 6 9. 09% 11 1 16. 67% 12 2 100. 00% 1 1 83. 95% 81 68 0. 00% 21 0 0. 00% 0 0 890 350

MIWP action 2019. 2: Improving data availability • Phase 1 (January – June 2019) • Further investigate and document the issues causing the low level of data accessibility (through the Geoportal workshop in January 2019 and follow-up analyses) • Jointly work on improving data accessibility in the INSPIRE geoportal (through changes in national discovery services and/or the EU Geoportal) following the current approach • Test the proposed simplified approach for data-service linking (MS metadata & services and INSPIRE geoportal) • Report status and propose follow-up actions to MIG meeting in June 2019

INSPIRE Geoportal Harvesting Console Together improving accessibility of MS data sets Main features: • Provides full MS control of the harvesting process including its initiation, monitoring/reviewing and publishing of harvested MD; • Supports MS with indications of concrete MD records with possible errors found e. g. missing resource linkages; • Reduces the number of requests from MS for the ad-hoc harvesting.

INSPIRE Geoportal Harvesting Console Together improving accessibility of MS data sets DEMO

INSPIRE Geoportal Version 1. 1 Together improving accessibility of MS data sets Main new features: Geoportal viewer: • new search functionality in the results list page; • new view service URL copy functionality; • added the Priority dataset keywords in the metadata detail page; • support to the metadata translation. Geoportal backend - adding more flexibility for WFS Download services: • all 4 sources of Unique Resource Identifiers are now considered and offers a detailed insight on where coupled resources were actually specified. • In case WFS does not make available a suitable Stored Query for predefined datasets and exactly one coupled resource is identified, the whole content of the WFS is considered as an as-is dataset. In this case, the dataset download link is built as a standard Get. Feature request.

Geoportal Resource Linkage Verification Tool Main features: • Provides MS data providers a SW tool to check whether the necessary linkages between MD – Services (View – Download) resources are provided correctly / consistently; • Eliminates/reduces the use of the Geoportal harvesting as a testing environment.

Simplification of data – service linking • Received confirmation of the interest to participate in the testing from: 11 countries IT, CZ, NL, LT, PL, FR, SE, SK, CY, EL, ES • Kick-off meeting (15. 04. before Easter or 29. 04) Required contribution before the kick-off: • MS to provide examples of extended metadata with links to functional services – before the kick- off • MS to provide information on the future strategy for the change deployment (all MD records to be update or only new/with wrong linkages etc. . ) Kick-off meeting to compare implementations and discuss open issues (e. g. identification of the use MD schema in case old and simplified ways are used together) • Update/extend discussion paper based on testing results • Report to MIG in June

INSPIRE Geoportal Version 1. 2 – 2. 0 Together improving accessibility of MS data sets Main new features V 1. 2: (June/July 2019) • • • Full multilingual support (provided + use of automatic translation EU services List of Feature types present in data sets Support of Metadata V 2. 0 Support of WCS, SOS Download services Bug fixing (Harvesting console etc. ) Version 2. 0: (October/November 2019) • Monitoring indicators • New INSPIRE Analytics platform – open API • Better integration with the Reference validator – maturity levels

Thank you Questions? robert. tomas@ec. europa. eu Make it work together