Part 1 Introduction to DGGS data and queries

Part 1: Introduction to DGGS data and queries Robert Gibb Manaaki Whenua – Landcare Research (NZ) 25 February 2021 New Opportunities for Geospatial and Statistical Data using DGGS (Discrete Global Grid System) Coffee Talk 2 https: //statswiki. unece. org/display/GFGS/Webinars Copyright © 2020 Open Geospatial Consortium OGC ®

DGGS – as a Reference system OGC What are Reference systems Referencing by coordinates Referencing by identifiers ogc. org | 2

C So what is a Reference System – OGC & ISO o p standards y r • RS is a ‘system for identifying position in the real world’ i OGC • traditional spatial or temporal RS provide answers to either g h t where is it? or when is it? © 2 0 • Two types in ISO TC 211 and OGC • Referencing by coordinates: O p e n G e o s p a t i a l • ISO 19111: 2019 – ‘geographic’ coordinates, • ISO 19108: 2002 – temporal coordinates & calendars. • Referencing by identifier: • ISO 19112: 2017 – geographic identifiers, • ISO 19108: 2002 – ordinal era as temporal identifier. Examples: • [40 o 22’ 44” S, 175 o 34’ 29” E] or [13 S 500245. 9 m 4398197. 8 m] • 2459130. 5 JD or 2020: 10: 08 • London, New Zealand, Yorkshire, The Alps, CO 80204 • Cenozoic, Jurassic, Anthropocene These reference systems reflect thinking over many centuries BUT are they fit for purpose for the demands of spatial data? ogc. org | 3

C o p y r i g h t Many vector data types and file types © 2 0 O p e n G e o s p a t i a l • point feat-id + coord tuple OGC • point cloud id + coord tuple • line feat-id + ordered list of points • polygon feat-id + interior point + ordered list of lines • tag cloud coord tuple per tag Feature geometry is structured in a file format, of which there are many, height & time may be added coordinates, or attributes, or put in metadata & Topology is typically computed from the feature geometry ogc. org | 4

C o p y r i g h t OGC Many raster data types © 2 0 O p e n G e o s p a t i a l Earth Observation • raster data – many unrelated cell sizes dependent on sensor size – many image data formats • geotiff, hdf 5, netcdf, jpeg 2000 … + 100 s of others • gdal format conversion – many API • WMS, WCS, + commercial • raster tiles – tile sizes and names are implementation specific Map or Globe Viewer • raster data - time • map may bedata in metadata, or cell sizeinmatched to or zoom inferred a file name band or an resolution level, name, and viewer associated lookup table, or occasionally in per pixel metadata. • map tile sizes and names are implementation specific, tile hierarchy matched to zoom levels Overall Vector & Raster: CRS is independent of dataset formats and API !!! & the CRS doesn’t provide Geometry or Topology ogc. org | 5

Referencing by identifiers – names for places OGC Gazetteers disambiguate identifier use & provide topological hierarchies: • Countries are union of states, counties, regions … § States, Regions are contain cities, suburbs (sort of) with geo. SPARQL provides spatial language & reasoning • Suburbs touch other suburbs • Streets, houses are in suburbs (sort of) . . this is knowledge represented as rdf facts • Identifiers have position, and topological facts, they may have formal boundaries Topology . . . Geometry is optional any database, xml, json can store lists of identifiers and rdf facts & spatial or temporal specialisation is a language issue & not a data format issue. Copyright © 2020 Open Geospatial Consortium ogc. org | 6

C o p y r i g h t © 2 0 O p e n G e o s p a t i a l C o n s o r t i u m Referencing by … ? OGC So what if we had a referencing system that natively provides: Identifiers, Position, Geometry & Topology? can we retain the simplicity of referencing by identifiers, make position & topology explicit and precise, and add geometric robustness of referencing by coordinates? ogc. org | 7

OGC Referencing by … ? OGC Abstract Specification Topic 21 v 2. 0 & ISO 19170 -1. . both submitted for publication in Jan 2021 Specifies Referencing by zonal identifiers with structured geometry as the Reference System in a Discrete Global Grid System Copyright © 2020 Open Geospatial Consortium ogc. org | 8

DGGS – Reference System Data OGC What are Reference Systems Referencing by coordinates Referencing by identifiers DGGS – Referencing by zonal identifiers with structured geometry DGGS Reference System Data Is there such a thing as RS Data? ogc. org | 9

DGGS implementation of various vector data types • DGGS – point list of 1 x identifier OGC • DGGS – point cloud list of identifiers • DGGS – line directed list of identifiers • DGGS – polygon • DGGS – tag cloud list of identifiers Lists of identifiers, directed lists don’t need special data formats Bonus 1: a DGGS zone ID can represent both time & space Bonus 2: DGGS zone size ~ spatio-temporal precision ogc. org | 10

DGGS equivalents of various raster data types Earth Observation • DGGS – raster data 1 x list of identifiers + many data attributes = DGGS data cube • DGGS – raster tile identifier is tile name & parallel job management OGC Map or Globe Viewer • DGGS – maps cell array zone size = zoom level • DGGS – map tiles identifier is tile name DGGS zone size ~ data precision and tiling performance Raster file formats can store spatio-temporal DGGS data too ogc. org | 11

DGGS – DGGS data – WKT type structure List of Zone. IDs Type Ordinate. List set of n. D points Cell. List set of n. D regions or spaces Tile. List set of tiles or ‘b-box’ Zone. Collection collection of other type(s) optional OGC Directed Lists of Zone. IDs Examples Type Examples POINT, MULTIPOINT Directed. Ordinate. List n. D line vertices LINESTRING, POLYGON (boundary), TRIANGLE, route, trajectory, timeseries POINT, MULTIPOINT, POLYGON, MULTIPOLYGON, point cloud, raster or WMS image, geo-located social media Directed. Cell. List n. D connected path route, trajectory, timeseries, corridor, path Tile. Set, DB shard, b-box, jobqueue MULTILINESTRING, GEOMETRYCOLLECTION Directed. Tile. List sequence of ‘bbox’ Directed. Zone. Collection graph or connected collection of similar or different type(s) ordered jobqueue TIN, network (of Directed. Ordinate. List or Directed. Cell. List) sort by decreasing size, compress ogc. org | 12

DGGS – Reference System data Traditional RS – No There is no WGS 84 data or UTM 3 data. OGC DGGS – Yes DGGS Reference Systems define their own geometry. Traditional RS are independent of the choice of feature geometry and (spatial) data format. Geometry is implied by Zone. ID, i. e. , an identifier. It’s the DGGS RS’s Navigator functions or language that “make these identifiers spatial/ temporal”. complexity simplicity ogc. org | 13

DGGS – Reference System Language OGC What are Reference Systems Referencing by coordinates Referencing by identifiers DGGS – Referencing by identifiers with structured geometry Reference System Data Is there such a thing as RS Data? Traditionally - No DGGS - YES! DGGS Reference System Language If we have data & identifiers, show me the language? ogc. org | 14

OGC DGGS’s spatio-temporal language Operators using Zonal identifier implement topological relations for: • • boundary, distance, buffer*, difference*, intersection*, sym. Difference*, union*, contains, touches, disjoint, within, equals, overlaps, within. Distance, intersect, relate Plus new operators for navigating the resolution hierarchy: • child*, parent* & sibling*, child. Of, parent. Of & sibling. Of Plus new 1 D operators borrowed from time can query any dimension: (eg above & below in time or before & after in height) • relative. Position, relate. Position operators marked * control the level-of-detail of their result sets, & all operators can be constrained to lower dimensionality e. g. 2 D query on a 3 D+T zonal identifier Copyright © 2020 Open Geospatial Consortium ogc. org | 15

DGGS: topological functions on Zonal. Identifier OGC <<query>> (1 D) relative. Position(another: Zonal. Identifier, project. To: Direct. Position[4]) <<query>>contains (another: Zonal. Identifier, project. To: Direct. Position[4]) crosses (another: Zonal. Identifier, project. To: Direct. Position[4]) disjoint (another: Zonal. Identifier, project. To: Direct. Position[4]) equals (another: Zonal. Identifier, project. To: Direct. Position[4]) intersects (another: Zonal. Identifier, project. To: Direct. Position[4]) overlaps (another: Zonal. Identifier, project. To: Direct. Position[4]) touches (another: Zonal. Identifier, project. To: Direct. Position[4]) within. Distance(another: Zonal. Identifier, dist: Distance, project. To: Direct. Position[4]) parent. Of (another: Zonal. Identifier, inherit. ID: Boolean, project. To: Direct. Position[4]) child. Of (another: Zonal. Identifier, inherit. ID: Boolean, project. To: Direct. Position[4]) sibling. Of (another: Zonal. Identifier, inherit. ID: Boolean, project. To: Direct. Position[4]) <<set>> buffer (dist: Distance, project. To: Direct. Position[4]) difference (another: Zonal. Identifier, range. Refine: refinement. Level. Range, project. To: Direct. Position[4]) intersection (another: Zonal. Identifier, range. Refine: refinement. Level. Range, project. To: Direct. Position[4]) sym. Difference(another: Zonal. Identifier, range. Refine: refinement. Level. Range, project. To: Direct. Position[4]) union (another: Zonal. Identifier, range. Refine: refinement. Level. Range, project. To: Direct. Position[4]) parent (inherit. ID: Boolean, levels: Integer, project. To: Direct. Position[4]) child (inherit. ID: Boolean, levels: Integer, project. To: Direct. Position[4]) sibling (inherit. ID: Boolean, levels: Integer, project. To: Direct. Position[4]) <<reference>> (1 D) <<reference>> relate. Position(another: Zonal. Identifier, relate: Relative. Position, project. To: Direct. Position[4]) relate (another: Zonal. Identifier, matrix: Character. String, project. To: Direct. Position[4]) ogc. org | 16

OGC DGGS – Examples What are Reference Systems Referencing by coordinates Referencing by identifiers DGGS – Referencing by identifiers with structured geometry DGGS Reference System Data Is there such a thing as RS Data? Traditionally - No DGGS - YES! DGGS Reference System Language If we have data & identifiers, show me the language? YES. . All the old topological operators, but now they are spatio-temporal, and they still fit with OGC API architecture. How can we apply it Give me some examples. . ogc. org | 17

New Opportunities for Part 2: Application of DGGS as a Geospatial and Statistical Data using DGGS basis for statistical areas Matt Purss Pangaea Innovations (AUS) 25 February 2021 (Discrete Global Grid System) Coffee Talk 2 https: //statswiki. unece. org/display/GFGS/Webinars Copyright © 2020 Open Geospatial Consortium OGC ®

Discrete Global Grid Systems: An overview Tetrahedron Octahedron Cube OGC Equal Area Tessellations of the Earth’s Surface Model Dodecahedron Icosahedron Simple cell Types: • • • n. D Spatial Analyses ↓ 1 D Array Processes 00 01 02 03 10 Fixed Cell Hierarchy Rectangular Triangular Hexagonal Globally Unique Cell Indices 11 12 13 20 21 22 23 30 31 32 33 Massively Scalable ogc. org | 19

Does DGGS have a role in current Statistics Programs? OGC (GFGS Coffee Talk 1 – Webinar 1) "Defining census enumeration areas, a new method" by Ana Santos, Statistics Portugal The use of irregular and “complex” geometries based on dynamic data as the base unit for statistical analysis is problematic: • • Provenance Challenges across Census datasets Precision errors can lead to slivers and geospatial errors between Enumerated Areas Aggregation of Enumerated Areas is non-trivial Spatial queries of Enumerated Areas is nontrival ogc. org | 20

Does DGGS have a role in current Statistics Programs? BGRI 1991 Spatio-temporal Comparison between Census datasets is a simple index lookup Census BGRI 199 1 Enumerated DGGS Zone. IDs Area ID – Resolution 9 & 10 EA-1 Q 1345893482, Q 1345893483, Q 1345893487, Q 1345893488, BGRI 200 1 EA-1 Q 134589348, Q 1345893482, Q 1345893483, Q 1345893484, Q 1345893486, Q 1345893487, Q 1345893488, DGGS provide a simple, globally consistent, hierarchical and fixed set of geometries that can support data comparisons within and across statistical datasets Q 1345893492, Q 1345893495, Q 1345893496, DGGS Zone. IDs – Resolution 11 Q 13458934821, Q 13458934823, Q 13458934824, Q 13458934828, Q 13458934829, Q 13458934845, Q 13458934864, Q 13458934865, Q 13458934868, Q 13458934821, Q 13458934822, Q 13458934823, Q 13458934824, Q 13458934826, Q 13458934827, Q 13458934828, Q 13458934829, Q 13458934871, Q 13458934872, Q 13458934873, Q 13458934874, Q 13458934876, Q 13458934877, Q 13458934878, Q 13458934879, Q 13458934836, Q 13458934837, Q 13458934838, Q 13458934839, Q 13458934881, Q 13458934882, Q 13458934883, Q 13458934884, Q 13458934886, Q 13458934887, Q 13458934888, Q 13458934889, Q 13458934845, Q 13458934846, Q 13458934871, Q 13458934872, Q 13458934873, Q 13458934874, Q 13458934881, Q 13458934882, Q 13458934883, Q 13458934886 Q 13458934836, Q 13458934837, Q 13458934839, Q 13458934862, Q 13458934863, Q 13458934864, Q 13458934865, Q 13458934868, BGRI 2001 OGC Q 13458934893, Q 13458934894, Q 13458934895, Q 13458934898, Q 13458934922, Q 13458934925, Q 13458934926, Q 13458934927, Q 13458934951, Q 13458934952, Q 13458934953, Q 13458934954, Q 13458934957, Q 13458934958, Q 13458934959, Q 13458934961, Q 13458934962, Q 13458934963, Q 13458934964, Q 13458934966, Q 13458934967, Q 13458934968, Q 13458934969, Q 13458934972, Q 13458934973, Q 13458934975 ogc. org | 21

Does DGGS have a role in current Statistics Programs? OGC (GFGS Coffee Talk 1 – Webinar 1) Raster Vector "New tools in creating and monitoring Enumeration Areas (EAs) for the 2020 round population censuses" by Mathias Kuepie, UNFPA / GRID 3 Project DGGS – Common geometry + common API = unified analysis across the raster-vector divide ogc. org | 22

OGC DGGS – Unifying key What are Reference Systems Referencing by coordinates Referencing by identifiers DGGS – Referencing by identifiers with structured geometry DGGS Reference System Data DGGS is the unifying key Is there such a thing as RS Data? Traditionally - No DGGS - YES! DGGS Reference System Language If we have data & identifiers, show me the language? YES. . All the old topological operators, but now they are spatio-temporal How can we apply it Give me some examples. . DGGS is a simpler Common global geometry & API framework ogc. org | 23

Discrete Global Grid Systems are the Key OGC • DGGS provide a consistent and common spatial framework that can provide a solid basis for working with statistical areas across jurisdictions Why? • Inherently Multi-Resolutional • Simple and Fixed Geometries • Global Coverage • Unique Spatial Identifiers • Can be used in a Federated Multi. Organisation Spatial Data Infrastructure. ogc. org | 24

DGGS Bringing it all together • Search and Query operations simplified • • Vector – Raster divide is bridged Scalable • • Array Set Theory / DB Index Lookup vs Spatial Computation Harmonised Data Model • • OGC Aggregate statistics achieved via summations/weighted averaging across simple “Equal-Area” cells User access privileges can be implemented • e. g. General public able to interact down to a certain prescribed resolution level, while data curator/admin able to interact down to the precision of the data. ogc. org | 25

DGGS Bringing it all together • DGGS can seamlessly work across jurisdictions • • • OGC Common global spatial data infrastructure is not dependant on agreement between individual data custodians. DGGS indexing of data enables consistent provenance of features to be recorded and tracked over time. OGC APIs for DGGS • Building a common web interface to support standardised access to multiple DGGS instances and non-DGGS enabled data. ogc. org | 26

OGC DGGS – What is coming next What are Reference Systems Referencing by coordinates Referencing by identifiers DGGS – Referencing by identifiers with structured geometry DGGS Reference System Data Is there such a thing as RS Data? Traditionally - No DGGS - YES! DGGS Reference System Language If we have data & identifiers, are the operators that form a language? YES. . All the old topological operators, but now they are spatio-temporal DGGS So what’s coming next ? DGGS is the unifying key DGGS easily brings everything together How can we apply it Give me some examples. . DGGS is a simpler Common global geometry & API framework ogc. org | 27

DGGS Next Steps cf - OGC 20 -039 OGC Testbed-16: DGGS and DGGS API ER 1 OGC Mature DGGS Reference Library Implementations Explore and elaborate DGGS Common for OGC APIs Explore opportunities and limitations of DGGS driven analytics & Elaborate DGGS specific OGC APIs Implement OGC Registries for DGGS implementations and data services Note 1: OGC 20 -039: https: //docs. ogc. org/per/20 -039 r 2. html ogc. org | 28

DGGS Many thanks OGC Many thanks. . questions & discussion. . Robert Gibb gibbr@landcareresearch. co. nz Matthew Purss mpurss@pangaeainnovations. com ogc. org | 29

DGGS – OGC API defined and implemented Implementations OGC API-Processes OGC API-Features Services present DGGS RS as collection of virtual vector layers. Geoserver implementation of OGC API-Processes Implemented Linked Data approach with an RDF data store. Very successful, results feeding into Geo. SPARQL revision. Vector data for Australia’s Level 1 Statistical Meshblocks and Hydrological Catchments were converted to DGGS RDF data with the DGGS Zone. ID as the predicate (i. e. , primary index) for the data. Each layer (item in the collection) corresponds to a level of the DGGS RS grid hierarchy. Clients can request: either JSON (with just Zone. IDs) or Geo. JSON (with Zone. IDs augmented with coordinates created on-the-fly). . Single wrapper library was created to deliver both H 3 and r. HEALPix, as virtual vector layers. All of Geoserver functionality provided to DGGS. Allowed exposing WMS and WFS to clients as a BONUS Attached to a backend DGGS Datastore with Zone. ID as primary index for Sentinel-2 pixel in a No-SQL DB. Proved very fast for both Sentinel-2 data, statistics, and index calculations. The OGC API - Features end -point presented this data as either JSON LD - using the DGGS Zone. ID as the only representation of geometry, or as Geo. JSON OGC DGGS Client Native DGGS Mapping Clients built to consume and overlay OGC API-Features and OGC API Processes. Desktop and Jupyter Notebook clients created. Participants delivered native DGGS vector and EO capabilities to users of both traditional GIS clients and native DGGS clients. Showing versatility of DGGS as a chameleon. ogc. org | 30

DGGS – OGC API defined and implemented OGC ogc. org | 31

OGC API Processes results with Sentinel 2: OGC B 04, B 03, B 02 & NDBI, NDVI, NDWI ogc. org | 32

OGC DGGS – reference library candidates Name & Author Cell geom. Repo & & Licence Refinement Langua ge DGGS Prov. Quantization DGG Interop S Nav. erabilit y H 3 Uber Hex 1: 7 Github Apache 2. 0 C, java Y single N Y N r. HEALPix R Gibb, Manaaki. Whenua Quad 1: 9 (+others) Github LPGL Python Y multi Y (cf Aus. PIX) Y N DGGrid Hex or Tri K Sahr, S Oregon Univ. 1: n Githib AGPL 3. 0 C++, cmd line Y multi N N N Open. EAGGR DSTL & Risk. Aware Tri or Hex Github LPGL 3. 0 C Y multi N N N DGGrid. R (cf DGGrid) Richard Barnes Hex or Tri 1: n Github R Y multi Y N N ogc. org | 33