GIS Data Acquisition Management Spatial Analysis Visualisation Owen

GIS Data Acquisition & Management; Spatial Analysis; Visualisation Owen Macdonald, GIS Support Officer, Geo. Sciences, Uo. E Owen. Macdonald@ed. ac. uk

Outline • Background/Basics • The Everyday – What do you need? Desktop Package or Web Map? • Visualisation – GIS Limitations/Novel Techniques

Background/Basics Quick Intro to GIS

What is GIS? • Geographical Information System…GI Science (Poss. rebrand: Spatial Data Science) • Conceptualising reality in a computer model • Underpinned by spatial analysis; more than just maps or manipulation • Why is GI important? • Everything happens somewhere • Ability to attach multitude of information to locations/geography units • Relationships between phenomena/variables • spatial coincidence • relate/transfer info (attributes) between layers where these coincide

GIS Workflow • Data acquisition/integration • Data management/database management • Data analysis/Spatial Analysis • Cartography/Visualisation

GIS Functionality • Data acquisition/integration • (Scan and) digitise • Tabular Data with Spatial Location (Point or Centroid); or can join to other located data/geography units • GPS/Survey/Laser Scanning/LAS/Point-Clouds • Aerial/Satellite Image • Data management/database management • • • Storage Editing Metadata (describe datasets meaningfully and in terms of quality/coverage) Indexing (speed up data retrieval) Map Projections – in here as apply to both analysis and visuals…

GIS Functionality • Data analysis/Spatial Analysis – Map 'Layers' (Layer Stack); Spatial Co-incidence of layers • • Querying; Select by Attributes; Select By Spatial Location; Proximity Analysis/Buffering Overlay/Intersect etc. Interpolation of irregularly spaced data/sample points Spatial Statistics – point patterns, hotspots/clustering, spatial autocorrelation Scale/Aggregation/Zonation issues – MAUP, spatial autocorrelation, areal interp, dasymetric map Spatial is 'a bit' special – but can apply various techniques above to any 2 D/3 D 'space' Decision making • Cartography/Visualisation • • • Display new computed variables/statistics Mostly 2 D though long-record of 3 D/4 D research; Novel techniques to present complex info in 2 D; ESDA; 'new' field of Visual Analytics 3 D GIS emerging (traditionally '2. 5 D') Decision making; Interpretation

Data Models and Types • Essentially two common models of space • Discrete Entity • Continuous Field • Discrete Entity (== Vector Types) • Objects exist with empty space in-between • Modelled in GIS by Vector data types/structures – points, lines, polygons • Continuous Field (== Raster Types) • Continuous grid of raster cells, each has a value for a single variable of interest • Empty space explicitly represented by No. Data (cf. NULL) cells, within grid extent

Data Types: Vector • Discrete entities with specific locations; empty space between • Multiple attributes for each feature Point Polyline Polygon Geographical Information Systems: Principles, Techniques, Management and Applications, 2 nd Edition, Abridged Introduction (P A Longley, M F Goodchild, D J Maguire, and D W Rhind). : 77.

Data Types: Raster • Continuous surface with changing values; 'empty' No. Data cells where req’d • • Elevation and derivatives Satellite imagery Photographs Scanned maps • Attributes • Each cell has single value • This can relate to Value Attributes – NB attributes of the value held by the cell, not the cell itself nor its spatial location!

From ESRI Map Book Volume 24, www. esri. com

Other Model/Data Types • Firstly, there are some other model frameworks/types: • Triangular Irregular Networks TINs ('vector surface'!); variable res’n across surface • Also ESRI (Arc. GIS) specific – Terrains, Multipatches; 2 D-3 D interface • However: • Built from points (TINs), and lines, polygons, groups of points, large point-clouds • Also: Other Data Formats • Binary Formats – handle large data volumes, compact size • Multidimensional data – Net. CDF

Open GIS – Formats, Apps, Data • Open Standards / Formats – Open Geospatial Consortium (OGC) • • GML; City. GML(3 D); KML (GE); Geo. JSON (Java. Script - NB Geo. JSON not maintained by OGC) Moving Features Net. CDF WMS; WMTS; WFS • Open. Source Applications – Open Source Geo. Spatial Foundation (OSGeo) • QGIS/GRASS (use GRASS in QGIS; QGIS plugins; extend with Python) • gv. SIG/u. Dig (former extensible/mobile; latter Java/Eclipse based) • RDBMS+Spatial Extensions – e. g. Postgres with Post. GIS extensions (also pg. Routing) • Open. Data • OS Open. Data (since 2010); Also extra OS/other data avail to HE via EDINA Digimap • Open. Street. Map (linked to likes of Open. Layers/Leaflet etc. ) – VGI (Volunteered GI) • Also traditionally ‘open’ providers: Natural Earth, USGS

Changes in GIS • Server GIS • Dumb-terminal connects to a server (possibly in same building) • Desktop GIS • Typical standalone full GIS package on desktop PC (e. g. Arc. GIS, Quantum GIS/QGIS) • Web GIS / Distributed GIS • Online processing services, Web Map/Feature Services, Remote Servers • Client-Side Web GIS • Thin or Thick Client, GIS in your web browser, Java. Script, Arc. GIS Online • Mobile GIS • GIS on your tablet/phone/PDA – Arc. Pad, gv. SIG • Ubiquitous GIS • Sensor Networks, GPS Trackers, Location Based Services, Geo-aware APIs

Everyday GIS What do you actually need?

Everyday GIS on your PC – Different Levels of 'GIS' • Do you really need 'Desktop'? – Ignorance of Arc. GIS no longer a crime! • • Google Maps Google Earth/Arc. Globe (KML/KML Timestamp) (Spatial) Database – render on webpage in SVG Need better SQL querying? – Store in Database: Oracle/Oracle Spatial, Postgres/Post. GIS Connect GIS to DBMS; Also connect R/stats package to GIS/Database Do all in R? Python? • Can access Arc. GIS/QGIS codebase directly without GUI performance issues • Or, mapplotlib basemap, Py. SAL – ESDA • Do you need more (specialist) than Desktop? – We’ll look at these later… • Arc. GIS Server/New Arc. GIS Pro (integrated 2 D 3 D environment) • ESDA – Geo. Da, CAST, Py. SAL – again perhaps R? • Crime Analysis - Crime. Stat

GIS in your Browser • Google Maps • Even simple click+create maps (but now may require some degree of login – My. Maps) Or can run a web script, e. g. : • Generates a KML doc and returns: • • URL to KML Google Maps URL to KML User decides which 'viewer' to use Allows for poor internet connections

GIS in your Browser • Or can use Java. Script and GM API within a web page using a map 'object' • HTML+Java. Script • Code to produce example image shown, is included below • Can add placemarks etc. to Java. Script (and thus map) • Mashup (verb. ) whole datasets on top of e. g. Google! Weather, Photos, Locations…

Map Mashups • Java. Script-powered web pages can also use HTML Forms to retrieve user input: • Co-ordinates • Origin/Destination Postcode • URLs built using web map sites’ APIs e. g. • www. streetmap. co. uk/newmap. srf? x=3210 00&y=621000&z&ar=n • Hit Go! and returned are one, or more, geo web pages: • Maps • Routing (e. g. Multimap, Google)

Map Mashups • Java. Script-powered web pages can also use HTML Forms to retrieve user input: • Co-ordinates • Origin/Destination Postcode • URLs built using web map sites’ APIs e. g. • www. streetmap. co. uk/newmap. srf? x=3210 00&y=621000&z&ar=n • Hit Go! and returned are one, or more, geo web pages: • Maps • Routing (e. g. Multimap, Google)

Map Mashups • Java. Script-powered web pages can also use HTML Forms to retrieve user input: • Co-ordinates • Origin/Destination Postcode • URLs built using web map sites’ APIs e. g. • www. streetmap. co. uk/newmap. srf? x=3210 00&y=621000&z&ar=n • Hit Go! and returned are one, or more, geo web pages: • Maps • Routing (e. g. Multimap, Google)

Open Alternatives • Javascript Map Libraries • • Open. Layers Leaflet Map. Box JS (built-on Leaflet) Arc. GIS API for JS (…Arc. GIS Online) • Map. Tiler – create hierarchical sets of map images for rapid display/zoom • Map. Server • Geo. Server • Map. Box • • Hosting – cf. Arc. GIS Online Mapnik – Develop desktop/web apps Tile. Mill – apps (built-on Mapnik) Arc 2 Earth – Arc. GIS published to Map. Box • NB Arc 2 Earth puts Google Maps in Arc. GIS! • Quantum GIS Server

Cloud GIS; GIS Apps • Arc. GIS Online (Cloud GIS) • • Login Hosting ESRI Desktop/Mobile Apps Arc. GIS Marketplace (incl. RS images 'through the cloud'!) • Other Apps/Marketplaces • Auto. CAD • Maths/Diagramming Tools • Arc. GIS Apps Including: • ESRI Business Apps • Where does one begin? • NB Arc. GIS for Developers acc’t – free!! • Map and 3 D Scene Viewers (Web) • Collector (Android Jan, i. OS Mar) • Explorer (Android, i. OS, Mac!!) • Maps for Office • Maps for Share. Point

Visualisation GIS Limitations / Novel (or newly implemented) Techniques

Cartographic Principles • Still apply… • • • Colour Theory Map Purpose Necessary (or unnecessary map marginalia – N arrow, scale bar or text? Etc. ) Colours or shading (changing hue or increasing intensity) Clarity/avoid Map Clutter (not overloaded, cf. Power. Point slides - ahem!) • But also IT concepts to consider… • Symbol type/style/size – e. g. halo effects to make stand out/legible • Label placement/collision detection rules • Image Resolution – end use? (web or print? )

Beyond 2 D: 2. 5 D, Not quite 3 D • Historically used in computer games • Appearance of 3 D world (e. g. buildings, extruded shapes) but movement constrained to XY plane (2 D) • Use of camera angle/projection and/or e. g. scaling of height to give depth perception also parallax scrolling and texture mapping and other effects Image: Lin. City / Wikimedia Commons / GPL

Beyond 2 D: 2. 5 D, Not quite 3 D • In GIS to date a 2. 5 D surface occupies 3 D space but still an XY plane at any point. • Each XY location in a layer will have one unique zvalue (height) • Heights obtained from a related DEM raster or from values stored in either data geometry (zvalues) or attribute table field • Extrusion and offset of layers possible • Discontinuities not modelled (functional surfaces only) – so no use for vertical geological faults (i. e. dip = 0 degrees) • May be possible in some open source GIS – e. g. GRASS r. plane? ? Contains OS data © Crown copyright and database right 2015

Beyond 2 D: 2. 5 D, Not quite 3 D • We can however perform 3 D analysis: • • Viewshed/visibility analysis Visual Impact/Skylines Building shadow, amount of sunlight Amount of internal space overlooked • ESRI also offer multipatches (outline of 3 D shapes and texture mapping) for some 3 D work – close, but? … Contains OS data © Crown copyright and database right 2015 • Watch this space…

Globe Viewers – 3 D? • ESRI Arc. Globe • Reprojects all layers to one CRS • Arc. Reader (Free basic ‘reader’) • Explorer for Arc. GIS (…Online, basic!) • Arc. GIS Explorer Desktop (Free!) • Lets you switch quickly 2 D<->3 D! Images: ESRI

Globe Viewers – 3 D? • Google Earth • Subsurface? • Surface discontinuities? Image via Google Earth plugin demos • Thematicmapping. org – B. Sandvik • KML produced, symbolising datasets • 1 D (bar length), 2 D, or 3 D symbols

Global Thematic Mapping • Can use GE or GE plugin • GE plugin may face extinction (beware ‘Google pulling the rug’) • Which might mean the end of many sites built-on GE https: //youtu. be/t 0 L 9 co. Owk. E 8 • • Flight trackers – “Follow in GE” Ship trackers – “View in GE” E. g. Google Earth Flight Simulator! http: //www. gefs-online. com/

Map projections / Co-ord Ref Systems (CRS) • Geographic Systems – curved Earth, angular units vs. Projected Systems – flat Earth, linear units • Important to understand basics – though not always critical to define/set CRS for all layers if working in one coord system and correct units/extent values seen – be able to spot if wrong! • However, if you have data based on multiple Earth surface models then you must ensure you specify the correct datum. GIS offers on-fly transformation between datums but layers in angular units will need projected to new data layer to match up exactly. • Get the datum right – what the data are in (e. g. GPS in WGS 84 geo), then project as req’d e. g. BNG • No unique lat, long for every point on Earth – depends on datum/geographic CRS used • Also becomes more critical when working with e. g. Globe Viewers where all layers are projected to e. g. Cubic projection! With Arc. Globe all layers must have a CRS defined. • Also bear in mind peculiarities of different projections. • Distortion – of familiar shape? area/proportion of map occupied? angular relationships? • Web Mercator / Web Maps incl. Google – Distortion far from equator, Scale change with latitude!

Interactive Cartography – ESDA • ESDA – Exploratory Spatial Data Analysis • Geo. Da/CAST/Py. SAL • Brushing & Linking • Explore Feature/Data/Stats Space & • Explore Real Space (simultaneously) • • 2 D Cartogram 3 D Box Plot Parallel Coordinate Plot Others…

Interactive Cartography – ESDA • ESDA – Exploratory Spatial Data Analysis • Geo. Da/CAST/Py. SAL • Brushing & Linking • Explore Feature/Data/Stats Space & • Explore Real Space (simultaneously) • • 2 D Cartogram 3 D Box Plot Parallel Coordinate Plot Others…

Interactive Cartography – ESDA • ESDA – Exploratory Spatial Data Analysis • Geo. Da/CAST/Py. SAL • Brushing & Linking • Explore Feature/Data/Stats Space & • Explore Real Space (simultaneously) • • 2 D Cartogram 3 D Box Plot Parallel Coordinate Plot Others…

Interactive Cartography – ESDA • ESDA – Exploratory Spatial Data Analysis • Geo. Da/CAST/Py. SAL • Brushing & Linking • Explore Feature/Data/Stats Space & • Explore Real Space (simultaneously) • • 2 D Cartogram 3 D Box Plot Parallel Coordinate Plot Others…

Interactive Cartography – VA • (Geo. Spatial) Visual Analytics • Geo. Vis -> Geo. VA? • May well include some existing types of analysis… • ESRI adopted 'analytics' in place of 'analysis‘! • Big in London in partic – CASA, gi. Centre at City • Plus Map. Tube, http: //spatial. ly/, London data/vis e. g. TFL, Crime • Use of linked views cf. Geo. Da/ESDA • 'Processing' (noun) prog. language for visuals, often used

Visualising Movement/Flows • Big Data(sets) • Twitter Feeds • APIs/Screenscraping • Smart Cities • Crowd-sourcing/VGI https: //www. youtube. com/watch? v=r. JC 7 B-9 Zfh. E

A Few 100(0) Other Things to Explore… • • • Tobler’s 1 st Law Topology Network / Metro maps… Database Theory / Best Practice Modern Databases (No. SQL!) Spatial Data Infrastructures (SDI) Spatial Regression/Error Modelling Metadata (Standards) Quirks of ESRI File Formats! Arc. GIS Quirks… Evolution of GIS – why it is the way it is… • • Human Computer Interaction User Needs Testing; Accessibilty Issues User Perception/Psychology Deeper Information Theory • User/Software Behaviour • Display of Info (Bertin) • Visualisation Luminaries • Tufte, Mac. Eachren, modern-days • Automated Carto / Map Generalisation • Heat Mapping • Space Time Cubes / Temporal GIS

…But never* be afraid to ask! Question Time? *See: Advice to the Aspiring Interactive Cartographer. . . http: //www. axismaps. com/blog/2012/09/advice-to-the-aspiring-interactive-cartographer/