COSC 6335 A Brief Introduction to Spatial Data

COSC 6335 A Brief Introduction to Spatial Data Mining inspired by Shashi Shekhar (UMN) Shashi Shekhar Spatial data mining is the process of discovering interesting, useful, non-trivial patterns from large spatial datasets Reading Material: http: //en. wikipedia. org/wiki/Spatial_analysis Spatial Statistics Software: http: //www. spatial-statistics. com/

Examples of Spatial Patterns Historic Examples (section 7. 1. 5, pp. 186) 1855 Asiatic Cholera in London: A water pump identified as the source Fluoride and healthy gums near Colorado river Theory of Gondwanaland - continents fit like pieces of a jigsaw puzlle Modern Examples Cancer clusters to investigate environment health hazards Crime hotspots for planning police patrol routes Bald eagles nest on tall trees near open water Nile virus spreading from north east USA to south and west Unusual warming of Pacific ocean (El Nino) affects weather in USA http: //en. wikipedia. org/wiki/Spatial_analysis Ch. Eick: Spatial Data Mining (inspired by a talk given at UH by Shashi Shekhar (UMN))

London Cholera in 1854 Ch. Eick: Spatial Data Mining (inspired by a talk given at UH by Shashi Shekhar (UMN))

Why Learn about Spatial Data Mining? Two basic reasons for new work Consideration of use in certain application domains Provide fundamental new understanding Application domains Scale up secondary spatial (statistical) analysis to very large datasets • Describe/explain locations of human settlements in last 5000 years • Find cancer clusters (CDC: https: //www. cdc. gov/nceh/clusters/default. htm ) to locate hazardous environments • Prepare land-use maps from satellite imagery (Satellite images help create land use maps in villages - Sub-Saharan Africa (scidev. net)) • Predict habitat suitable for endangered species (Feds Seek to Protect Lizard Habitat in the Permian Basin (sanangelolive. com)) Find new spatial patterns • Find groups of co-located geographic features Ch. Eick: Spatial Data Mining (inspired by a talk given at UH by Shashi Shekhar (UMN))

Why Learn about Spatial Data Mining? - 2 New understanding of geographic processes for Critical questions Ex. How is the health of planet Earth? Ex. Characterize effects of human activity on environment and ecology Ex. Predict effect of El Nino on weather, and economy Traditional approach: manually generate and test hypothesis But, spatial data is growing too fast to analyze manually • Satellite imagery, GPS tracks, sensors on highways, … Number of possible geographic hypothesis too large to explore manually • Large number of geographic features and locations • Number of interacting subsets of features grow exponentially • Ex. Find tele connections between weather events across ocean and land areas SDM may reduce the set of plausible hypothesis Identify hypothesis supported by the data For further exploration using traditional statistical methods Ch. Eick: Spatial Data Mining (inspired by a talk given at UH by Shashi Shekhar (UMN))

Autocorrelation Items in a traditional data are independent of each other, whereas properties of locations in a map are often “auto-correlated”. First law of geography [Tobler]: Everything is related to everything, but nearby things are more related than distant things. People with similar backgrounds tend to live in the same area Economies of nearby regions tend to be similar Changes in temperature occur gradually over space(and time) Waldo Tobler in 2000 Papers on “Laws in Geography”: http: //www. geog. ucsb. edu/~good/papers/393. pdf http: //www. cs. uh. edu/~ceick/DM/GOO 10. pdf Ch. Eick: Spatial Data Mining (inspired by a talk given at UH by Shashi Shekhar (UMN))

Characteristics of Spatial Data Mining Auto correlation Patterns usually have to be defined in the spatial attribute subspace and not in the complete attribute space Longitude and latitude (or other coordinate systems) are the glue that link different data collections together People are used to maps in GIS; therefore, data mining results have to be summarized on the top of maps Patterns not only refer to points, but can also refer to lines, or polygons or other higher order geometrical objects Patterns exist at different levels of granularity Large number of patterns, large dataset sizes Spatial patterns, e. g. spatial clusters can have arbitrary shapes Regional knowledge is of particular importance due to lack of global knowledge in geography ( spatial heterogeniety) Ch. Eick: Spatial Data Mining (inspired by a talk given at UH by Shashi Shekhar (UMN))

Why Regional Knowledge Important in Spatial Data Mining? A special challenge in spatial data mining is that information is usually not uniformly distributed in spatial datasets. It has been pointed out in the literature that “whole map statistics are seldom useful”, that “most relationships in spatial data sets are geographically regional, rather than global”, and that “there is no average place on the Earth’s surface” [Goodchild 03, Openshaw 99]. Therefore, it is not surprising that domain experts are mostly interested in discovering hidden patterns at a regional scale rather than a global scale. Michael Frank Goodchild Ch. Eick: Spatial Data Mining (inspired by a talk given at UH by Shashi Shekhar (UMN))

Spatial Autocorrelation: Distance-based measure K-function Definition (https: //www 3. nd. edu/~mhaenggi/ee 87021/Dixon-K-Function. pdf ) Test against randomness for point pattern [number of events within distance h of an arbitrary event] • λ is intensity of event Model departure from randomness in a wide range of scales Inference For Poisson complete spatial randomness (CSR): K(h) = πh 2 Plot Khat(h) against h, compare to Poisson CSR • >: cluster • <: decluster/regularity K-Function based Spatial Autocorrelation Ch. Eick: Spatial Data Mining (inspired by a talk given at UH by Shashi Shekhar (UMN))

Basic Approach Using K-Functions 10 Ch. Eick: Spatial Data Mining (inspired by a talk given at UH by Shashi Shekhar (UMN))

Example: Collocation Red and Green Objects FOR radii r 1, …, rn DO FOR all green objects g DO Compute #-of-red objects within radius rj of g ENDDO Compute average roj of values observed in previous loop Put entry (rj, (roj/total_number_of_red_objects)) into Curve ENDDO Ch. Eick: Spatial Data Mining (inspired by a talk given at UH by Shashi Shekhar (UMN))

Associations, Spatial associations, Co-location Answers: and find patterns from the following sample dataset? Ch. Eick: Spatial Data Mining (inspired by a talk given at UH by Shashi Shekhar (UMN))

Illustration of Cross-Correlation Illustration of Cross K-function for Example Data Cross-K Function for Example Data Ch. Eick: Spatial Data Mining (inspired by a talk given at UH by Shashi Shekhar (UMN))

Colocation Rules – Spatial Interest Measures http: //www. youtube. com/watch? v=RPy. Jw. Yqy. Bu. I Ch. Eick: Spatial Data Mining (inspired by a talk given at UH by Shashi Shekhar (UMN))

Spatial Association Rules • Spatial Association Rules • A special reference spatial feature • Transactions are defined around instance of special spatial feature • Item-types = spatial predicates • Example: Table 7. 5 (pp. 204) Ch. Eick: Spatial Data Mining (inspired by a talk given at UH by Shashi Shekhar (UMN))

Co-location rules vs. traditional association rules Association rules Co-location rules Underlying space discrete sets continuous space item-types events /Boolean spatial features collection Transaction (T) Neighborhood (N) prevalence measure support participation index conditional probability metric Pr. [ A in T | B in T ] Pr. [ A in N(L) | B at location L ] Participation index = min{pr(fi, c)} Where pr(fi, c) of feature fi in co-location c = {f 1, f 2, …, fk}: = fraction of instances of fi with feature {f 1, …, fi-1, fi+1, …, fk} nearby N(L) = neighborhood of location L Ch. Eick: Spatial Data Mining (inspired by a talk given at UH by Shashi Shekhar (UMN))

Spatial Regression Not covered in 2020; if you interested in this topic, see: http: //www 2. cs. uh. edu/~ceick/DM/Spatial Regression. pptx Ch. Eick: Spatial Data Mining (inspired by a talk given at UH by Shashi Shekhar (UMN))

Conclusions Spatial Data Mining Spatial patterns are opposite of random Common spatial patterns: location prediction, feature interaction, hot spots, geographically referenced statistical patterns, co-location, emergent patterns, … SDM = search for unexpected interesting patterns in large spatial databases Spatial patterns may be discovered using Techniques like classification, associations, clustering and outlier detection New techniques are needed for SDM due to • Spatial Auto-correlation • Importance of non-point data types (e. g. polygons) • Continuity of space • Regional knowledge; also establishes a need for scoping • Separation between spatial and non-spatial subspace—in traditional approaches clusters are usually defined over the complete attribute space Knowledge sources are available now Raw knowledge to perform spatial data mining is mostly available online now (e. g. relational databases, Google Earth) GIS tools are available that facilitate integrating knowledge from different source Ch. Eick: Spatial Data Mining (inspired by a talk given at UH by Shashi Shekhar (UMN))

Example Videos Discussing Spatial Analysis http: //www. esri. com/what-is-gis/index. html What is GIS? http: //www. youtube. com/watch? v=Zq. Mul 3 OIQNI&feature=related (Geographically weighted regression software advertisement video) http: //www. youtube. com/watch? v=Rh. Ddtqg. Iy 9 Q&feature=related (Spatial Analysis and Remote Sensing Degree at UA) http: //www. youtube. com/watch? v=_SBLBk. P 9 O 9 I&feature=related Arc. GIS Spatial Analyst Overview http: //www. youtube. com/watch? v=m. BSXBq. EP-7 Y&feature=related (Arc. GIS 9. 3: Advanced planning and analysis - Part 1) http: //www. youtube. com/watch? v=agzjyi 0 rn. Oo&feature=related (Example using Spatial Analysis to Analyze Medical Data; the video is not really that “great”; if you know a better one share it with us! https: //sigspatial 2020. sigspatial. org/ ACM SIGSPATIAL Conference, discusses advances in Geographical Information Systems and related areas Ch. Eick: Spatial Data Mining (inspired by a talk given at UH by Shashi Shekhar (UMN))

December 3, News The final creation scores for Tasks 6 and 7 will be computed by averaging the student peer review creation score with a creation score created by Dr. Eick (Task 6) and Nour (Task 7). Nour, Dr. Eick and Eli from Kritik are still working on procedures on how to weight and curve the various Problem. Set tasks. We should have things figured out by Dec. 9! If there is some interest there will be a Collocation Mining Group Project Post Analysis and Discussion Meeting on Tuesday, December 22, 3: 30 p; you will be polled about having this event or not on December 14! Dr. Eick and Nour will still have office hours through Tuesday, December 8, 2020. The course final exam scheduled for Th. , December 10, 2 p will take 115 minutes and will be similar in nature as the Midterm exam. It will count 29% towards the overall course grade. 2020 COSC 6335 Course Element Weighting: Problem. Sets: 48%, Exams: 48% and Attendance&Online. Credit: 4%. Final Exam Topic List has been posted on the course website. Ch. Eick: Spatial Data Mining (inspired by a talk given at UH by Shashi Shekhar (UMN))

The American Artist Jenny Holzer Specializes in Language Art Some of her artwork, namely her electronic displays, have been displayed at Times Square (New York City) and other major cities. She is well know for her work on truisms: https: //www. artsy. net/artwork/jenny-holzer-selections-from-truisms Videos: https: //www. bing. com/videos/search? q=jenny+holzer+kind+of+blue &&view=detail&mid=AF 31227 CB 4 F 992 E 46 B 2 BAF 31227 CB 4 F 992 E 46 B 2 B&&FORM=VDRVSR Jenny Holzer: Programming | Art 21 "Extended Play" – You. Tube A photo Dr. Eick has taken of her exhibit “Kind of Blue” in the Modern Art Museum of Fort Worth serves as the icon for the DM 2020 Teams page. Ch. Eick: Spatial Data Mining (inspired by a talk given at UH by Shashi Shekhar (UMN))