GraphBased Concept Learning Jesus A Gonzalez Lawrence B

Graph-Based Concept Learning Jesus A. Gonzalez, Lawrence B. Holder, and Diane J. Cook Department of Computer Science and Engineering University of Texas at Arlington Box 19015, Arlington, TX 76019 -0015 {gonzalez, holder, cook}@cse. uta. edu http: //cygnus. uta. edu/subdue/

MOTIVATION AND GOAL l l Need for non-logic-based relational concept learner Empirical and theoretical comparisons of relational learners Logic-based relational learners (ILP) l FOIL [Quinlan et al. ] l Progol [Muggleton et al. ] Graph-based relational learner l SUBDUE

SUBDUE KNOWLEDGE DISCOVERY SYSTEM l SUBDUE discovers patterns (substructures) in structural data sets l SUBDUE represents data as a labeled graph. l. Vertices represent objects or attributes l. Edges represent relationships between objects l. Input: Labeled graph l. Output: Discovered patterns and instances

SUBDUE EXAMPLE Input Output shape triangle object on shape object 4 instances of square

SUBDUE’S SEARCH l Starts with a single vertex and repeatedly expands by one edge l Computationally-constrained beam search l Polynomially-constrained inexact graph matching l Search space is all sub-graphs of input graph l Guided by compression heuristic l Minimum description length

EVALUATION CRITERION MINIMUM DESCRIPTION LENGTH l Minimum Description Length (MDL) principle l The best theory to describe a set of data is the one that minimizes the DL of the entire data set. l DL of the graph: the number of bits necessary to completely describe the graph. l Search for the substructure that results in the maximum compression.

CONCEPT LEARNING SUBDUE l Modify Subdue for concept learning (Subdue. CL) l Accept positive and negative graphs as input examples l Find substructure describing positive examples, but not negative examples l Learn multiple rules (DNF)

CONCEPT LEARNING SUBDUE l Evaluation criteria based on number of positive examples covered without covering negative examples l Substructure value = 1 - Error

CONCEPT LEARNING SUBDUE EXAMPLE l Examples in graph format (chess domain): WK = White King WR = White Rook BK = Black King lt = less than adj = adjacent pos = position eq = equal a) Board Configuration b) Graph Representation

PRELIMINARY RESULTS • Comparison with FOIL and Progol • Significance test p for the Vote domain SUBDUE: error = 0. 051163 +/- 0. 044935 FOIL: error = 0. 069767 +/- 0. 054814 PROGOL: error = 0. 230233 +/- 0. 066187 SUBDUE – FOIL = -0. 018605 +/- 0. 052347 (p=0. 145068) SUBDUE - PROGOL = -0. 179070 +/- 0. 074394 (p=0. 000016) FOIL - PROGOL = -0. 160465 +/- 0. 067979 (p=0. 000019) ANOVA: 0. 000000 • Significance test p for the Chess domain SUBDUE: error = 0. 004600 +/- 0. 006186 FOIL: error = 0. 006600 +/- 0. 007183 PROGOL: error = 0. 002600 +/- 0. 002675 SUBDUE - FOIL = -0. 002000 +/- 0. 007542 (p=0. 211723) SUBDUE - PROGOL = 0. 002000 +/- 0. 004989 (p=0. 118354) FOIL - PROGOL = 0. 004000 + /- 0. 007242 (p=0. 057322) ANOVA: 0. 306232

RELATED THEORY Galois lattice [reference? ] l Subdue’s search space is similar to the Galois lattice l Polynomial convergence results for the Galois lattice apply to Subdue l PAC analysis of conceptual graphs [reference? ] l Subdue’s representation is a superset of conceptual graphs l PAC sample complexity results for conceptual graphs apply to Subdue l

CONCLUSIONS Empirical results indicate Subdue is competitive with ILP systems l More empirical comparisons are necessary l Theoretical results on Galois lattice and conceptual graphs apply to Subdue l Need to identify specific components of theory directly applicable to Subdue l Expand theories where needed l