A Fast Kernel for Attributed Graphs Yu Su

A Fast Kernel for Attributed Graphs Yu Su University of California at Santa Barbara with Fangqiu Han, Richard E. Harang, and Xifeng Yan

INTRODUCTION A Fast Kernel for Attributed Graphs

Graph Kernel o A graph kernel defines a similarity measure over graphs — a core problem in graph mining o Inner product in some (latent) feature space o Decouple data representation from learning machine n Once a graph kernel is supplied, a whole toolbox of kernel machines become readily applicable n SVM, Kernel PCA, Support Vector Regression, Clustering, etc. n A good graph kernel is thus the key A Fast Kernel for Attributed Graphs

Broad Applications Chemo- & Bioinformatics Natural Language Processing Software Engineering Semantic web A Fast Kernel for Attributed Graphs

Trends and Challenges in the Big Data Era Increasing graph size More efficient methods More versatile methods Richer graph attributes This work: A linear-time kernel that can handle both categorical and numerical attributes. A Fast Kernel for Attributed Graphs

Graph Kernel as a Measure of Graph Similarity ① Decompose each graph into a (multi-)set of features n n Subgraphs (Gartner et al. 2003, NP-hard!) Random walks (Gartner et al. 2003, Kashima et al. 2003) Subtrees (Shervashidze and Borgwardt 2009) Vectors (Neumann et al. 2016) A Fast Kernel for Attributed Graphs

Graph Kernel as a Measure of Graph Similarity ① Decompose each graph into a (multi-)set of features n n Subgraphs (Gartner et al. 2003, NP-hard!) Random walks (Gartner et al. 2003, Kashima et al. 2003) Subtrees (Shervashidze and Borgwardt 2009) Vectors (Neumann et al. 2016) ② Compare feature sets n Pair-wise comparison (quadratic) A Fast Kernel for Attributed Graphs

Graph Kernel as a Measure of Graph Similarity ① Decompose each graph into a (multi-)set of features n n Subgraphs (Gartner et al. 2003, NP-hard!) Random walks (Gartner et al. 2003, Kashima et al. 2003) Subtrees (Shervashidze and Borgwardt 2009) Vectors (Neumann et al. 2016) ② Compare feature sets n Pair-wise comparison (quadratic) n Inner product (linear; only when features are discrete) A Fast Kernel for Attributed Graphs

Graph Kernel as a Measure of Graph Similarity ① Decompose each graph into a (multi-)set of features n n Subgraphs (Gartner et al. 2003, NP-hard!) Random walks (Gartner et al. 2003, Kashima et al. 2003) Subtrees (Shervashidze and Borgwardt 2009) Vectors (Neumann et al. 2016) ② Compare feature sets n Pair-wise comparison (quadratic) n Inner product (linear; only when features are discrete) n Discretization (linear; can handle numerical attributes) A Fast Kernel for Attributed Graphs

Graph Kernel as a Measure of Graph Similarity ① Decompose each graph into a (multi-)set of features n n Subgraphs (Gartner et al. 2003, NP-hard!) Random walks (Gartner et al. 2003, Kashima et al. 2003) Subtrees (Shervashidze and Borgwardt 2009) Vectors (Neumann et al. 2016) ② Compare feature sets n Pair-wise comparison (quadratic) n Inner product (linear; only when features are discrete) n Discretization (linear; can handle numerical attributes) vector features + discretization A Fast Kernel for Attributed Graphs

METHOD A Fast Kernel for Attributed Graphs

Descriptor Matching (DM) Kernel: An Overview A Fast Kernel for Attributed Graphs

Desired Descriptor Property: Preserve Similarity o Similar nodes should have similar descriptors n So it becomes meaningful to compare graph similarity by matching their descriptors o Nodes are more similar if their attributes and neighbors are more similar n Recursive definition of similarity makes it natural to generate descriptors in a recursive manner A Fast Kernel for Attributed Graphs

Desired Descriptor Property: Highly Discriminative A Fast Kernel for Attributed Graphs

Descriptor Generation via Propagation A Fast Kernel for Attributed Graphs

Descriptor Matching o Optimal matching: Maximum weighted bipartite matching n Cubic time. Not a valid kernel (Vert 2008) A Fast Kernel for Attributed Graphs

Descriptor Matching o Optimal matching: Maximum weighted bipartite matching n Cubic time. Not a valid kernel (Vert 2008) o Discretization: Uniform binning n Linear time. Valid kernel. Unweighted, independent bins. A Fast Kernel for Attributed Graphs

Descriptor Matching o Optimal matching: Maximum weighted bipartite matching n Cubic time. Not a valid kernel (Vert 2008) o Discretization: Uniform binning n Linear time. Valid kernel. Unweighted, independent bins. o Discretization: Data-dependent hierarchical binning n Linear time. Valid kernel. Weighted, multi-resolution bins. n Vocabulary-Guided pyramid matching (VG) kernel (Grauman and Darrell 2006) A Fast Kernel for Attributed Graphs

Descriptor Matching via Pyramid Matching Kernel A Fast Kernel for Attributed Graphs

EVALUATION A Fast Kernel for Attributed Graphs

Efficiency on Synthetic Graphs DM: this work PK: ML’ 16 GH: NIPS’ 13 WLSP: JMLR’ 11 SP: ICDM’ 05 CSM: ICML’ 12 Number of nodes A Fast Kernel for Attributed Graphs

Accuracy on Real-world Graphs o DM is among the best in 9 out of the 10 datasets, and is significantly better than PK on 8 dataset (Student’s t test at p=0. 05). A Fast Kernel for Attributed Graphs

Summaries o A graph kernel n Can be computed in linear time w. r. t. graph size n Can handle both categorical and numerical attributes o Key ideas n Descriptor generation via categorical attribute propagation n Descriptor matching via hierarchical data-dependent discretization o Competitive performance n Efficient: scale to graphs with 100, 000 nodes n Accurate: best on 9 out of 10 datasets A Fast Kernel for Attributed Graphs

Thank You! A Fast Kernel for Attributed Graphs