Dept of Computer Science Rutgers Node and Graph

Dept. of Computer Science Rutgers Node and Graph Similarity: Theory and Applications Danai Koutra (CMU) Tina Eliassi-Rad (Rutgers) Christos Faloutsos (CMU) ICDM 2014, Monday December 15 th 2014, Shenzhen, China Copyright for the tutorial materials is held by the authors. The authors grant IEEE ICDM permission to distribute the materials through its website.

Part 2 a Graph Similarity: known node correspondence ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 2

Summary • Two main approaches: – Avoiding the correspondence problem • Faster • Might give good estimates – By finding the node correspondence • Potentially more accurate • Various methods for unipartite graphs • One approach explicitly for bipartite graphs [Koutra+, ICDM’ 13] + extension to unipartite graphs ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 3

Roadmap • Known node correspondence • Unknown node correspondence – Unipartite graphs – Bipartite graphs – Summary ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 4

Graph Similarity with unknown node correspondence • INPUT: 2 anonymized networks – GIVEN: – NOT GIVEN: node IDs side-info class labels 1 2 • OUTPUT: structural similarity score or node mapping ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 5

Can we identify users across social networks? Same or “similar” users? 8/29/12 ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 6

More Data Mining Applications information network Smarter Commerce internal external networks x-fer learning social network better INPUT for graph similarity + graph kernel better anomaly detection healthcare ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos team formation 7

More Applications? chemical compound comparison protein-protein alignment link prediction & viral marketing Optical character recognition IR: synonym extraction Structure matching in DB ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos wiki translation 8

Roadmap • Known node correspondence • Unknown node correspondence – Unipartite graphs • Avoiding the correspondence problem • Solving the correspondence problem – Bipartite graphs – Summary ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 9

ecce n Main Idea d c c , eg 1. Extract graph features tric ity e-va e-ve ls ctor s 2. sim(GA, GB) = “similarity” between the features or apply some classifier ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 10

Spectral methods: λ-distance d(GA, GB) = λi = eigenvalues of: • Adjacency A • Laplacian L = D – A Σ(λAi - λBi)2 O(n 3), SVD 0 1 0 1 0 d 1 d 2 - 1 0 1 d 3 0 1 0 • Normalized Laplacian Lnorm = D-1/2 A D-1/2 d 1 -. 5 0 1 0 d 1 -. 5 d 2 -. 5 1 0 1 d 2 -. 5 0 1 0 d 3 -. 5 [Bunke ’ 06, Wilson ’ 08, El. Ghawalby ’ 08, Haemers+ ’ 04, Brouwer ’ 09] ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 11

λ-distance: Disadvantages 1) co-spectral graphs with different structure 2) subtle changes in the graphs => big differences in spectra 3) O(n 3) runtime SVD [Haemers+ ’ 04, Brouwer ’ 09] ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 12

Comparison of methods revisited edge weight returns focus Metric P 1 P 2 P 3 P 4 Vertex/Edge Overlap Graph Edit Distance (XOR) Signature Similarity λ-distance (adjacency matrix) λ-distance (graph laplacian) λ-distance (normalized lapl. ) DELTACON 0 DELTACON ✗ ✗ ✗ ? ✗ ✔ ✗ ? ✔ ✔ ✔ ✔ [Koutra, Faloutsos, Vogelstein. SDM’ 12] ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 13

SN: Socially Relevant Features GA sim(GA, GB) = 1 -d(GA, GB) d: earth mover’s distance between distributions Leadership dominated by one node? ICDM’ 14 Tutorial Bonding clust. coefficient GB Diversity # disjoint communities [Macindoe ’ 10] D. Koutra & T. Eliassi-Rad & C. Faloutsos 14

Net. Simile GA egonet FG = n o d e s 1 2. . . n features 1 2 3. . . 7 . . . ① # of neighbors ② clustering coefficient ③ edges in egonet ④ # of neighbors of egonet … ‘signature’ vector P= median mean Canberra distance GA s. d. skewness P kurtosis P P [Berlingerio, Koutra, Eliassi-Rad, Faloutsos ‘ 13] ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 15

Application: Discontinuity Detection in Yahoo! IM 1. Microsoft offers to buy Yahoo!. 2. New features for flickr were announced. nodes: IM users edges: communication events [Berlingerio, Koutra, Eliassi-Rad, Faloutsos ‘ 13] ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 16

Roadmap • Known node correspondence • Unknown node correspondence – Unipartite graphs • Avoiding the correspondence problem • Solving the correspondence problem – Bipartite graphs – Summary ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 17

Eigen-Decomposition Approach GA GB A B Goal: min || PAPT - B||F 2 P permutation matrix [Umeyama ‘ 88 ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 18

Eigen-Decomposition Approach GA GB Near-optimal for noiseless graphs O(n 3) runtime eigendecomp. A B A = UAOnly ΛAUATfor graphs of same B =size U BΛ BU BT P = Hungarian method on ICDM’ 14 Tutorial |UA| |UBT| D. Koutra & T. Eliassi-Rad & C. Faloutsos [Umeyama ‘ 88 19

NMF-based Approach GA GB A B Goal: min || PAPT - B|| F 2 PPT= PTP = I P>=0 non-0 elements per row/column [Ding+ ‘ 08 ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 20

NMF-based Algorithm • Step 1: P 0 = |UA| |UBT| [Umeyama] [Ding+ ‘ 08 ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 21

NMF-based Algorithm • Step 1: P 0 = |UA| |UBT| [Umeyama] • Step 2: Non-Negative Matrix Factorization to find P∞ O(n 3) runtime + Guaranteed convergence [Ding+ ‘ 08 ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 22

Net. Align Wkk’ A L B Possible matchings! [Bayati+ ’ 11] ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 23

Net. Align Wkk’ A L B NP-hard Goal: find matching M C L s. t. it maximizes a linear combination of the weights w & the number of overlapped edges. [Bayati+ ’ 11] ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 24

Two Solutions Wkk’ A Approach ICDM’ 14 Tutorial B Sparse L Dense L Speed ✔ Belief Propagation [Bayati+’ 11] Lagrangian [Klau+’ 09] L 30% faster ✔ D. Koutra & T. Eliassi-Rad & C. Faloutsos 25

More Approaches • • Cavity approach Graduated Assignment Pattern Recognition Biological Networks • • Linear Programming EM Similarity Flooding Path-following Graph Kernels Spectral methods …. ICDM’ 14 Tutorial [Bradde+ ‘ 10] [Gold ’ 96] [Conte+ ‘ 04] [Berg, Lassig ‘ 04, Iso. Rank [Singh ’ 07] [Almohamad & Duffuaa ’ 93] [Luo ‘ 02] [Melnik+ ’ 02] [Zaslavskiy ’ 09] [Smalter+ ‘ 08] [Qiu, Hancock ‘ 06] D. Koutra & T. Eliassi-Rad & C. Faloutsos 26

Roadmap • Known node correspondence • Unknown node correspondence – Unipartite graphs • Avoiding the correspondence problem • Solving the correspondence problem – Bipartite graphs – Summary ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 27

Bipartite Graph Alignment INPUT: A, B groups u 1 1 0 0 s 1 1 0 0 e 0 0 1 0 r 1010 s 1 1 0 1 A B groups u 1 1 0 0 0 s 01010 e 00111 r 00010 s 00010 [Koutra, Tong, Lubensky ‘ 13 ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 28

Bipartite Graph Alignment INPUT: A, B groups u 1 1 0 0 s 1 1 0 0 e 0 0 1 0 r 1010 s 1 1 0 1 A B groups u 1 1 0 0 0 s 0 1 0 e 0 0 1 1 1 r 0 0 0 1 0 s 0 0 0 1 0 P (users) OUTPUT: P and … (permutation matrices) 0 1 0 0 0 0 0 1 ICDM’ 14 Tutorial A B [Koutra, Tong, Lubensky ‘ 13 D. Koutra & T. Eliassi-Rad & C. Faloutsos 29

Bipartite Graph Alignment INPUT: A, B groups A B Graph 100 u 1 isomorphism s 1 1 0 0 Hard (Pe 0 or 0 1 NP-complete? ) 0 r 1010 s 1 1 0 1 OUTPUT: P and Q groups u 1 1 0 0 0 s 0 1 0 e 0 0 1 1 1 r 0 0 0 1 0 s 0 0 0 1 0 Subgraph isomorphism Q (groups) P (users) NP-complete (permutation matrices) 0 1 0 0 0 0 0 1 ICDM’ 14 Tutorial Q: Now what? A: Constraints/Relaxations B A s. t. A min || PAQ - B||F 2 B permutation of users/groups in A D. Koutra & T. Eliassi-Rad & C. Faloutsos 30

Main Idea g INPUT: A, B u 1100 … … 1101 g AB u P (users) OUTPUT: P, Q correspondence matrices B A 11000 … … … 01010 Q (groups) A B s. t. min || PAQ - B||F 2 0 1 0 0 0 0 0 1 ICDM’ 14 Tutorial 0. 9. 7. 3 0. 3. 2 0. 6. 5 0. 1. 4. 8 [Koutra, Tong, Lubensky ‘ 13 D. Koutra & T. Eliassi-Rad & C. Faloutsos 31

Main Idea g INPUT: A, B u AB u P (users) OUTPUT: P, Q correspondence matrices 1100 … … 1101 g A B 11000 … … … 01010 Q (groups) A B s. t. min || PAQ - B||F 2 CONSTRAINTS: (a) Pij, Qij = probabilities (not 1 -1 mapping) (b) sparse matrices P and Q (more efficient for large scale graphs) [Koutra, Tong, Lubensky ‘ 13 ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 32

Main Idea g INPUT: A, B u 1100 … … 1101 g AB u P (users) OUTPUT: P, Q correspondence matrices B A 11000 … … … 01010 Q (groups) A B min || PAQ - B||F 2 +λ|| P||1 + μ|| Q||1 P, Q P. ICDM’ 14 Tutorial A Q B sparsity [Koutra, Tong, Lubensky ‘ 13 D. Koutra & T. Eliassi-Rad & C. Faloutsos 33

Big-Align: Accuracy vs. Runtime Bi. G-Align skip Bi. G-Align exact NMF-based Umeyama Net. Align Big-Align improves both speed and accuracy. [Koutra, Tong, Lubensky ‘ 13 ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 34

Extension: Uni-Align DETAILS n nodes min || PAQ - B||F 2 P fixed d features • node degree • clustering coeff • … … [Koutra, Tong, Lubensky ‘ 13 ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 35

Extension: Uni-Align DETAILS n nodes min || PAQ - B||F 2 P d features SVD A = USVT P = g*(A, B, S, U)= = closed-form solution [Koutra, Tong, Lubensky ‘ 13] ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos O(n. d 2) 36

Uni-Align: Accuracy vs. Runtime Uni-Align NMF-based Net. Align subgraphs (64 K x 64 K users) Umeyama Uni-Align is faster and more accurate than other approaches. [Koutra, Tong, Lubensky ‘ 13 ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 37

Beyond Bi. G-Align: Multi-way Linkage ~ S 1: Dynamic Graph Linkage S 2: Community-level Linkage –~ –~ S 3: Hetero. Graph Linkage S 4: Multi-relational DB Linkage All build upon Bi. G-Align ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 38

Roadmap • Known node correspondence • Unknown node correspondence – Unipartite graphs – Bipartite graphs – Summary ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 39

Summary • Two main approaches: – Avoiding the correspondence problem • Faster • Might give good estimates – By finding the node correspondence • Potentially more accurate • Various methods for unipartite graphs • One approach explicitly for bipartite graphs [Koutra+, ICDM’ 13] + extension to unipartite graphs ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 40

References • M. Berlingerio, D. Koutra, T. Eliassi-Rad, C. Faloutsos. Network similarity via multiple social theories. IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM), 2013. • Geng Li, Murat Semerci, Bülent Yener, and Mohammed J. Zaki. 2012. Effective graph classification based on topological and label attributes. Stat. Anal. Data Min. 5, 4 (August 2012), 265 -283. • J. T. Vogelstein and C. E. Priebe. Shuffled graph classification: Theory and connectome applications. 2011 • Macindoe, O. and Richards, W. 2010. Graph comparison using fine structure analysis. In. Social. Com/PASSAT. 193– 200. • A. E. Brouwer and E Spence, Cospectral graphs on 12 vertices. Elec. Journ. Combin. , Vol. 16 (20) (2009). • R. C. Wilson and P. Zhu, A study of graph spectra for comparing graphs and trees. Journal of Pattern Recognition, vol. 41, no. 9, pp. 2833– 2841, 2008. ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 41

References • Elghawalby, H. , & Hancock, E. R. (2008). Measuring Graph Similarity Using Spectral Geometry, 517 -526. • H. Bunke, P. J. Dickinson, M. Kraetzl, and W. D. Wallis, A Graph-Theoretic Approach to Enterprise Network Dynamics (PCS). Birkhauser, 2006. • W. H. Haemers, E. Spence, Enumeration of cospectral graphs. European J. Combin, 25 (2004), 199 -211. • A. Kelmans. Comparisons of graphs by their number of spanning trees. Discrete Math. , 16 (1976), pp. 241– 261 • M. Fiedler, Algebraic connectivity of graphs. Czechoslovak. Mathematical Journal, vol. 23, no. 98, pp. 298– 305, 1973. ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 42

References Graph Alignment • D. Koutra, H. Tong, D. Lubensky. BIG-ALIGN: Fast Bipartite Graph Alignment. In IEEE 13 th International Conference on Data Mining (ICDM), pp. 389 -398, 2013. • J. T. Vogelstein, J. M. Conroy, L. J. Podrazik, S. G. Kratzer, D. E. Fishkind, R. J. Vogelstein, and C. E. Priebe. Fast inexact graph matching with applications in statistical connectomics. Co. RR, abs/1112. 5507, 2011. • S. Bradde, A. Braunstein, H. Mahmoudi, F. Tria, M. Weigt, and R. Zecchina. Aligning graphs and finding substructures by a cavity approach. Europhysics Letters, 89, 2010. • M. Zaslavskiy, F. Bach, and J. -P. Vert. A path following algorithm for the graph matching problem. IEEE TPAMI, 31(12): 2227– 2242, Dec. 2009. • G. W. Klau. A new graph-based method for pairwise global network alignment. BMC, 10(S-1), 2009. • A. Narayanan and V. Shmatikov. De-anonymizing social networks. In SSP, pages 173 – 187, may 2009. ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 43

References • M. Bayati, M. Gerritsen, D. Gleich, A. Saberi, and Y. Wang. Algorithms for large, sparse network alignment problems. In ICDM, pages 705– 710, 2009. • L. Zager and G. Verghese. Graph similarity scoring and matching. Applied Mathematics Letters, 21(1): 86– 94, 2008. • C. H. Q. Ding, T. Li, and M. I. Jordan. Nonnegative matrix factorization for combinatorial optimization: Spectral clustering, graph matching, and clique finding. In ICDM, pages 183– 192, 2008. • A. Smalter, J. Huan, and G. Lushington. GPM: A Graph Pattern Matching Kernel with Diffusion for Chemical Compound Classification. In ICBBE, 2008. • Rohit Singh, Jinbo Xu, and Bonnie Berger. 2007. Pairwise global alignment of protein interaction networks by matching neighborhood topology. In RECOMB ’ 07. • H. Qiu and E. R. Hancock. Graph matching and clustering using spectral partitions. IEEE TPAMI, 39(1): 22– 34, 2006. • D. Conte, P. Foggia, C. Sansone, and M. Vento. Thirty years of graph matching in pattern recognition. IJPRAI, 18(3): 265– 298, 2004. ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 44

References • J. Berg and M. Lassig. Local graph alignment and motif search in biological networks. PNAS, 101(41): 14689– 14694, Oct. 2004. • S. Melnik, H. Garcia-Molina, and E. Rahm. Similarity flooding: A versatile graph matching algorithm and its application to schema matching. In ICDE, 2002. • B. Luo and E. R. Hancock. Iterative procrustes alignment with the EM algorithm. Image Vision Comput. , 20(5 -6): 377– 396, 2002. • S. Gold and A. Rangarajan. A graduated assignment algorithm for graph matching. IEEE TPAMI, 18(4): 377– 388, 1996. • S. Umeyama. An eigendecomposition approach to weighted graph matching problems. IEEE TPAMI, 10(5): 695– 703, 1988. ICDM’ 14 Tutorial D. Koutra & T. Eliassi-Rad & C. Faloutsos 45