CMU SCS Data Mining in Streams and Graphs

CMU SCS Data Mining in Streams and Graphs Christos Faloutsos CMU CALD Day '06 C. Faloutsos

CMU SCS Outline • • Problem definition / Motivation Graphs and power laws Streams and forecasting Conclusions CALD Day '06 C. Faloutsos 2

CMU SCS Motivation • Data mining: ~ find patterns (rules, outliers) • How do real graphs look like? • How do (numerical) streams look like? CALD Day '06 C. Faloutsos 3

CMU SCS Joint work with • Dr. Deepayan Chakrabarti (CMU/Yahoo R. L. ) CALD Day '06 C. Faloutsos 4

CMU SCS Graphs - why should we care? Internet Map [lumeta. com] Food Web [Martinez ’ 91] Protein Interactions [genomebiology. com] Friendship Network [Moody ’ 01] CALD Day '06 C. Faloutsos 5

CMU SCS Graphs - why should we care? • network of supply-chain companies • network of companies & board-of-directors members • ‘viral’ marketing • web-log (‘blog’) news propagation • computer network security: email/IP traffic and anomaly detection • . . CALD Day '06 C. Faloutsos 6

CMU SCS Problem #1 - network and graph mining • How does the Internet look like? • How does the web look like? • What constitutes a ‘normal’ social network? • What is ‘normal’/‘abnormal’? • which patterns/laws hold? CALD Day '06 C. Faloutsos 7

CMU SCS Graph mining • Are real graphs random? CALD Day '06 C. Faloutsos 8

CMU SCS Laws and patterns NO!! • Diameter • in- and out- degree distributions • other (surprising) patterns CALD Day '06 C. Faloutsos 9

CMU SCS Solution • Power law in the degree distribution [SIGCOMM 99] internet domains log(degree) ibm. com att. com -0. 82 log(rank) CALD Day '06 C. Faloutsos 13

CMU SCS But: • Q 1: How about graphs from other domains? • Q 2: How about temporal evolution? CALD Day '06 C. Faloutsos 14

CMU SCS The Peer-to-Peer Topology [Jovanovic+] • Frequency versus degree • Number of adjacent peers follows a power-law CALD Day '06 C. Faloutsos 15

CMU SCS More power laws: citation counts: (citeseer. nj. nec. com 6/2001) log(count) Ullman log(#citations) CALD Day '06 C. Faloutsos 16

Swedish sex-web CMU SCS Nodes: people (Females; Males) Links: sexual relationships Albert Laszlo Barabasi http: //www. nd. edu/~networks/ Publication%20 Categories/ 04%20 Talks/2005 -norway 3 hours. ppt Liljeros et al. Nature 2001 4781 Swedes; 18 -74; 59% response rate. CALD Day '06 C. Faloutsos 17

Swedish sex-web CMU SCS Nodes: people (Females; Males) Links: sexual relationships Albert Laszlo Barabasi http: //www. nd. edu/~networks/ Publication%20 Categories/ 04%20 Talks/2005 -norway 3 hours. ppt 4781 Swedes; 18 -74; 59% response rate. CALD Day '06 C. Faloutsos 18 Liljeros et al. Nature 2001

CMU SCS More power laws: • web hit counts [w/ A. Montgomery] log(count) Web Site Traffic Zipf ``ebay’’ users sites log(in-degree) CALD Day '06 C. Faloutsos 19

CMU SCS epinions. com • who-trusts-whom [Richardson + Domingos, KDD 2001] count trusts-2000 -people user (out) degree CALD Day '06 C. Faloutsos 20

CMU SCS More Power laws • Also hold for other web graphs [Barabasi+], [Tomkins+], with additional ‘rules’ (bipartite cores follow power laws) CALD Day '06 C. Faloutsos 21

CMU SCS But: • Q 1: How about graphs from other domains? • Q 2: How about temporal evolution? CALD Day '06 C. Faloutsos 22

CMU SCS att. com log(degree) ibm. com Time Evolution: rank R 0. 82 log(rank Domain level The rank exponent has not changed! CALD Day '06 C. Faloutsos 23

CMU SCS Any other pattern, over time? CALD Day '06 C. Faloutsos 24

CMU SCS Time evolution • with Jure Leskovec (CMU) • and Jon Kleinberg (Cornell) CALD Day '06 C. Faloutsos 25

CMU SCS Evolution of the Diameter • Prior work on Power Law graphs hints at slowly growing diameter: – diameter ~ O(log N) • What is happening in real data? CALD Day '06 C. Faloutsos 26

CMU SCS Evolution of the Diameter • Prior work on Power Law graphs hints at slowly growing diameter: – diameter ~ O(log N) • What is happening in real data? • Diameter shrinks over time – As the network grows the distances between nodes slowly decrease CALD Day '06 C. Faloutsos 27

CMU SCS Diameter – Ar. Xiv citation graph • Citations among physics papers • 1992 – 2003 • One graph per year diameter time [years] CALD Day '06 C. Faloutsos 28

CMU SCS Diameter – “Autonomous Systems” • Graph of Internet • One graph per day • 1997 – 2000 diameter number of nodes CALD Day '06 C. Faloutsos 29

CMU SCS Diameter – “Affiliation Network” • Graph of collaborations in physics – authors linked to papers • 10 years of data diameter time [years] CALD Day '06 C. Faloutsos 30

CMU SCS Diameter – “Patents” • Patent citation network • 25 years of data diameter time [years] CALD Day '06 C. Faloutsos 31

CMU SCS Temporal Evolution of the Graphs • N(t) … nodes at time t • E(t) … edges at time t • Suppose that N(t+1) = 2 * N(t) • Q: what is your guess for E(t+1) =? 2 * E(t) CALD Day '06 C. Faloutsos 32

CMU SCS Temporal Evolution of the Graphs • N(t) … nodes at time t • E(t) … edges at time t • Suppose that N(t+1) = 2 * N(t) • Q: what is your guess for E(t+1) =? 2 * E(t) • A: over-doubled! – But obeying the ``Densification Power Law’’ CALD Day '06 C. Faloutsos 33

CMU SCS Densification – Physics Citations • Citations among physics papers E(t) • 2003: – 29, 555 papers, 352, 807 citations ? ? N(t) CALD Day '06 C. Faloutsos 34

CMU SCS Densification – Physics Citations • Citations among physics papers E(t) • 2003: – 29, 555 papers, 352, 807 citations 1. 69 N(t) CALD Day '06 C. Faloutsos 35

CMU SCS Densification – Physics Citations • Citations among physics papers E(t) • 2003: – 29, 555 papers, 352, 807 citations 1. 69 1: tree N(t) CALD Day '06 C. Faloutsos 36

CMU SCS Densification – Physics Citations • Citations among physics papers E(t) • 2003: – 29, 555 papers, 352, 807 citations clique: 2 1. 69 N(t) CALD Day '06 C. Faloutsos 37

CMU SCS Densification – Patent Citations • Citations among patents granted E(t) • 1999 1. 66 – 2. 9 million nodes – 16. 5 million edges • Each year is a datapoint CALD Day '06 N(t) C. Faloutsos 38

CMU SCS Densification – Autonomous Systems • Graph of Internet • 2000 E(t) 1. 18 – 6, 000 nodes – 26, 000 edges • One graph per day N(t) CALD Day '06 C. Faloutsos 39

CMU SCS Densification – Affiliation Network • Authors linked to their publications • 2002 E(t) 1. 15 – 60, 000 nodes • 20, 000 authors • 38, 000 papers – 133, 000 edges CALD Day '06 N(t) C. Faloutsos 40

CMU SCS Outline • Problem definition / Motivation • Graphs and power laws – algorithms: graph partitioning using MDL • Streams and forecasting • Conclusions CALD Day '06 C. Faloutsos 41

CMU SCS Graph partitioning • Documents x terms • Customers x products • Users x web-sites • Q: HOW MANY PIECES? CALD Day '06 C. Faloutsos 42

CMU SCS Graph partitioning • Documents x terms • Customers x products • Users x web-sites • Q: HOW MANY PIECES? • A: MDL/ compression CALD Day '06 C. Faloutsos 43

CMU SCS Cross-associations 1 x 2 CALD Day '06 C. Faloutsos 2 x 2 44

CMU SCS Cross-associations 2 x 3 CALD Day '06 3 x 3 C. Faloutsos 3 x 4 45

CMU SCS Cross-associations CALD Day '06 C. Faloutsos 46

CMU SCS Cross-associations missing edge CALD Day '06 C. Faloutsos outlier edge 47

CMU SCS education math nuclear physics bio CALD Day '06 C. Faloutsos 48

CMU SCS Conclusions • Real graphs obey some surprising patterns – which can help us spot anomalies / outliers • MDL helps partition a graph into ‘natural’ groups CALD Day '06 C. Faloutsos 49

CMU SCS Outline • • Problem definition / Motivation Graphs and power laws Streams and forecasting Conclusions CALD Day '06 C. Faloutsos 50

CMU SCS Why care about streams? • Sensor devices – Temperature, weather measurements – Road traffic data – Geological observations – Patient physiological data • Embedded devices – Network routers – Intelligent (active) disks CALD Day '06 C. Faloutsos 51

CMU SCS Co-evolving time sequences Joint work with • Jimeng Sun (CMU) • Spiros Papadimitriou (CMU/IBM) • Dr. Yasushi Sakurai (NTT) CALD Day '06 C. Faloutsos 52

CMU SCS Outline • Problem definition / Motivation • Graphs and power laws • Streams and forecasting – single stream mining & forecasting – multiple-stream mining and summarization • Conclusions CALD Day '06 C. Faloutsos 53

CMU SCS Results - Synthetic data AWSOM AR Seasonal AR • Triangle pulse • Mix (sine + square) • AR captures wrong trend (or none) • Seasonal AR estimation fails CALD Day '06 C. Faloutsos 54

CMU SCS Results – real data l l l Automobile traffic Daily periodicity l Rush-hour peaks Bursty “noise” at smaller time scales CALD Day '06 C. Faloutsos 55

CMU SCS Results - Real data • Sunspot intensity Slightly time-varying “period” • AR captures wrong trend • Seasonal ARIMA – wrong trend; needs human CALD Day '06 C. Faloutsos 56

CMU SCS Outline • Problem definition / Motivation • Graphs and power laws • Streams and forecasting – single stream mining & forecasting – multiple-stream mining and summarization • Conclusions CALD Day '06 C. Faloutsos 57

CMU SCS Motivation chlorine concentrations Phase 1 : : Phase 2 Phase 3 sensors : : near leak : : sensors : : away : : from leak water distribution network normal operation Hundreds of measurements, possibly, correlated. CALD Day '06 C. Faloutsos 58

CMU SCS Motivation chlorine concentrations Phase 1 : : Phase 2 : : Phase 3 : : sensors near leak sensors away from leak water distribution network normal operation Hundreds of measurements, possibly, correlated. CALD Day '06 C. Faloutsos 59

CMU SCS Motivation chlorine concentrations Phase 1 : : Phase 2 : : Phase 3 : : sensors near leak sensors away from leak water distribution network normal operation major leak Hundreds of measurements, possibly, correlated. CALD Day '06 C. Faloutsos 60

CMU SCS Motivation chlorine concentrations Phase 1 : : Phase 2 : : Phase 3 : : sensors near leak sensors away from leak water distribution network normal operation major leak Hundreds of measurements, possibly, correlated. CALD Day '06 C. Faloutsos 61

CMU SCS Motivation chlorine concentrations Phase 1 : : k=1 : : actual measurements (n streams) k hidden variable(s) Find: “hidden (latent) variables”, to summarize the key trends CALD Day '06 C. Faloutsos 62

CMU SCS Motivation chlorine concentrations Phase 1 Phase 2 : : k=2 : : actual measurements (n streams) k hidden variable(s) Find: “hidden (latent) variables”, to summarize the key trends CALD Day '06 C. Faloutsos 63

CMU SCS Motivation chlorine concentrations Phase 1 Phase 2 : : Phase 3 Phase 1 Phase 2 Phase 3 : : k=1 : : actual measurements (n streams) k hidden variable(s) Find: “hidden (latent) variables”, to summarize the key trends CALD Day '06 C. Faloutsos 64

CMU SCS Stream mining • Solution: SPIRIT [VLDB’ 05] – incremental, on-line PCA CALD Day '06 C. Faloutsos 65

CMU SCS Stream mining • Solution: SPIRIT [VLDB’ 05] – incremental, on-line PCA CALD Day '06 C. Faloutsos 66

CMU SCS SPIRIT • also to monitor a data center (‘self-*’ storage project www. pdl. cmu. edu/Self. Star/ • see demo of SPIRIT at http: //warsteiner. db. cs. cmu. edu • (needs JVM plugin) demo CALD Day '06 C. Faloutsos 67

CMU SCS Conclusions • Graphs & streams pose fascinating problems • MDL, PCA/SVD (wavelets): powerful tools • self-similarity, fractals and power laws work, when textbook methods fail! CALD Day '06 C. Faloutsos 68

CMU SCS Other projects • video data mining [Pan + Yang] • Virus propagation (Wang) • Anomaly detection in network traffic (Wang, Olston, ++) CALD Day '06 C. Faloutsos 69

CMU SCS Books • Manfred Schroeder: Fractals, Chaos, Power Laws: Minutes from an Infinite Paradise W. H. Freeman and Company, 1991 (Probably the BEST book on fractals!) CALD Day '06 C. Faloutsos 70

CMU SCS Contact info • christos@cs. cmu. edu • www. cs. cmu. edu/~christos • Wean Hall 7107 • Ph#: x 8. 1457 CALD Day '06 C. Faloutsos 71