School of Computer Science Carnegie Mellon Data Mining

School of Computer Science Carnegie Mellon Data Mining using Fractals and Power laws Christos Faloutsos Carnegie Mellon University Boston U. , 2005 C. Faloutsos 1

School of Computer Science Carnegie Mellon THANK YOU! • Prof. Azer Bestavros • Prof. Mark Crovella • Prof. George Kollios Boston U. , 2005 C. Faloutsos 2

School of Computer Science Carnegie Mellon Overview • Goals/ motivation: find patterns in large datasets: – (A) Sensor data – (B) network/graph data • Solutions: self-similarity and power laws • Discussion Boston U. , 2005 C. Faloutsos 3

School of Computer Science Carnegie Mellon Applications of sensors/streams • ‘Smart house’: monitoring temperature, humidity etc • Financial, sales, economic series Boston U. , 2005 C. Faloutsos 4

School of Computer Science Carnegie Mellon Motivation - Applications • Medical: ECGs +; blood pressure etc monitoring • Scientific data: seismological; astronomical; environment / anti-pollution; meteorological [Kollios+, ICDE’ 04] Boston U. , 2005 C. Faloutsos 5

School of Computer Science Carnegie Mellon Motivation - Applications (cont’d) • civil/automobile infrastructure – bridge vibrations [Oppenheim+02] – road conditions / traffic monitoring # cars 2000 1800 1600 1400 1200 1000 800 600 400 200 0 Boston U. , 2005 Automobile traffic C. Faloutsos time 6

School of Computer Science Carnegie Mellon Motivation - Applications (cont’d) • Computer systems – web servers (buffering, prefetching) – network traffic monitoring –. . . http: //repository. cs. vt. edu/lbl-conn-7. tar. Z Boston U. , 2005 C. Faloutsos 7

School of Computer Science Carnegie Mellon Web traffic • [Crovella Bestavros, SIGMETRICS’ 96] 1000 sec Boston U. , 2005 C. Faloutsos 8

School of Computer Science Carnegie Mellon Self-* Storage (Ganger+) § “self-*” = self-managing, self-tuning, self-healing, … § Goal: 1 petabyte (PB) for CMU researchers § www. pdl. cmu. edu/Self. Star survivable, self-managing storage infrastructure ~1 PB Boston U. , 2005 . . . C. Faloutsos a storage brick (0. 5– 5 TB) 9

School of Computer Science Carnegie Mellon Problem definition • Given: one or more sequences x 1 , x 2 , … , xt , …; (y 1, y 2, … , yt, …) • Find – patterns; clusters; outliers; forecasts; Boston U. , 2005 C. Faloutsos 10

School of Computer Science Carnegie Mellon Problem #1 # bytes • Find patterns, in large datasets time Boston U. , 2005 C. Faloutsos 11

School of Computer Science Carnegie Mellon Problem #1 # bytes • Find patterns, in large datasets time Poisson indep. , ident. distr Boston U. , 2005 C. Faloutsos 12

School of Computer Science Carnegie Mellon Problem #1 # bytes • Find patterns, in large datasets time Poisson indep. , ident. distr Boston U. , 2005 C. Faloutsos 13

School of Computer Science Carnegie Mellon Problem #1 # bytes • Find patterns, in large datasets time Poisson indep. , ident. distr Boston U. , 2005 Q: Then, how to generate such bursty traffic? C. Faloutsos 14

School of Computer Science Carnegie Mellon Overview • Goals/ motivation: find patterns in large datasets: – (A) Sensor data – (B) network/graph data • Solutions: self-similarity and power laws • Discussion Boston U. , 2005 C. Faloutsos 15

School of Computer Science Carnegie Mellon Problem #2 - network and graph mining • How does the Internet look like? • How does the web look like? • What constitutes a ‘normal’ social network? • What is the ‘network value’ of a customer? • which gene/species affects the others the most? Boston U. , 2005 C. Faloutsos 16

School of Computer Science Carnegie Mellon Network and graph mining Friendship Network [Moody ’ 01] Food Web [Martinez ’ 91] Protein Interactions [genomebiology. com] Graphs are everywhere! Boston U. , 2005 C. Faloutsos 17

School of Computer Science Carnegie Mellon Problem#2 Given a graph: • which node to market-to / defend / immunize first? • Are there un-natural subgraphs? (eg. , criminals’ rings)? [from Lumeta: ISPs 6/1999] Boston U. , 2005 C. Faloutsos 18

School of Computer Science Carnegie Mellon Solutions • New tools: power laws, self-similarity and ‘fractals’ work, where traditional assumptions fail • Let’s see the details: Boston U. , 2005 C. Faloutsos 19

School of Computer Science Carnegie Mellon Overview • Goals/ motivation: find patterns in large datasets: – (A) Sensor data – (B) network/graph data • Solutions: self-similarity and power laws • Discussion Boston U. , 2005 C. Faloutsos 20

School of Computer Science Carnegie Mellon What is a fractal? = self-similar point set, e. g. , Sierpinski triangle: . . . zero area: (3/4)^inf infinite length! (4/3)^inf Q: What is its dimensionality? ? Boston U. , 2005 C. Faloutsos 21

School of Computer Science Carnegie Mellon What is a fractal? = self-similar point set, e. g. , Sierpinski triangle: . . . zero area: (3/4)^inf infinite length! (4/3)^inf Q: What is its dimensionality? ? A: log 3 / log 2 = 1. 58 (!? !) Boston U. , 2005 C. Faloutsos 22

School of Computer Science Carnegie Mellon Intrinsic (‘fractal’) dimension • Q: fractal dimension of • Q: fd of a plane? a line? Boston U. , 2005 C. Faloutsos 23

School of Computer Science Carnegie Mellon Intrinsic (‘fractal’) dimension • Q: fractal dimension of • Q: fd of a plane? a line? • A: nn ( <= r ) ~ r^2 • A: nn ( <= r ) ~ r^1 fd== slope of (log(nn) (‘power law’: y=x^a) vs. . log(r) ) Boston U. , 2005 C. Faloutsos 24

School of Computer Science Carnegie Mellon Sierpinsky triangle == ‘correlation integral’ log(#pairs within <=r ) = CDF of pairwise distances 1. 58 log( r ) Boston U. , 2005 C. Faloutsos 25

School of Computer Science Carnegie Mellon Observations: Fractals <-> power laws Closely related: • fractals <=> • self-similarity <=> • scale-free <=> • power laws ( y= xa ; F=K r-2) 1. 58 • (vs y=e-ax or y=xa+b) Boston U. , 2005 log(#pairs within <=r ) C. Faloutsos log( r ) 26

School of Computer Science Carnegie Mellon Outline • • Problems Self-similarity and power laws Solutions to posed problems Discussion Boston U. , 2005 C. Faloutsos 27

School of Computer Science Carnegie Mellon Solution #1: traffic • disk traces: self-similar: (also: [Leland+94]) • How to generate such traffic? #bytes time Boston U. , 2005 C. Faloutsos 28

School of Computer Science Carnegie Mellon Solution #1: traffic • disk traces (80 -20 ‘law’) – ‘multifractals’ 20% 80% #bytes time Boston U. , 2005 C. Faloutsos 29

School of Computer Science Carnegie Mellon 80 -20 / multifractals 20 Boston U. , 2005 80 C. Faloutsos 30

School of Computer Science Carnegie Mellon 80 -20 / multifractals 20 80 • p ; (1 -p) in general • yes, there are dependencies Boston U. , 2005 C. Faloutsos 31

School of Computer Science Carnegie Mellon More on 80/20: PQRS • Part of ‘self-* storage’ project time Boston U. , 2005 cylinder# C. Faloutsos 32

School of Computer Science Carnegie Mellon More on 80/20: PQRS • Part of ‘self-* storage’ project Boston U. , 2005 p q r s C. Faloutsos q r s 33

School of Computer Science Carnegie Mellon Overview • Goals/ motivation: find patterns in large datasets: – (A) Sensor data – (B) network/graph data • Solutions: self-similarity and power laws – sensor/traffic data – network/graph data • Discussion Boston U. , 2005 C. Faloutsos 34

School of Computer Science Carnegie Mellon Problem #2 - topology How does the Internet look like? Any rules? Boston U. , 2005 C. Faloutsos 35

School of Computer Science Carnegie Mellon Patterns? • avg degree is, say 3. 3 • pick a node at random – guess its degree, exactly (-> “mode”) count avg: 3. 3 Boston U. , 2005 degree C. Faloutsos 36

School of Computer Science Carnegie Mellon Patterns? • avg degree is, say 3. 3 • pick a node at random – guess its degree, exactly (-> “mode”) • A: 1!! count avg: 3. 3 Boston U. , 2005 degree C. Faloutsos 37

School of Computer Science Carnegie Mellon Patterns? • avg degree is, say 3. 3 • pick a node at random - what is the degree you expect it to have? • A: 1!! • A’: very skewed distr. • Corollary: the mean is meaningless! • (and std -> infinity (!)) count avg: 3. 3 Boston U. , 2005 degree C. Faloutsos 38

School of Computer Science Carnegie Mellon Solution#2: Rank exponent R • A 1: Power law in the degree distribution [SIGCOMM 99] internet domains log(degree) att. com ibm. com -0. 82 log(rank) Boston U. , 2005 C. Faloutsos 39

School of Computer Science Carnegie Mellon Solution#2’: Eigen Exponent E Eigenvalue Exponent = slope E = -0. 48 May 2001 Rank of decreasing eigenvalue • A 2: power law in the eigenvalues of the adjacency matrix Boston U. , 2005 C. Faloutsos 40

School of Computer Science Carnegie Mellon Power laws - discussion • do they hold, over time? • do they hold on other graphs/domains? Boston U. , 2005 C. Faloutsos 41

School of Computer Science Carnegie Mellon Power laws - discussion • • do they hold, over time? Yes! for multiple years [Siganos+] do they hold on other graphs/domains? Yes! – web sites and links [Tomkins+], [Barabasi+] – peer-to-peer graphs (gnutella-style) – who-trusts-whom (epinions. com) Boston U. , 2005 C. Faloutsos 42

School of Computer Science Carnegie Mellon att. com log(degree) ibm. com Time Evolution: rank R 0. 82 log(rank Domain level • The rank exponent has not changed! [Siganos+] Boston U. , 2005 C. Faloutsos 43

School of Computer Science Carnegie Mellon The Peer-to-Peer Topology count [Jovanovic+] degree • Number of immediate peers (= degree), follows a power-law Boston U. , 2005 C. Faloutsos 44

School of Computer Science Carnegie Mellon epinions. com • who-trusts-whom [Richardson + Domingos, KDD 2001] count (out) degree Boston U. , 2005 C. Faloutsos 45

School of Computer Science Carnegie Mellon Why care about these patterns? • better graph generators [BRITE, INET] – for simulations – extrapolations • ‘abnormal’ graph and subgraph detection Boston U. , 2005 C. Faloutsos 46

School of Computer Science Carnegie Mellon Outline • • problems Fractals Solutions Discussion – what else can they solve? – how frequent are fractals? Boston U. , 2005 C. Faloutsos 47

School of Computer Science Carnegie Mellon What else can they solve? • • • separability [KDD’ 02] forecasting [CIKM’ 02] dimensionality reduction [SBBD’ 00] non-linear axis scaling [KDD’ 02] disk trace modeling [PEVA’ 02] selectivity of spatial/multimedia queries [PODS’ 94, VLDB’ 95, ICDE’ 00] • . . . Boston U. , 2005 C. Faloutsos 48

School of Computer Science Carnegie Mellon Full Content Indexing, Search and Retrieval from Digital Video Archives • Query (6 TB of data) • Search results (ranked) Storyboard Collage with maps, common phrases, named entities and dynamic query sliders www. informedia. cs. cmu. edu Boston U. , 2005 C. Faloutsos 49

School of Computer Science Carnegie Mellon What else can they solve? • • • separability [KDD’ 02] forecasting [CIKM’ 02] dimensionality reduction [SBBD’ 00] non-linear axis scaling [KDD’ 02] disk trace modeling [PEVA’ 02] selectivity of spatial/multimedia queries [PODS’ 94, VLDB’ 95, ICDE’ 00] • . . . Boston U. , 2005 C. Faloutsos 50

School of Computer Science Carnegie Mellon Problem #3 - spatial d. m. Galaxies (Sloan Digital Sky Survey w/ B. - ‘spiral’ and ‘elliptical’ Nichol) galaxies - patterns? (not Gaussian; not uniform) -attraction/repulsion? - separability? ? Boston U. , 2005 C. Faloutsos 51

School of Computer Science Carnegie Mellon Solution#3: spatial d. m. log(#pairs within <=r ) CORRELATION INTEGRAL! - 1. 8 slope - plateau! ell-ell - repulsion! spi-spi spi-ell log(r) Boston U. , 2005 C. Faloutsos 52

School of Computer Science Carnegie Mellon Solution#3: spatial d. m. [w/ Seeger, Traina, SIGMOD 00] log(#pairs within <=r ) - 1. 8 slope - plateau! ell-ell - repulsion! spi-spi spi-ell log(r) Boston U. , 2005 C. Faloutsos 53

School of Computer Science Carnegie Mellon spatial d. m. r 1 r 2 Heuristic on choosing # of clusters r 2 r 1 Boston U. , 2005 C. Faloutsos 54

School of Computer Science Carnegie Mellon Solution#3: spatial d. m. log(#pairs within <=r ) - 1. 8 slope - plateau! ell-ell - repulsion! spi-spi spi-ell log(r) Boston U. , 2005 C. Faloutsos 55

School of Computer Science Carnegie Mellon Problem#4: dim. reduction • given attributes x 1, . . . xn cc – possibly, non-linearly correlated • drop the useless ones Boston U. , 2005 C. Faloutsos mpg 56

School of Computer Science Carnegie Mellon Problem#4: dim. reduction • given attributes x 1, . . . xn cc – possibly, non-linearly correlated • drop the useless ones mpg (Q: why? A: to avoid the ‘dimensionality curse’) Solution: keep on dropping attributes, until the f. d. changes! [SBBD’ 00] Boston U. , 2005 C. Faloutsos 57

School of Computer Science Carnegie Mellon Outline • • problems Fractals Solutions Discussion – what else can they solve? – how frequent are fractals? Boston U. , 2005 C. Faloutsos 58

School of Computer Science Carnegie Mellon Fractals & power laws: appear in numerous settings: • medical • geographical / geological • social • computer-system related • <and many-many more! see [Mandelbrot]> Boston U. , 2005 C. Faloutsos 59

School of Computer Science Carnegie Mellon Fractals: Brain scans • brain-scans Log(#octants) 2. 63 = fd Boston U. , 2005 C. Faloutsos octree levels 60

School of Computer Science Carnegie Mellon f. MRI brain scans • Center for Cognitive Brain Imaging @ CMU • Tom Mitchell, Marcel Just, ++ f. MRI Goal: human brain function Which voxels are active, for a given cognitive task? Boston U. , 2005 C. Faloutsos 61

School of Computer Science Carnegie Mellon More fractals • periphery of malignant tumors: ~1. 5 • benign: ~1. 3 • [Burdet+] Boston U. , 2005 C. Faloutsos 62

School of Computer Science Carnegie Mellon More fractals: • cardiovascular system: 3 (!) lungs: ~2. 9 Boston U. , 2005 C. Faloutsos 63

School of Computer Science Carnegie Mellon Fractals & power laws: appear in numerous settings: • medical • geographical / geological • social • computer-system related Boston U. , 2005 C. Faloutsos 64

School of Computer Science Carnegie Mellon More fractals: • Coastlines: 1. 2 -1. 58 1 1. 3 Boston U. , 2005 C. Faloutsos 65

School of Computer Science Carnegie Mellon Boston U. , 2005 C. Faloutsos 66

School of Computer Science Carnegie Mellon GIS points Cross-roads of Montgomery county: • any rules? Boston U. , 2005 C. Faloutsos 67

School of Computer Science Carnegie Mellon GIS log(#pairs(within <= r)) A: self-similarity: • intrinsic dim. = 1. 51 log( r ) Boston U. , 2005 C. Faloutsos 68

School of Computer Science Carnegie Mellon Examples: LB county • Long Beach county of CA (road end-points) log(#pairs) 1. 7 log(r) Boston U. , 2005 C. Faloutsos 69

School of Computer Science Carnegie Mellon More power laws: areas – Korcak’s law Scandinavian lakes Any pattern? Boston U. , 2005 C. Faloutsos 70

School of Computer Science Carnegie Mellon More power laws: areas – Korcak’s law log(count( >= area)) Scandinavian lakes area vs complementary cumulative count (log-log axes) Boston U. , 2005 log(area) C. Faloutsos 71

School of Computer Science Carnegie Mellon More power laws: Korcak log(count( >= area)) Japan islands; area vs cumulative count (log-log axes) Boston U. , 2005 log(area) C. Faloutsos 72

School of Computer Science Carnegie Mellon More power laws • Energy of earthquakes (Gutenberg-Richter law) [simscience. org] Energy released log(count) day Boston U. , 2005 Magnitude = log(energy) C. Faloutsos 73

School of Computer Science Carnegie Mellon Fractals & power laws: appear in numerous settings: • medical • geographical / geological • social • computer-system related Boston U. , 2005 C. Faloutsos 74

School of Computer Science Carnegie Mellon A famous power law: Zipf’s law log(freq) “a” • Bible - rank vs. frequency (log-log) “the” “Rank/frequency plot” log(rank) Boston U. , 2005 C. Faloutsos 75

School of Computer Science Carnegie Mellon TELCO data count of customers ‘best customer’ # of service units Boston U. , 2005 C. Faloutsos 76

School of Computer Science Carnegie Mellon SALES data – store#96 count of products “aspirin” # units sold Boston U. , 2005 C. Faloutsos 77

School of Computer Science Carnegie Mellon Olympic medals (Sidney’ 00, Athens’ 04): log(#medals) log( rank) Boston U. , 2005 C. Faloutsos 78

School of Computer Science Carnegie Mellon Even more power laws: • Income distribution (Pareto’s law) • size of firms • publication counts (Lotka’s law) Boston U. , 2005 C. Faloutsos 79

School of Computer Science Carnegie Mellon Even more power laws: library science (Lotka’s law of publication count); and citation counts: (citeseer. nj. nec. com 6/2001) log(count) Ullman log(#citations) Boston U. , 2005 C. Faloutsos 80

School of Computer Science Carnegie Mellon Even more power laws: • web hit counts [w/ A. Montgomery] Web Site Traffic log(count) Zipf “yahoo. com” log(freq) Boston U. , 2005 C. Faloutsos 81

School of Computer Science Carnegie Mellon Fractals & power laws: appear in numerous settings: • medical • geographical / geological • social • computer-system related Boston U. , 2005 C. Faloutsos 82

School of Computer Science Carnegie Mellon Power laws, cont’d • In- and out-degree distribution of web sites [Barabasi], [IBM-CLEVER] log indegree from [Ravi Kumar, Prabhakar Raghavan, Sridhar Rajagopalan, Andrew Tomkins ] Boston U. , 2005 - log(freq) C. Faloutsos 83

School of Computer Science Carnegie Mellon Power laws, cont’d • In- and out-degree distribution of web sites [Barabasi], [IBM-CLEVER] • length of file transfers [Crovella+Bestavros ‘ 96] • duration of UNIX jobs [Harchol-Balter] Boston U. , 2005 C. Faloutsos 84

School of Computer Science Carnegie Mellon Conclusions • Fascinating problems in Data Mining: find patterns in – sensors/streams – graphs/networks Boston U. , 2005 C. Faloutsos 85

School of Computer Science Carnegie Mellon Conclusions - cont’d New tools for Data Mining: self-similarity & power laws: appear in many cases Bad news: lead to skewed distributions (no Gaussian, Poisson, uniformity, independence, mean, variance) Boston U. , 2005 C. Faloutsos Good news: • ‘correlation integral’ for separability • rank/frequency plots • 80 -20 (multifractals) • • (Hurst exponent, strange attractors, renormalization theory, ++) 86

School of Computer Science Carnegie Mellon Resources • Manfred Schroeder “Chaos, Fractals and Power Laws”, 1991 • Jiawei Han and Micheline Kamber “Data Mining: Concepts and Techniques”, 2001 Boston U. , 2005 C. Faloutsos 87

School of Computer Science Carnegie Mellon References • [vldb 95] Alberto Belussi and Christos Faloutsos, Estimating the Selectivity of Spatial Queries Using the `Correlation' Fractal Dimension Proc. of VLDB, p. 299 -310, 1995 • M. Crovella and A. Bestavros, Self similarity in World wide web traffic: Evidence and possible causes , SIGMETRICS ’ 96. Boston U. , 2005 C. Faloutsos 88

School of Computer Science Carnegie Mellon References • J. Considine, F. Li, G. Kollios and J. Byers, Approximate Aggregation Techniques for Sensor Databases (ICDE’ 04, best paper award). • [pods 94] Christos Faloutsos and Ibrahim Kamel, Beyond Uniformity and Independence: Analysis of R-trees Using the Concept of Fractal Dimension, PODS, Minneapolis, MN, May 24 -26, 1994, pp. 4 -13 Boston U. , 2005 C. Faloutsos 89

School of Computer Science Carnegie Mellon References • [vldb 96] Christos Faloutsos, Yossi Matias and Avi Silberschatz, Modeling Skewed Distributions Using Multifractals and the `80 -20 Law’ Conf. on Very Large Data Bases (VLDB), Bombay, India, Sept. 1996. • [sigmod 2000] Christos Faloutsos, Bernhard Seeger, Agma J. M. Traina and Caetano Traina Jr. , Spatial Join Selectivity Using Power Laws, SIGMOD 2000 Boston U. , 2005 C. Faloutsos 90

School of Computer Science Carnegie Mellon References • [vldb 96] Christos Faloutsos and Volker Gaede Analysis of the Z-Ordering Method Using the Hausdorff Fractal Dimension VLD, Bombay, India, Sept. 1996 • [sigcomm 99] Michalis Faloutsos, Petros Faloutsos and Christos Faloutsos, What does the Internet look like? Empirical Laws of the Internet Topology, SIGCOMM 1999 Boston U. , 2005 C. Faloutsos 91

School of Computer Science Carnegie Mellon References • [ieee. TN 94] W. E. Leland, M. S. Taqqu, W. Willinger, D. V. Wilson, On the Self-Similar Nature of Ethernet Traffic, IEEE Transactions on Networking, 2, 1, pp 1 -15, Feb. 1994. • [brite] Alberto Medina, Anukool Lakhina, Ibrahim Matta, and John Byers. BRITE: An Approach to Universal Topology Generation. MASCOTS '01 Boston U. , 2005 C. Faloutsos 92

School of Computer Science Carnegie Mellon References • [icde 99] Guido Proietti and Christos Faloutsos, I/O complexity for range queries on region data stored using an R-tree (ICDE’ 99) • Stan Sclaroff, Leonid Taycher and Marco La Cascia , "Image. Rover: A content-based image browser for the world wide web" Proc. IEEE Workshop on Content-based Access of Image and Video Libraries, pp 2 -9, 1997. Boston U. , 2005 C. Faloutsos 93

School of Computer Science Carnegie Mellon References • [kdd 2001] Agma J. M. Traina, Caetano Traina Jr. , Spiros Papadimitriou and Christos Faloutsos: Triplots: Scalable Tools for Multidimensional Data Mining, KDD 2001, San Francisco, CA. Boston U. , 2005 C. Faloutsos 94

School of Computer Science Carnegie Mellon Thank you! Contact info: christos <at> cs. cmu. edu www. cs. cmu. edu /~christos (w/ papers, datasets, code for fractal dimension estimation, etc) Boston U. , 2005 C. Faloutsos 95