Link Analysis Mining Massive Datasets WuJun Li Department

Link Analysis Mining Massive Datasets Wu-Jun Li Department of Computer Science and Engineering Shanghai Jiao Tong University Lecture 7: Link Analysis 1

Link Analysis Algorithms § § § Page. Rank Hubs and Authorities Topic-Sensitive Page. Rank Spam Detection Algorithms Other interesting topics we won’t cover § Detecting duplicates and mirrors § Mining for communities (community detection) (Refer to Chapter 10 of the textbook) 2

Link Analysis Outline § Page. Rank § Topic-Sensitive Page. Rank § Hubs and Authorities § Spam Detection 3

Link Analysis Page. Ranking web pages § Web pages are not equally “important” § www. joe-schmoe. com v www. stanford. edu § Inlinks as votes § www. stanford. edu has 23, 400 inlinks § www. joe-schmoe. com has 1 inlink § Are all inlinks equal? § Recursive question! 4

Link Analysis Page. Rank Simple recursive formulation § Each link’s vote is proportional to the importance of its source page § If page P with importance x has n outlinks, each link gets x/n votes § Page P’s own importance is the sum of the votes on its inlinks 5

Page. Rank Link Analysis Simple “flow” model The web in 1839 y a/2 Yahoo y/2 y = y /2 + a /2 a = y /2 + m m = a /2 m Amazon a M’soft a/2 m 6

Link Analysis Page. Rank Solving the flow equations § 3 equations, 3 unknowns, no constants § No unique solution § All solutions equivalent modulo scale factor § Additional constraint forces uniqueness § y+a+m = 1 § y = 2/5, a = 2/5, m = 1/5 § Gaussian elimination method works for small examples, but we need a better method for large graphs 7

Link Analysis Page. Rank Matrix formulation § Matrix M has one row and one column for each web page § Suppose page j has n outlinks § If j i, then Mij=1/n § Else Mij=0 § M is a column stochastic matrix § Columns sum to 1 § Suppose r is a vector with one entry per web page § ri is the importance score of page i § Call it the rank vector § |r| = 1 8

Page. Rank Link Analysis Example Suppose page j links to 3 pages, including i j i i = 1/3 M r r 9

Link Analysis Page. Rank Eigenvector formulation § The flow equations can be written r = Mr § So the rank vector is an eigenvector of the stochastic web matrix § In fact, its first or principal eigenvector, with corresponding eigenvalue 1 10

Page. Rank Link Analysis Example y a m y 1/2 0 a 1/2 0 1 m 0 1/2 0 Yahoo r = Mr Amazon M’soft y = y /2 + a /2 a = y /2 + m m = a /2 y 1/2 0 a = 1/2 0 1 m 0 1/2 0 y a m 11

Link Analysis Page. Rank Power Iteration method § § § Simple iterative scheme (aka relaxation) Suppose there are N web pages Initialize: r 0 = [1/N, …. , 1/N]T Iterate: rk+1 = Mrk Stop when |rk+1 - rk|1 < § |x|1 = 1≤i≤N|xi| is the L 1 norm § Can use any other vector norm e. g. , Euclidean 12

Page. Rank Link Analysis Power Iteration Example y a y 1/2 a 1/2 0 m 0 1/2 Yahoo Amazon y a = m m 0 1 0 M’soft 1/3 1/3 1/2 1/6 5/12 1/3 1/4 3/8 11/24. . . 1/6 2/5 1/5 13

Link Analysis Page. Rank Random Walk Interpretation § Imagine a random web surfer § At any time t, surfer is on some page P § At time t+1, the surfer follows an outlink from P uniformly at random § Ends up on some page Q linked from P § Process repeats indefinitely § Let p(t) be a vector whose ith component is the probability that the surfer is at page i at time t § p(t) is a probability distribution on pages 14

Link Analysis Page. Rank The stationary distribution § Where is the surfer at time t+1? § Follows a link uniformly at random § p(t+1) = Mp(t) § Suppose the random walk reaches a state such that p(t+1) = Mp(t) = p(t) § Then p(t) is called a stationary distribution for the random walk § Our rank vector r satisfies r = Mr § So it is a stationary distribution for the random surfer 15

Link Analysis Page. Rank Existence and Uniqueness A central result from theory of random walks (aka Markov processes): For graphs that satisfy certain conditions, the stationary distribution is unique and eventually will be reached no matter what the initial probability distribution at time t = 0. 16

Link Analysis Page. Rank Spider traps § A group of pages is a spider trap if there are no links from within the group to outside the group § Random surfer gets trapped § Spider traps violate the conditions needed for the random walk theorem 17

Page. Rank Link Analysis Microsoft becomes a spider trap Yahoo Amazon y a y 1/2 a 1/2 0 m 0 1/2 m 0 0 1 M’soft y a = m 1 1 1/2 3/4 1/2 7/4 5/8 3/8 2 . . . 0 0 3 18

Link Analysis Page. Rank Random teleports § The Google solution for spider traps § At each time step, the random surfer has two options: § With probability , follow a link at random § With probability 1 - , jump to some page uniformly at random § Common values for are in the range 0. 8 to 0. 9 § Surfer will teleport out of spider trap within a few time steps 19

Page. Rank Link Analysis Random teleports ( = 0. 8) 0. 2*1/3 Yahoo 1/2 0. 8*1/2 0. 2*1/3 Amazon M’soft y y 1/2 a 1/2 m 0 1/2 0 0. 8 1/2 0 0 0 1/2 1 y 1/2 0. 8* 1/2 0 y 1/3 + 0. 2* 1/3 1/3 1/3 + 0. 2 1/3 1/3 1/3 y 7/15 1/15 a 7/15 1/15 m 1/15 7/15 13/15 20

Page. Rank Link Analysis Random teleports ( = 0. 8) 1/2 0 0. 8 1/2 0 0 0 1/2 1 Yahoo Amazon y a = m y 7/15 1/15 a 7/15 1/15 m 1/15 7/15 13/15 M’soft 1 1. 00 0. 60 1. 40 1/3 1/3 + 0. 2 1/3 1/3 1/3 0. 84 0. 60 1. 56 0. 776 0. 536. . . 1. 688 7/11 5/11 21

Link Analysis Page. Rank Matrix formulation § Suppose there are N pages § Consider a page j, with set of outlinks O(j) § We have Mij = 1/|O(j)| when j i and Mij = 0 otherwise § The random teleport is equivalent to § adding a teleport link from j to every other page with probability (1 - )/N § reducing the probability of following each outlink from 1/|O(j)| to /|O(j)| § Equivalent: tax each page a fraction (1 - ) of its score and redistribute evenly 22

Link Analysis Page. Rank § Construct the N*N matrix A as follows § Aij = Mij + (1 - )/N § Verify that A is a stochastic matrix § The Page. Rank vector r is the principal eigenvector of this matrix § satisfying r = Ar § Equivalently, r is the stationary distribution of the random walk with teleports 23

Link Analysis Page. Rank Dead ends § Pages with no outlinks are “dead ends” for the random surfer § Nowhere to go on next step 24

Page. Rank Link Analysis Microsoft becomes a dead end 1/2 0 0. 8 1/2 0 0 0 1/2 0 Yahoo Amazon y a = m M’soft 1 1 0. 6 1/3 1/3 + 0. 2 1/3 1/3 1/3 y 7/15 1/15 a 7/15 1/15 m 1/15 7/15 1/15 0. 787 0. 648 0. 547 0. 430. . . 0. 387 0. 333 0 0 0 Nonstochastic! 25

Link Analysis Page. Rank Dealing with dead ends § Teleport § Follow random teleport links with probability 1. 0 from dead ends § Adjust matrix accordingly § Prune and propagate § § Preprocess the graph to eliminate dead ends Might require multiple passes Compute Page. Rank on reduced graph Approximate values for dead ends by propagating values from reduced graph 26

Link Analysis Page. Rank Computing Page. Rank § Key step is matrix-vector multiplication § rnew = Arold § Easy if we have enough main memory to hold A, rold, rnew § Say N = 1 billion pages § We need 4 bytes for each entry (say) § 2 billion entries for vectors, approx 8 GB § Matrix A has N 2 entries § 1018 is a large number! 27

Link Analysis Page. Rank Rearranging the equation r = Ar, where Aij = Mij + (1 - )/N ri = 1≤j≤N Aij rj ri = 1≤j≤N [ Mij + (1 - )/N] rj = 1≤j≤N Mij rj + (1 - )/N 1≤j≤N rj = 1≤j≤N Mij rj + (1 - )/N, since |r| = 1 r = Mr + [(1 - )/N]N where [x]N is an N-vector with all entries x 28

Link Analysis Page. Rank Sparse matrix formulation § We can rearrange the Page. Rank equation: § r = Mr + [(1 - )/N]N § [(1 - )/N]N is an N-vector with all entries (1 - )/N § M is a sparse matrix! § 10 links per node, approx 10 N entries § So in each iteration, we need to: § Compute rnew = Mrold § Add a constant value (1 - )/N to each entry in rnew 29

Page. Rank Link Analysis Sparse matrix encoding § Encode sparse matrix using only nonzero entries § Space proportional roughly to number of links § say 10 N, or 4*10*1 billion = 40 GB § still won’t fit in memory, but will fit on disk source degree destination nodes node 0 3 1, 5, 7 1 5 17, 64, 113, 117, 245 2 2 13, 23 30

Link Analysis Page. Rank Basic Algorithm § Assume we have enough RAM to fit rnew, plus some working memory § Store rold and matrix M on disk Basic Algorithm: § Initialize: rold = [1/N]N § Iterate: § § § Update: Perform a sequential scan of M and rold to update rnew Write out rnew to disk as rold for next iteration Every few iterations, compute |rnew-rold| and stop if it is below threshold § Need to read in both vectors into memory 31

Page. Rank Link Analysis Update step Initialize all entries of rnew to (1 - )/N For each page p (out-degree n): Read into memory: p, n, dest 1, …, destn, rold(p) for j = 1. . n: rnew(destj) += *rold(p)/n rnew 0 1 2 3 4 5 6 src 0 degree 3 destination 1, 5, 6 1 4 17, 64, 113, 117 2 2 13, 23 rold 0 1 2 3 4 5 6 32

Link Analysis Page. Rank Analysis § In each iteration, we have to: § Read rold and M § Write rnew back to disk § IO Cost = 2|r| + |M| § What if we had enough memory to fit both rnew and rold? § What if we could not even fit rnew in memory? § 10 billion pages 33

Link Analysis Page. Rank Strip-based update Problem: thrashing 34

Link Analysis Page. Rank Block Update algorithm 35

Page. Rank Link Analysis Block Update algorithm rnew 0 1 2 3 src 0 1 degree 3 2 destination 0, 1 0 rold 2 1 0 3 3 1 2 2 3 2 2 0 1 2 3 36

Link Analysis Page. Rank Block Update algorithm § Some additional overhead § But usually worth it § Cost per iteration § |M|(1+ ) + (k+1)|r| 37

Link Analysis Outline § Page. Rank § Topic-Sensitive Page. Rank § Hubs and Authorities § Spam Detection 38

Link Analysis Topic-Sensitive Page. Rank Some problems with Page. Rank § Measures generic popularity of a page § Biased against topic-specific authorities § Ambiguous queries e. g. , jaguar § Uses a single measure of importance § Other models e. g. , hubs-and-authorities § Susceptible to link spam § Artificial link topographies created in order to boost page rank 39

Link Analysis Topic-Sensitive Page. Rank § Instead of generic popularity, can we measure popularity within a topic? § E. g. , computer science, health § Bias the random walk § When the random walker teleports, he picks a page from a set S of web pages § S contains only pages that are relevant to the topic § E. g. , Open Directory (DMOZ) pages for a given topic (www. dmoz. org) § For each teleport set S, we get a different rank vector r. S 40

Link Analysis Topic-Sensitive Page. Rank Matrix formulation § § Aij = Mij + (1 - )/|S| if i is in S Aij = Mij otherwise Show that A is stochastic We have weighted all pages in the teleport set S equally § Could also assign different weights to them 41

Topic-Sensitive Page. Rank Link Analysis Example 0. 2 0. 5 0. 4 2 1 1 0. 8 Suppose S = {1}, = 0. 8 0. 5 0. 4 3 1 0. 8 4 Node 1 2 3 4 Iteration 0 1 1. 0 0. 2 0 0. 4 0 0 2… 0. 52 0. 08 0. 32 stable 0. 294 0. 118 0. 327 0. 261 Note how we initialize the Page. Rank vector differently from the unbiased Page. Rank case. 42

Link Analysis Topic-Sensitive Page. Rank How well does TSPR work? § Experimental results [Haveliwala 2000] § Picked 16 topics § Teleport sets determined using DMOZ § E. g. , arts, business, sports, … § “Blind study” using volunteers § 35 test queries § Results ranked using Page. Rank and TSPR of most closely related topic § E. g. , bicycling using Sports ranking § In most cases volunteers preferred TSPR ranking 43

Link Analysis Topic-Sensitive Page. Rank Which topic ranking to use? § User can pick from a menu § Use Bayesian classification schemes to classify query into a topic § Can use the context of the query § E. g. , query is launched from a web page talking about a known topic § History of queries e. g. , “basketball” followed by “jordan” § User context e. g. , user’s My Yahoo settings, bookmarks, … 44

Link Analysis Outline § Page. Rank § Topic-Sensitive Page. Rank § Hubs and Authorities § Spam Detection 45

Link Analysis Hubs and Authorities § Suppose we are given a collection of documents on some broad topic § e. g. , stanford, evolution, iraq § perhaps obtained through a text search § Can we organize these documents in some manner? § Page. Rank offers one solution § HITS (Hypertext-Induced Topic Selection) is another § proposed at approx the same time (1998) 46

Link Analysis Hubs and Authorities HITS Model § § Interesting documents fall into two classes Authorities are pages containing useful information § § § course home pages of auto manufacturers Hubs are pages that link to authorities § § course bulletin list of US auto manufacturers 47

Hubs and Authorities Link Analysis Idealized view Hubs Authorities 48

Link Analysis Hubs and Authorities Mutually recursive definition § A good hub links to many good authorities § A good authority is linked from many good hubs § Model using two scores for each node § Hub score and Authority score § Represented as vectors h and a 49

Link Analysis Hubs and Authorities Transition Matrix A § HITS uses a matrix A[i, j] = 1 if page i links to page j, 0 if not § AT, the transpose of A, is similar to the Page. Rank matrix M, but AT has 1’s where M has fractions 50

Hubs and Authorities Link Analysis Example Yahoo A= Amazon y a m y 1 1 1 a 1 0 1 m 0 1 0 M’soft 51

Link Analysis Hubs and Authorities Hub and Authority Equations § The hub score of page P is proportional to the sum of the authority scores of the pages it links to § h = λAa § Constant λ is a scale factor § The authority score of page P is proportional to the sum of the hub scores of the pages it is linked from § a = μAT h § Constant μ is scale factor 52

Link Analysis Hubs and Authorities Iterative algorithm § § § Initialize h, a to all 1’s h = Aa Scale h so that its max entry is 1. 0 a = A Th Scale a so that its max entry is 1. 0 Continue until h, a converge 53

Hubs and Authorities Link Analysis Example 111 A= 101 010 110 AT = 1 0 1 110 = a(yahoo) a(amazon) = a(m’soft) = 1 1 1 . . . 1 0. 75. . . 1 1 0. 732 1 h(yahoo) = h(amazon) = h(m’soft) = 1 1 1 . . . 1 1 1 2/3 0. 71 0. 73. . . 1/3 0. 29 0. 27. . . 1. 000 0. 732 0. 268 1 4/5 1 54

Link Analysis Hubs and Authorities Existence and Uniqueness h = λAa a = μAT h h = λμAAT h a = λμATA a Under reasonable assumptions about A, the dual iterative algorithm converges to vectors h* and a* such that: • h* is the principal eigenvector of the matrix AAT • a* is the principal eigenvector of the matrix ATA 55

Hubs and Authorities Link Analysis Bipartite cores Hubs Authorities Most densely-connected core (primary core) Less densely-connected core (secondary core) 56

Link Analysis Hubs and Authorities Secondary cores § A single topic can have many bipartite cores § § corresponding to different meanings, or points of view abortion: pro-choice, pro-life evolution: darwinian, intelligent design jaguar: auto, Mac, NFL team, panthera onca § How to find such secondary cores? 57

Link Analysis Hubs and Authorities Non-primary eigenvectors § AAT and ATA have the same set of eigenvalues § An eigenpair is the pair of eigenvectors with the same eigenvalue § The primary eigenpair (largest eigenvalue) is what we get from the iterative algorithm § Non-primary eigenpairs correspond to other bipartite cores § The eigenvalue is a measure of the density of links in the core 58

Link Analysis Hubs and Authorities Finding secondary cores § Once we find the primary core, we can remove its links from the graph § Repeat HITS algorithm on residual graph to find the next bipartite core § Technically, not exactly equivalent to non-primary eigenpair model 59

Link Analysis Hubs and Authorities Creating the graph for HITS § We need a well-connected graph of pages for HITS to work well 60

Link Analysis Hubs and Authorities Page. Rank and HITS § Page. Rank and HITS are two solutions to the same problem § What is the value of an inlink from S to D? § In the Page. Rank model, the value of the link depends on the links into S § In the HITS model, it depends on the value of the other links out of S § The destinies of Page. Rank and HITS post-1998 were very different § Why? 61

Link Analysis Outline § Page. Rank § Topic-Sensitive Page. Rank § Hubs and Authorities § Spam Detection 62

Link Analysis Spam Detection Web Spam § Search has become the default gateway to the web § Very high premium to appear on the first page of search results § e. g. , e-commerce sites § advertising-driven sites 63

Link Analysis Spam Detection What is web spam? § Spamming = any deliberate action solely in order to boost a web page’s position in search engine results, incommensurate with page’s real value § Spam = web pages that are the result of spamming § This is a very broad defintion § SEO industry might disagree! § SEO = search engine optimization § Approximately 10 -15% of web pages are spam 64

Link Analysis Spam Detection Web Spam Taxonomy § We follow the treatment by Gyongyi and Garcia. Molina [2004] § Boosting techniques § Techniques for achieving high relevance/importance for a web page § Hiding techniques § Techniques to hide the use of boosting § From humans and web crawlers 65

Link Analysis Spam Detection Boosting techniques § Term spamming § Manipulating the text of web pages in order to appear relevant to queries § Link spamming § Creating link structures that boost page rank or hubs and authorities scores 66

Link Analysis Spam Detection Term Spamming § Repetition § of one or a few specific terms e. g. , free, cheap, viagra § Goal is to subvert TF. IDF ranking schemes § Dumping § of a large number of unrelated terms § e. g. , copy entire dictionaries § Weaving § Copy legitimate pages and insert spam terms at random positions § Phrase Stitching § Glue together sentences and phrases from different sources 67

Link Analysis Spam Detection Link spam § Three kinds of web pages from a spammer’s point of view § Inaccessible pages § Accessible pages § e. g. , web log comments pages § spammer can post links to his pages § Own pages § Completely controlled by spammer § May span multiple domain names 68

Link Analysis Spam Detection Link Farms § Spammer’s goal § Maximize the page rank of target page t § Technique § Get as many links from accessible pages as possible to target page t § Construct “link farm” to get page rank multiplier effect 69

Spam Detection Link Analysis Link Farms Accessible Own 1 Inaccessible t 2 M One of the most common and effective organizations for a link farm 70

Spam Detection Link Analysis Own Accessible Inaccessibl e t 1 2 M Suppose rank contributed by accessible pages = x Let page rank of target page = y Rank of each “farm” page = y/M + (1 - )/N y = x + M[ y/M + (1 - )/N] + (1 - )/N Very small; ignore = x + 2 y + (1 - )M/N + (1 - )/N y = x/(1 - 2) + c. M/N where c = /(1+ ) 71

Spam Detection Link Analysis Own Accessible Inaccessibl e t 1 2 M § y = x/(1 - 2) + c. M/N where c = /(1+ ) § For = 0. 85, 1/(1 - 2)= 3. 6 § Multiplier effect for “acquired” page rank § By making M large, we can make y as large as we want 72

Link Analysis Spam Detection Detecting Spam § Term spamming § Analyze text using statistical methods e. g. , Naïve Bayes classifiers § Similar to email spam filtering § Also useful: detecting approximate duplicate pages § Link spamming § Open research area § One approach: Trust. Rank 73

Link Analysis Spam Detection Trust. Rank idea § Basic principle: approximate isolation § It is rare for a “good” page to point to a “bad” (spam) page § Sample a set of “seed pages” from the web § Have an oracle (human) identify the good pages and the spam pages in the seed set § Expensive task, so must make seed set as small as possible 74

Link Analysis Spam Detection Trust Propagation § Call the subset of seed pages that are identified as “good” the “trusted pages” § Set trust of each trusted page to 1 § Propagate trust through links § Each page gets a trust value between 0 and 1 § Use a threshold value and mark all pages below the trust threshold as spam 75

Link Analysis Spam Detection Rules for trust propagation § Trust attenuation § The degree of trust conferred by a trusted page decreases with distance § Trust splitting § The larger the number of outlinks from a page, the less scrutiny the page author gives each outlink § Trust is “split” across outlinks 77

Link Analysis Spam Detection Simple model § Suppose trust of page p is t(p) § Set of outlinks O(p) § For each q in O(p), p confers the trust § t(p)/|O(p)| for 0< <1 § Trust is additive § Trust of p is the sum of the trust conferred on p by all its inlinked pages § Note similarity to Topic-Specific Page. Rank § Within a scaling factor, trust rank = biased page rank with trusted pages as teleport set 78

Link Analysis Spam Detection Picking the seed set § Two conflicting considerations § Human has to inspect each seed page, so seed set must be as small as possible § Must ensure every “good page” gets adequate trust rank, so need make all good pages reachable from seed set by short paths 79

Link Analysis Spam Detection Approaches to picking seed set § Suppose we want to pick a seed set of k pages § Page. Rank § Pick the top k pages by page rank § Assume high page rank pages are close to other highly ranked pages § We care more about high page rank “good” pages 80

Link Analysis Spam Detection Inverse page rank § Pick the pages with the maximum number of outlinks § Can make it recursive § Pick pages that link to pages with many outlinks § Formalize as “inverse page rank” § Construct graph G’ by reversing each edge in web graph G § Page rank in G’ is inverse page rank in G § Pick top k pages by inverse page rank 81

Link Analysis Spam Detection Spam Mass § In the Trust. Rank model, we start with good pages and propagate trust § Complementary view: what fraction of a page’s page rank comes from “spam” pages? § In practice, we don’t know all the spam pages, so we need to estimate 82

Link Analysis Spam Detection Spam mass estimation r(p) = page rank of page p r+(p) = page rank of p with teleport into “good” pages only r-(p) = r(p) – r+(p) Spam mass of p = r-(p)/r(p) 83

Link Analysis Spam Detection Good pages § For spam mass, we need a large set of “good” pages § Need not be as careful about quality of individual pages as with Trust. Rank § One reasonable approach §. edu sites §. gov sites §. mil sites 84

Link Analysis Acknowledgement § Slides are from § Prof. Jeffrey D. Ullman § Dr. Anand Rajaraman 85