Page Rank Page Rank Page Rank Page RankMap

Page. Rank内容概述 什么是Page. Rank的简化模型 Page. Rank的随机浏览模型 Page. Rank的Map. Reduce实现

用Map. Reduce实现Page. Rank Phase 1: Graph. Builder 建立网页之间的超链接图 Phase 2: Page. Rank. Iter 迭代计算各个网页的Page. Rank值 Phase 3: Rank. Viewer 按Page. Rank值从大到小输出

Page. Rank in Map. Reduce n 1 [n 2, n 4] n 2 [n 3, n 5] n 2 n 3 [n 4] n 4 [n 5] n 4 n 5 [n 1, n 2, n 3] Map n 1 n 4 n 2 n 5 n 3 n 4 n 1 n 2 n 5 Reduce n 1 [n 2, n 4] n 2 [n 3, n 5] n 3 [n 4] n 4 [n 5] n 5 [n 1, n 2, n 3] n 3 n 5

Phase 2：Page. Rank. Iter伪代码

Phase 3：Rankviewer Page. Rank. Viewer：将最终结果排序输出。 • Page. Rank. Viewer从最后一次迭代的结果读出文件，并将文件名 和其PR值读出，并以PR值为key网页名为value，并且以PR值从 大到小的顺序输出。 • 排序过程中可以采用框架自身的排序处理，重载key的比较 函数，使其经过shuffle和sort后反序（从大到小）输出 public static class Dec. Float. Writable extends Float. Writable { … @Override public int compare. To(Object o) { return -super. compare. To(o); } }

多趟Map. Reduce的处理 public class Page. Rank. Driver { private static int times = 10; public static void main(String args[]) throws Exception { String[] for. GB = {"", args[1]+"/Data 0"}; for. GB[0] = args[0]; Graph. Builder. main(for. GB); String[] for. Itr = {"Data", "Data"}; for (int i=0; i<times; i++) { for. Itr[0] = args[1]+"/Data"+(i); for. Itr[1] = args[1]+"/Data"+(i+1); Page. Rank. Iter. main(for. Itr); } String[] for. RV = {args[1]+"/Data"+times, args[1]+"/Final. Rank"}; Page. Rank. Viewer. main(for. RV); } } 也可以使用org. apache. hadoop. util. Program. Driver

Dijkstra最短路径算法 1 10 2 0 3 9 6 7 5 2 Example from CLR 4

Dijkstra最短路径算法 1 10 10 2 0 3 9 6 7 5 5 2 Example from CLR 4

Dijkstra最短路径算法 1 8 14 10 2 0 3 9 6 7 5 5 7 2 Example from CLR 4

Dijkstra最短路径算法 1 8 13 10 2 0 3 9 6 7 5 5 7 2 Example from CLR 4

Dijkstra最短路径算法 11 8 9 10 2 0 3 9 6 7 5 5 7 2 Example from CLR 4

Dijkstra最短路径算法 1 8 9 10 2 0 3 9 6 7 5 5 7 2 Example from CLR 4

Reference 1. The Anatomy of a Large-Scale Hypertextual Web Search Engine. Sergey Brin and Lawrence Page. 2. Ecient Computation of Page. Rank. Taher H. Haveliwala. 3. The Page. Rank Citation Ranking: Bring Order to the Web 4. Data-Intensive Text Processing with Map. Reduce Jimmy Lin and Chris Dyer