Hierarchical Clustering and Network Topology Identification Rui Castro

  • Slides: 12
Download presentation
Hierarchical Clustering and Network Topology Identification Rui Castro Mark Coates Rob Nowak Department of

Hierarchical Clustering and Network Topology Identification Rui Castro Mark Coates Rob Nowak Department of Electrical and Computer Engineering Copyright © 2004 - Rui Castro

Topology Identification Ratnasamy & Mc. Canne (1999) Duffield, et al (2000, 01, 02) Bestravos,

Topology Identification Ratnasamy & Mc. Canne (1999) Duffield, et al (2000, 01, 02) Bestravos, et al (2001) Coates, et al (2001) Shih & Hero (2002) Pairwise delay measurements reveal topology Copyright © 2004 - Rui Castro

Topology Identification Challenges: • 12 % never respond, 15 % multiple interfaces - Barford

Topology Identification Challenges: • 12 % never respond, 15 % multiple interfaces - Barford et al (2000) • detect level-2 topology “invisible” to IP layer (e. g. , switches) Copyright © 2004 - Rui Castro

Relationship between Topology ID and Hierarchical Clustering Copyright © 2004 - Rui Castro

Relationship between Topology ID and Hierarchical Clustering Copyright © 2004 - Rui Castro

Sandwich Probing Do not need clock synchronization!! Copyright © 2004 - Rui Castro

Sandwich Probing Do not need clock synchronization!! Copyright © 2004 - Rui Castro

Sandwich Probing Topology imposes constraints we can infer that receivers 3 & 4 have

Sandwich Probing Topology imposes constraints we can infer that receivers 3 & 4 have a longer shared path than 3 &more 5 shared queues larger Copyright © 2004 - Rui Castro

Delay Covariance more shared queues larger covariance Copyright © 2004 - Rui Castro

Delay Covariance more shared queues larger covariance Copyright © 2004 - Rui Castro

Measurement Framework Key Assumptions: Multiple measurements • stationarity • fixed (but unknown) routes •

Measurement Framework Key Assumptions: Multiple measurements • stationarity • fixed (but unknown) routes • temporal independence individual measurement • spatial independence CLT Copyright © 2004 - Rui Castro

Maximum Likelihood Tree - MLT The maximum likelihood tree (MLT) is defined as Two

Maximum Likelihood Tree - MLT The maximum likelihood tree (MLT) is defined as Two Approaches: where • • product of Gaussian densities measurements Binary tree construction based on bottom-up, recursive selection andunknown pair-merging process similarity metric values, measurement likelihood Markov Chain Monte Carlo (MCMC) tree search forest of possible trees, monotonicity constrain set, for tree Copyright © 2004 - Rui Castro

Internet Experiments – Sandwich Probing Traceroute topology UNO MCMC ALT topology Copyright © 2004

Internet Experiments – Sandwich Probing Traceroute topology UNO MCMC ALT topology Copyright © 2004 - Rui Castro

Internet Experiments – RTT Delay Covariance Traceroute topology Estimated topology Thanks to Yolanda Tsang

Internet Experiments – RTT Delay Covariance Traceroute topology Estimated topology Thanks to Yolanda Tsang & Mehmet Yildiz Copyright © 2004 - Rui Castro

Final Remarks and Comments • Clever probing and sampling schemes reveal “hidden” network structure

Final Remarks and Comments • Clever probing and sampling schemes reveal “hidden” network structure and behavior • Likelihood based methods are a natural choice to account for uncertainty in the data • Sampling methods relying solely on RTT can be devised R. Castro, M. Coates and R. Nowak, "Likelihood Based Hierarchical Clustering", Complex interplay between. August measurement/probing IEEE Transactions in Signal Processing, 2004. techniques, statistical modeling, and computational R. Castro, M. Coates, G. Liang, R. Nowak and B. Yu, "Network Tomography: methods for optimization Recent Developments", Statistical Science, 2004 (invited paper, to appear). Copyright © 2004 - Rui Castro

Network Topology Network Topology A network topology isNetwork Topology Network Topology A network topology is
Network Topology Network Topology A network topology isNetwork Topology Network Topology A network topology is
Hierarchical Clustering Hierarchical Clustering Produces a set ofHierarchical Clustering Hierarchical Clustering Produces a set of
Hierarchical Clustering Hierarchical Clustering Produces a set ofHierarchical Clustering Hierarchical Clustering Produces a set of
HIERARCHICAL CLUSTERING Hierarchical Clustering adalah metode analisis kelompokHIERARCHICAL CLUSTERING Hierarchical Clustering adalah metode analisis kelompok
Hierarchical Clustering Hierarchical Clustering Produces a set ofHierarchical Clustering Hierarchical Clustering Produces a set of
Hierarchical Clustering Hierarchical Clustering Produces a set ofHierarchical Clustering Hierarchical Clustering Produces a set of
HIERARCHICAL CLUSTERING HIERARCHICAL CLUSTERING Strategi pengelompokkannya umumnya adaHIERARCHICAL CLUSTERING HIERARCHICAL CLUSTERING Strategi pengelompokkannya umumnya ada
FLAT CLUSTERING HIERARCHICAL CLUSTERING What is clustering GroupingFLAT CLUSTERING HIERARCHICAL CLUSTERING What is clustering Grouping
Clustering Outline Introduction Kmeans clustering Hierarchical clustering COBWEBClustering Outline Introduction Kmeans clustering Hierarchical clustering COBWEB
NETWORK SYSTEM What is Network topology Network topologyNETWORK SYSTEM What is Network topology Network topology