A Constraint Satisfaction Approach to Testbed Embedding Services

  • Slides: 21
Download presentation
A Constraint Satisfaction Approach to Testbed Embedding Services John Byers Dept. of Computer Science,

A Constraint Satisfaction Approach to Testbed Embedding Services John Byers Dept. of Computer Science, Boston University www. cs. bu. edu/~byers Joint work with Jeffrey Considine (BU) and Ketan Mayer-Patel (UNC)

Experimental Methodologies • Simulation s “Blank slate” for crafting experiments s Fine-grained control, specifying

Experimental Methodologies • Simulation s “Blank slate” for crafting experiments s Fine-grained control, specifying all details s No external surprises, not especially realistic • Emulation s All the benefits of simulation, plus: running real protocols on real systems • Internet experimentation s None of the benefits of simulation, minus: unpredictability, unrepeatability, etc. s But realistic!

Our Position • All three approaches have their place. • Improving aspects of all

Our Position • All three approaches have their place. • Improving aspects of all three is essential. s Focus of recent workshops like MOME Tools s Internet experimentation is the most primitive by far. • Our question: Can we bridge over some of the attractive features of simulation and emulation into wide-area testbed experimentation? • Towards an answer: s Which services would be useful? s Outline design of a set of interesting services.

Useful Services • Canonical target testbed: Planet. Lab • What services would we like

Useful Services • Canonical target testbed: Planet. Lab • What services would we like to bridge over? s Abstract: repeatability, representativeness s Concrete: • • • specify parameters of an experiment just like in ns locate one or more sub-topologies matching specification run experiment monitor it while running (“measurement blackboard”) put it all in a cron job

Embedding Services 1. Topology specification 2. Testbed characterization 1. Relevant parameters unknown, but measurable

Embedding Services 1. Topology specification 2. Testbed characterization 1. Relevant parameters unknown, but measurable 3. Embedding discovery 1. Automatically find one or more embeddings of specified topology Synergistic relationships between above services. s s Existing measurements guide discovery. Discovery feeds back into measurement process.

Emulab/Netbed • In the emulation world, Emulab and Netbed researchers have worked extensively on

Emulab/Netbed • In the emulation world, Emulab and Netbed researchers have worked extensively on related problems [OSDI ’ 02, Hot. Nets-I, CCR ’ 03] • Rich experimental specification language. • Optimization-based solver to map desired topology onto Netbed to: s s balance load across Netbed processors minimize inter-switch bandwidth minimize interference between experiments incorporate wide-area constraints

Wide-area challenges • Conditions change continuously on wide-area testbeds - “Measure twice, embed once”.

Wide-area challenges • Conditions change continuously on wide-area testbeds - “Measure twice, embed once”. • The space of possible embeddings is very large; finding feasible ones is the challenge. • We argue for a constraint satisfaction approach rather than optimization-based. s Pros and cons upcoming.

Specifying Topologies • N nodes in testbed, k nodes in specification • k x

Specifying Topologies • N nodes in testbed, k nodes in specification • k x k constraint matrix C = {ci, j} • Entry ci, j constrains the end-to-end path between embedding of virtual nodes i and j. • For example, place bounds on RTTs: ci, j = [li, j, hi, j] represents lower and upper bounds on target RTT. • Constraints can be multi-dimensional. • Constraints can also be placed on nodes. • More complex specifications possible. . .

Feasible Embeddings • Def’n: A feasible embedding is a mapping f such that for

Feasible Embeddings • Def’n: A feasible embedding is a mapping f such that for all i, j where f(i) = x and f(j) = y: li, j ≤ d (x, y) ≤ hi, j • Do not need to know d (x, y) exactly, only that li, j ≤ l’(x, y) ≤ d (x, y) ≤ h’ (x, y) ≤ hi, j • Key point: Testbed need not be exhaustively characterized, only sufficiently well to embed.

Why Constraint-Based? • Simplicity: binary yes-no answer. • Allows sampling from feasible embeddings. •

Why Constraint-Based? • Simplicity: binary yes-no answer. • Allows sampling from feasible embeddings. • Admits a parsimonious set of measurements to locate a feasible embedding. • For infeasible set of constraints, hints for relaxing constraints can be provided. • Optimization approaches depend crucially on user’s setting of weights.

Hardness • Finding an embedding is as hard as subgraph isomorphism (NP-complete) • Counting

Hardness • Finding an embedding is as hard as subgraph isomorphism (NP-complete) • Counting or sampling from set of feasible embeddings is #P-Complete. • Approximation algorithms are not much better. • Uh-oh. . .

Our Approach • Brute force search. • We’re not kidding. • Situation is not

Our Approach • Brute force search. • We’re not kidding. • Situation is not as dire as it sounds. s Several methods for pruning the search tree. s Adaptive measurements. s Many problem instances not near boundary of solubility and insolubility. • Off-line searches up to thousands of nodes. • On-line searches up to hundreds of nodes.

Adaptive Measurements • Must we make O(N 2) measurements? No. s Delay: coordinate-based [Cox

Adaptive Measurements • Must we make O(N 2) measurements? No. s Delay: coordinate-based [Cox et al (today)] s Loss: tomography-based [Chen et al ’ 03] s Underlays may make them for you [Nakao et al ’ 03] • In our setting: s We don’t always need exact values. s Pair-wise measurements are expensive. • How do we avoid measurements? s Interactions with search. s Inferences of unmeasured paths.

Triangle Inequality Inferences Suppose constraints are on delays, and the triangle inequality holds. j

Triangle Inequality Inferences Suppose constraints are on delays, and the triangle inequality holds. j [10, 15 ] [ 90, 100 ] i [ 75 , 115 ] k Using APSP algorithms, can compute all upper & lower bounds.

Experimental Setup • Starting from Planet. Lab production node list, we removed any hosts…

Experimental Setup • Starting from Planet. Lab production node list, we removed any hosts… s not responding to pings s with full file systems s with CPU load over 2. 0 (measured with uptime) • 118 hosts remaining • Used snapshot of pings between them

Finding Cliques • Biggest clique of nodes within 10 ms s Unique 11 node

Finding Cliques • Biggest clique of nodes within 10 ms s Unique 11 node clique covering 6 institutions • If 1 ms lower bound added, s Twenty 6 node cliques s 5 institutions always present, only 2 others Size 2 3 4 5 6 7 8 9 10 11 0 -10 ms Cliques 403 8 936 5 1475 9 1645 17 1327 8 771 8 315 8 86 3 14 3 1 1 -10 ms Cliques 325 35 501 61 387 84 142 60 20 0 0

Finding Groups of Cliques # of Cliques Clique Size 1 2 325 3 501

Finding Groups of Cliques # of Cliques Clique Size 1 2 325 3 501 4 387 5 6 142 20 2 6898 6238 1004 0 0 0 3 12950 0 0 4 0 0 0 1 -10 ms within same clique, 20 -50 ms otherwise 7 0

Triangle Inequality in Planet. Lab • In our Planet. Lab snapshot, 4. 4% of

Triangle Inequality in Planet. Lab • In our Planet. Lab snapshot, 4. 4% of all triples i, j, k violate the triangle inequality • Consider a looser version of TI, e. g. di, j ≤ α ( di, k + dk, j ) + β • There are fewer than 1% violations if s α = 1. 15, β = 1 ms s α = 1. 09, β = 5 ms

Inference Experiments • Metric: mean range of inference matrix • Compare measurement orderings s

Inference Experiments • Metric: mean range of inference matrix • Compare measurement orderings s Random s Smallest Lower Bound First (greedy) s Largest Range First (greedy)

Inference Results Random order performs poorly Largest range first performs best

Inference Results Random order performs poorly Largest range first performs best

Future Work • Build a full system s s More interesting constraints Better search

Future Work • Build a full system s s More interesting constraints Better search and pruning Synergistic search and measurement Integration with simulation/emulation tools • Other questions s What do search findings say about Planet. Lab? s Can we plan deployment?