Algorithmic and Domain Centralization in Distributed Constraint Optimization

Algorithmic and Domain Centralization in Distributed Constraint Optimization Problems John P. Davin Carnegie Mellon University June 27, 2005 Committee: Manuela Veloso, Co-Chair Pragnesh Jay Modi, Co-Chair Scott Fahlman Stephen F. Smith Carnegie Mellon University School of Computer Science In partial fulfillment of the 5 th year master's degree. Full paper: http: //www. cs. cmu. edu/~jdavin/thesis/

DCOP • DCOP - Distributed Constraint Optimization Problem – Provides a model for many multi-agent optimization problems (scheduling, sensor nets, military planning). – More expressive than Distributed Constraint Satisfaction. – Computationally challenging (NP Complete).

Centralization in DCOPs • Centralization = aggregating information about the problem in a single agent. resulting in a larger local search space. • We define two types of centralization: – Algorithmic – a DCOP algorithm actively centralizes parts of the problem through communication. allows a centralized search procedure. – Domain – the DCOP definition inherently has parts of the problem already centralized. eg. , multiple variables per agent (ex. scheduling)

Motivation • Current state of DCOP research: – Two existing DCOP algorithms – Adopt + Opt. APO – exhibit differing levels of centralization. – DCOP has been applied to several domains (eg, meeting scheduling). – Only 1 variable per agent problems used in existing work. • Still Needed: – It is unclear exactly how Adopt + Opt. APO are affected by centralization. – Domains with multiple variables per agent have not been explored

Thesis Statement • Questions: – How does algorithmic centralization affect performance? – How can we take advantage of domain centralization?

Spectrum of centralization

Outline • Introduction • Part 1: Algorithmic centralization in DCOPs • Part 2: Domain centralization in DCOPs • Conclusions

Part 1: Algorithmic Centralization in DCOP Algorithms

Evaluation Metrics • How does algorithmic centralization affect performance? • How do we measure performance?

Evaluation Metrics: Cycles • Cycle = one unit of algorithm progress in which all agents process incoming messages, perform computation, and send outgoing messages. – Independent of machine speed, network conditions, etc. – Used in prior work [Yokoo et al. , Mailler et al. ]

Evaluation Metrics: Constraint Checks • Constraint check = the act of evaluating a constraint between N variables. – Provides a measure of computation. • Concurrent constraint checks (CCC) = maximum constraint checks from the agents during a cycle.

Problems with previous measures • Cycles do not measure computational time (they don’t reflect the length of a cycle). • Constraint checks alone do not measure communication overhead. We need a metric that combines both of the previously used metrics.

CBR: Cycle-Based Runtime 1. The length of a cycle is determined by communication and computation: 2. (t = time for one constraint check. ) 3. 4. 5. Define ccc(m) as the total constraint checks: Cycle-Based Runtime:

CBR parameters • • LL== communication latency (time to communicate in each cycle). can be parameterized based on the system environment. Eg. , L=1, 100, 1000. • We assume t=1, since constraint checks are usually faster than communication time. L is defined in terms of t (L=10 indicates communication is 10 times slower than a constraint check).

Comparing Adopt and Opt. APO • Algorithmic centralization: – Adopt is non-centralized. – Opt. APO is partially centralized. Prior work [Mailler & Lesser] has shown that Opt. APO completes in fewer cycles than Adopt for graph coloring problems at density 2 n and 3 n. But how do they compare when we use CBR to measure both communication time and computation?

DCOP Algorithms: Adopt • ADOPT, [Jay Modi, et al. ] – Variables are ordered in a priority tree (or chain). x 1 – Agents pass their current value down the tree to neighboring agents using VALUE messages. – Agents send COST messages up to their parents. Cost messages inform the parent of the lower and upper bounds at the subtree. – These costs are dependent on the values of the agent’s ancestor variables. x 2 x 3 x 4 VALUE messages COST messages Credit: Jay Modi

DCOP Algorithms: Opt. APO – Optimal Asynchronous Partial Overlay. [Mailler and Lesser] • • • cooperative mediation – an agent is dynamically appointed as mediator, and collects constraints for a subset of the problem. Opt. APO agents attempt to minimize the number of constraints that are centralized. The mediator solves its subproblem using Branch & Bound centralized search [Freuder and Wallace]. x 1 x 2 x 5 mediator x 3 x 4 Values + constraints

Results [Davin+Modi, ’ 05] • Tested on graph coloring problems, |D|=3 (3 -coloring). • # Variables = 8, 12, 16, 20, with link density = 2 n or 3 n. • 50 randomly generated problems for each size. Cycles: CCC: Opt. APO takes fewer cycles, but more constraint checks.

How do Adopt and Opt. APO compare using CBR? Density 2

How do Adopt and Opt. APO compare using CBR? Density 3 For L values < 1000, Adopt has a lower CBR than Opt. APO’s high number of constraint checks outweigh its lower number of cycles.

How much centralization occurs in Opt. APO? Opt. APO sometimes centralizes all of the problem structure.

How does the distribution of computation differ? • We measure the distribution of computation during a cycle as: This is the ratio of the maximum computing agent to the total computation during a cycle. – A value of 1. 0 indicates one agent did all the computation. – Lower values indicate more evenly distributed load.

How does the distribution of computation differ? • Load was measured during the execution of one representative graph coloring problem with 8 variables, density 2: Opt. APO has varying load, because one agent (the mediator) does all of the search within each cycle. Adopt’s load is evenly balanced.

Communication Tradeoffs of Centralization • How does centralization affect performance under a range of communication latencies? Adopt has the lowest CBR at L=1, 100, and crosses over between L=100 and 1000. • Opt. APO outperforms Branch & Bound at density 2 but not at density 3.

Part 2: Domain Centralization in DCOPs

How can we take advantage of domain centralization? • Adopt treats variables within an agent as independent pseudo-agents. – Does not take advantage of partially centralized domains. x 1 agent 2 agent 1 x 3 x 2 x 4

How can we take advantage of domain centralization? • Instead, we could use centralized search to optimize the agent’s local variables. We call this Adopt. MVA, for Multiple Variables per Agent. – Potentially more efficient • Adopt’s variable ordering heuristics do not apply to agent orderings – Need agent ordering heuristics for Adopt. MVA. – Also need intra-agent heuristics for the local search. agent 2 x 1 x 4 agent 1 x 2 x 3

Extending Adopt to Adopt. MVA • In Adopt, a context is a partial solution of the form {(xj, dj), (xk, dk)…}. • We define a context where S is the set of all possible assignments to an agent’s local variables. • We must modify Adopt’s cost function to include the cost of constraints between variables in s: Intra-agent cost • Inter-agent cost Use Branch & Bound search to find the minimizes the local cost. that

Results • Setup: Randomly generated meeting scheduling problems: – Based on an 8 -hour day (|D| = 8). – Number of attendees (A) per meeting is randomly chosen from a geometric progression. – All meeting scheduling problems generated were fully schedulable. • We compared using a lexicographic agent ordering for both algorithms.

How does Adopt. MVA perform vs Adopt? High density meeting scheduling (4 meetings per agent): CBR: Cycles: [20 problems per datapoint. ]

How does Adopt. MVA perform vs Adopt? Graph Coloring with 4 variables per agent, link density 2: Cycles: CBR: Adopt. MVA uses fewer cycles than Adopt, and has a lower CBR at L=1000.

Adopt and Adopt. MVA Variable ordering Original Problem Adopt hierarchy Adopt. MVA hierarchy –Adopt. MVA’s order has a reduced granularity from Adopt’s order. Adopt can interleave the variables of an agent, while Adopt. MVA can only order agents.

Inter-agent ordering heuristics • We tested several heuristics for ordering the agents: – Lexicographic – Inter-agents links – order by # of links to other agents. – Adopt. To. MVA-Max – compute the Brelaz ordering over variables, and convert it to an agent ordering using the maximum priority variable within each agent. – Adopt. To. MVA-Min – Adopt. To. MVA-Max but using the minimum priority variables.

Comparison of agent orderings • Intra-agent ordering = MVA-Higher. Vars Low Density meeting scheduling Graph Coloring with 4 variables per agent High Density meeting scheduling

Comparison of agent orderings • Intra-agent ordering = MVA-Higher. Vars Low Density meeting scheduling High Density meeting scheduling Ordering makes an order of magnitude difference in some cases. Adopt. To. MVA-Min was the best on 8 out of 11 cases, but with high variance.

Intra-agent Branch & Bound Heuristics • Best-first Value ordering heuristic: we put the best domain value first in the value ordering. • Variable ordering heuristics: – Lexicographic – Random – Brelaz [Graph Coloring only] – order by number of links to other variables within the agent. – MVA-All. Vars – order by # of links to external variables. – MVA-Lower. Vars – MVA-All. Vars but only consider lower priority variables. – MVA-Higher. Vars – MVA-All. Vars but only consider higher priority variables.

Comparison of intra-agent search heuristics • Goal: Reduce constraint checks used by Branch & Bound. • Metric: Average CCC per Cycle (Total. CCC / Total Cycles). High density Meeting Scheduling Cycles Avg CCC Nearly all differences are statistically significant. Excepting MVA-All. Vars vs MVA-Higher. Vars is the most computationally efficient heuristic. Low density Meeting Scheduling produced similar results.

Comparison of intra-agent search heuristics Graph Coloring Cycles Avg CCC Confirms Brelaz is the most efficient heuristic for graph coloring.

How does Meeting Scheduling scale as agents and meeting size are increased? Original Data Outliers removed (A = avg # of attendees (meeting size)) • Original data had several outliers (two standard deviations away from the mean) so they were removed for easier interpretation. Meeting size has a large effect on solution difficulty.

Thesis contributions • Formalization of algorithmic vs. domain centralization. • Empirical comparison of Adopt + Opt. APO showing new results. • CBR - a performance metric which accounts for both communication + computation in DCOP algorithms. • Adopt. MVA - a DCOP algorithm which takes advantage of domain centralization. We also contribute an efficient intra-agent search heuristic for meeting scheduling. • Empirical analysis of the meeting scheduling problem. • [Impact of Problem Centralization in DCO Algorithms, AAMAS ’ 05, Davin + Modi].

Future Work • Improve Adopt. MVA: – Agent ordering heuristics – is there a heuristic which will work better than the ones tested? – Intra-agent search heuristics – develop a heuristic that is both informed and randomly varied. • Test DCOP algorithms in a fully distributed testbed.

The End My future plans: AAMAS in July and MSN Search (Microsoft) in the Fall. Contact: jdavin@cmu. edu