MultiAgent Based Search vs Local Search and Backtrack

Multi-Agent Based Search vs. Local Search and Backtrack Search for Solving Tight CSPs: A Practical Case Study Hui Zou and Berthe Y. Choueiry Constraint Systems Laboratory Department of Computer Science and Engineering University of Nebraska-Lincoln {hzou|choueiry}@cse. unl. edu

Introduction Search algorithms: systematic or local-repair Complex, real-world optimization problems – Systematic search thrashes – Local search gets stuck in ‘local optima’ – Remedial: random walk, breakout, restart strategies, etc. Multi-agent-based search [Liu & al. AIJ 02] – provides us with a new way – Advantages & shortcomings via a practical application

Background - GTA Graduate Teaching Assistants (GTA) problem: In a semester, given – a set of courses – a set of graduate teaching assistants – a set of constraints that specify allowable assignments Find a consistent and satisfactory assignment of GTAs to courses Types of constraints: unary, binary, non-binary – Each course has a load, indicates weight of the course – Each GTA has a (hiring) capacity, limits max. load Detailed modeling in [Glaubius & Choueiry ECAI 02 WS on Modeling]

Background - GTA (cont’) ü In practice, this problem is tight, even over-constrained ü Our goal: ensure GTA support to as many courses as possible Problem size: Date set Spring 2001 b Fall 2002 Spring 2003 Mark # variables Domain size Problem size B 69 35 3. 5× 10106 O 69 26 4. 3× 1097 B 65 35 2. 3× 10100 O 65 34 3. 5× 1099 B 31 33 1. 2× 1047 O 31 28 7. 7× 1044 B 54 36 1. 1× 1084 O 54 34 5. 0× 1082 B – boosted to make problem solvable O – original, not necessary solvable

Background - GTA (cont’) Optimization criteria: 1. Maximize the number of courses covered 2. Maximize the geometric average of the assignments wrt the GTAs’ preference values (between 0 and 5). Problem: – Constraints are hard, must be met – Maximal consistent partial-assignment problem (MPA-CSP? ) – Not a MAX-CSP (which maximizes #constraints satisfied)

Background - MAS for CSPs Multi-Agent System: agents interact & cooperate in order to achieve a set of goals – Agents: autonomous (perceive & act), goal-directed, can communicate – Interaction protocols: governing communications among agents – Environment: where agents live & act ERA [Liu & al. AIJ 2002] – Environment, Reactive rules, and Agents – A multi-agent approach to solving a general CSP – Transitions between states when agents move

Background - ERA’s components ERA=Environment + Reactive rules + Agents Environment: a n×m two-dimensional array – – – n: the number of variables (agents) m: the maximum domain size, |Dmax| e(i, j). value: domain value of agent i at position j e(i, j). violation: violation value of agent i at position j Zero position: where e(i, j). violation=0 When all agents are in zero position, we have a complete solution Example:

Background - ERA’s components ERA=Environment + Reactive rules + Agents Reactive rules: – Least-move: choose a position with the min. violation value – Better-move: choose a position with a smaller violation value – Random-move: randomly choose a position Combinations of these basic rules form different behaviors.

Background - ERA’s components ERA=Environment + Reactive rules + Agents: a variable is represented by an agent At each state, an agent chooses a position to move to, following the reactive rules. The agents keep moving until all have reached zero position, or a certain time period has elapsed. All agents in zero position Some agents in zero position Assignments are made only for agents in zero position

Background - ERA vs local search ERA operates by local repairs, how different is it from local search? ERA – Each agent has an evaluation function – At each state, any agent moves wherever it desires to move Control is localized: Each agent is in pursuit of its own happiness Local search with min-conflict – One evaluation function for the whole state (cost), summarizes the quality of the state – At each state, few agents are allowed to move (most unhappy ones) Control is centralized: towards one common good

Background - Example ( ERA ) 4 -queen problem 2 2 0 1 Init 2 1 2 Eval (agent Q 1) 1 Move (agent Q 1) 2 2 Eval (agent Q 2) 1 1 Eval (agent Q 3) 3 Move (agent Q 3) 0 3 Eval (agent Q 4) 1 0 Move(agent 4)

Background - Example (ERA vs. Local search) ERA – any agent can kick any other agent from its position Local search with min-conflict – cannot repair a variable without violating a previously repaired variable

Empirical study - In general Apply ERA on GTA assignment problem: 0. (Test & understand the behavior of ERA) 1. Compare performance of: – ERA: Fr. BLR – LS: hill-climbing, min-conflict & random walk – BT: B&B-like, many orderings (heuristic, random) 2. Observe behavior of ERA on solvable vs. unsolvable problems 3. Observe behavior of individual agents in ERA 4. Identify a limitation of ERA: deadlock phenomenon 8 instances of the GTA assignment problem

Empirical study 1 - Performance comparison Unused GTAs CC (× 108) Unassigned Courses Solution Quality Unused GTAs Available Resource CC (× 108) Unassigned Courses 35 29. 6 1. 18 6 4. 05 2 6. 5 3. 77 5 3. 69 0 6. 4 0. 87 0 3. 20 0 5. 3 0. 18 O × 26 69 26 29. 6 0. 88 16 3. 79 0 2. 5 4. 09 13 3. 54 0 0. 9 0. 39 24 2. 55 8 8. 3 7. 39 B √ 35 65 31 29. 3 1. 06 2 3. 12 0 2. 5 1. 71 4 3. 01 0 3. 8 0. 33 0 3. 18 1 1. 9 2. 68 O √ 34 65 30 29. 3 1. 02 2 3. 12 0 1. 5 2. 46 4 3. 04 1 3. 7 0. 10 0 3. 27 0 0. 8 1. 15 B √ 33 31 16. 5 13 1. 27 1 3. 93 0 3. 5 2. 39 2 3. 40 0 5. 0 0. 85 0 3. 62 2 3. 0 0. 02 O × 28 31 11. 5 13 0. 88 4 3. 58 0 1. 8 2. 56 4 3. 61 0 2. 0 0. 16 8 3. 22 1 2. 0 0. 51 B √ 36 54 29. 5 27. 4 1. 08 3 4. 49 2 4. 2 1. 17 3 3. 62 0 3. 9 0. 32 0 3. 03 1 2. 8 0. 49 O √ 34 54 27. 5 27. 4 1. 00 3 4. 45 0 2. 2 1. 53 4 3. 63 0 3. 3 1. 42 0 3. 26 0 0. 8 0. 14 CC (× 108) Solution Quality 69 Available Resource Unassigned Courses 35 Unused GTAs Ratio= C L √ Solution Quality Total load (L ) B Available Resource Total capacity (C) Spring 2003 # Courses Fall 2002 # GTAs Fall 2001 b Multi-agent Search (ERA) Local Search (LS) Solvable? Spring 2001 b Systematic Search (BT) Original/Boosted Date set - Only ERA finds complete solutions to all solvable instances Observations: - On unsolvable problems, ERA leaves too many unused GTAs - LS and BT exhibit similar behaviors

Empirical study 2 - Solvable vs unsolvable Observation: ERA performance on solvable problems - Number of agents in zeroposition per iteration - ERA behavior differs on solvable vs. unsolvable instances ERA performance on unsolvable problems

Empirical study 3 - Behavior of individual agents Instances • solvable • unsolvable Motion of agents • variable • stable • constant Observations: Solvable Unsolvable Variable None Most Stable A few Constant Most None

Empirical study 4 - Deadlock Observation: ERA is not able to avoid deadlocks and yields a degradation of the solution on unsolvable CSPs. – – Each circle corresponds to a given GTA Each square represents an agent A blank squares indicate that an agent is on a zero-position The squares with same color indicate agents involved in a deadlock

Discussion Goal ERA Control Schema Undoing assignments Conflict resolution Local √ Non-committal + Escape local optima – May yield instability Global LS + Stable behavior – Liable to local optima Systematic BT Actions + Stable behavior – Thrashes + Flexible + Solves tight CSPs – Deadlock – Shorter solutions × + Quickly stabilizes – Fails to solve tight CSPs even with randomness & restart strategies ~ Heuristic + Longer solutions – Problem-dependent + Quickly stabilizes – Fails to solve tight CSPS even with backtracking & restart strategies + advantages – shortcomings

Dealing with the deadlock Possible approaches: — Direct communications, negotiation mechanisms — Hybrids of search ü Global control ü Conflict resolution Experiments: — Enhancing ERA with global control – – — Don’t accept a move that deteriorates the global goal Lead to local-search-like behavior (i. e. , local optima) ERA with conflict resolution – – – add dummy resources find a complete solution when LS and BT fail remove dummy assignments, solutions are still better

Future research directions – Test approach using other search techniques – – Validate conclusions on other CSPs – – BT search: Randomized, credit-based Other local repair: squeaky-wheel method Market-based techniques, etc. random instances, real-world problems Try search-hybridization techniques References: R. Glaubius and B. Y. Choueiry, Constraint Modeling and Reformulation in the Context of Academic Task Assignment. In Workshop Modeling and Solving Problems with Constraints, ECAI 2002. J. Liu, H. Jing, and Y. Y. Tang. Multi-Agent Oriented Constraint Satisfaction. Artificial Intelligence, 136: 101144, 2002.

Questions