The story of distributed constraint optimization in LA

The story of distributed constraint optimization in LA: Relaxed Master’s Thesis Steven Okamoto Computer Science Department University of Southern California December 8, 2003

Motivation: Role allocation in multiagent teams n n Large teams of agents, robots, people important in many domains Key challenge: Allocate large number of roles to agents q q q Dynamic environment (roles change) Heterogeneous roles (agents have differing capabilities to perform roles) Role constraints (some roles require other roles to be performed)

The Problem n n Distributed constraint-based reasoning important for role allocation Key weaknesses of existing distributed constraintbased role allocation algorithms: q Time: must run to completion q Communication q Dynamism: run again, many role changes q Simplified agents and roles: n Only perform one task n No interactions between roles

LA-DCOP n n Low communication, approximate DCOP Key Ideas: q q q n Key results q q n Thresholds: global probabilistic information, improve output Tokens: reduces communication, highly distributed Potential tokens: AND-constrained roles Empirical evaluation using abstract simulator: higher output, lower communication than competition Calculation of best threshold in simple cases Key limitations q Inappropriate for domains requiring strict optimality

Outline n Generalized Assignment Problem q n Extended Generalized Assignment Problem q n n Natural representation of role allocation problem Extends GAP to allow dynamism and constraints among roles Distributed Constraint Optimization Problem LA-DCOP algorithm Theoretical calculation of thresholds Empirical results

Generalized Assignment Problem - GAP n n n Known to be NP-complete Example: Set of agents E, set of roles R E = {e 1, e 2} Capability: ability to perform roles R = {r 1, r 2, r 3, r 4} Cap(ei, rj) →[0, 1] A= Resources: each agent has 1. 0 resources Resources(ei, rj) →[0, 1] e 1 performs r 1 Allocation matrix A = (ai, j) e 2 performs r 2 ai, j = 1 if ei is performing rj and r 3 0 otherwise no one performs r 4

GAP – The Goal reward for all roles performed by ei Agents do not exceed resources total reward over all agents n Goal is to find A that maximizes total reward: No role is performed by more than one agent n While respecting constraints:

Extended GAP – E-GAP n n Dynamism: E, R, Cap, Resources indexed by time Coordination Constraints: # q AND – all roles in a set must be simultaneously performed n n Ex. Big fire requires multiple firefighting roles Delay Cost number 1 of ifallrconstrained is being performed sum=for roles mconstrained roles being performed number of constrained roles n Find sequence A of allocations

E-GAP – The Goal n n Reward for performed Penalty for roles unperformed roles Find A that maximizes Roles are never performed by Agents never exceed more than one agent at a time resources While respecting constraints

Distributed Constraint Optimization Problem - DCOP n Set of variables V: agents q n n Can have multiple values Values: roles Objective function: total reward Cost functions: reward, delay cost, resources, single agent allocation Complete graph

LA-DCOP n Low communication, approximate DCOP q Create a token for each role n q q q Token required to perform role Distribute tokens to agents Agents pass tokens around Keep token: 2 levels 1. Thresholds: DCSP 2. Kickout: locally optimal q Potential tokens: AND constraints

LA-DCOP - Example Threshold = 0. 7 Only one role at a time A: 0. 8 B: 0. 9 A: 0. 2 B: 0. 8 Kickout Threshold A A B A: 0. 9 B: 0. 4 A: 0. 8 B: 0. 6

LA-DCOP - Retainers n AND constrained: deadlock, starvation q n Potential tokens q q n Inefficient for agents to wait for coalition Promise to perform role when coalition formed: agent is “retained” Agent can do other roles in the meantime Algorithm: q Give real tokens to “owner” n n n Distributes potential tokens Monitors for coalition formation “Locks” coalition by sending out real tokens

Agent algorithm Check threshold Enter retainer for potential token Add real token Compute locally optimal allocation Keep only retained tasks that would perform Retainer lock

AND Owner Algorithm Run by the owner of AND-constrained set of roles Distribute tokens Wait for at least one token for each task Choose best agents from among retainers

Picking Thresholds – Steady State n n Steady state – output stops changing Simple case: q q q n M roles N agents K types of roles, chosen randomly Static environment Independent, uniform(0, 1) capabilities Roles require 0. 25, 0. 5, or 0. 75 resources EU(T) = E(#tasks executed|T) E(capability|T)

Picking thresholds Only increasing part of function Maximum at this value

How well does the prediction match? Predicted max

Comparison to other algorithms Communication Output High is good n n DSA: Distributed, approximate Greedy: Greedy auction with centralized auctioneer Low is good

10% of agents have non. Threshold and dynamics zero capability for each type

Retainers

Summary n Thresholds q q n Tokens q n n Increase output: higher output than competition Calculate offline from probabilistic information Reduce communication: orders of magnitude Potential tokens: AND constraints Trades off optimality for efficiency

Calculating E(#tasks executed) n n n Allocate roles using fewest resources first Z = #tasks requiring each resource amount = M/3 Nx = #agents required to execute all tasks requiring x resources q q q n N 0. 25 = Z / 4 N 0. 5 = Z / 2 N 0. 75 = Z Ncapable = E(#capable agents)

Calculating threshold n Capable agent has at least one capability > T EU(T) = E(#capable agents) E(#tasks per capable agent) E(capability of capable agent) = E(#tasks executed) E(capability of capable agent) Pr(all caps<T) n n E(#capable agents) = (1 – TK) N E(capability of capable agent) = (1 + T) / 2

Calculating E(#tasks executed) n n If Ncapable < N 0. 25 E(#tasks executed) = 4 * Ncapable Else if Ncapable < (N 0. 25 + N 0. 5) E(#tasks executed) = Z + 2 * (Ncapable – N 0. 25) Else if Ncapable < (Z / 4 + Z / 2 + Z) E(#tasks executed) = Z + Ncapable – (N 0. 25 + N 0. 5) = (5/4) * Z + Ncapable Else E(#tasks executed) = M