NonBinary Constraint Satisfaction Toby Walsh Cork Constraint Computation

Non-Binary Constraint Satisfaction Toby Walsh Cork Constraint Computation Center

Overview • Introduction to CSPs – Local Consistencies, search algorithms • Modelling (4 case studies) – Auxiliary variables, implied constraints – Redundant models and channelling constraints • Eliminating non-binary constraints – Encodings, decomposition • Theoretical properties – Tightness, relational consistencies

Resources • Course links – www. cs. york. ac. uk/~tw/Links/csps/ • Benchmark problems – www. csplib. org • Constraints solvers – LP based like ECLIPSE, Java based liked Jsolver, open source like Choco, …

Constraint programming • Ongoing “dream” of declarative programming – State the constraints – Solver finds a solution • Paradigm of choice for many hard combinatorial problems – Scheduling, assignment, routing, …

Constraints are everywhere! • No meetings before 10 am • Network traffic < 100 Gbytes/sec • PCB width < 21 cm • Salary > 45 k Euros …

Constraint satisfaction • Constraint satisfaction problem (CSP) is a triple <V, D, C> where: – V is set of variables – Each X in V has set of values, D_X • Usually assume finite domain • {true, false}, {red, blue, green}, [0, 10], … – C is set of constraints Goal: find assignment of values to variables to satisfy all the constraints

Example CSP • Course timetabling – Variable for each course X 1, X 2. . – Domain are possible times for course Wed 9 am, Fri 10 am, . . – Constraints: X 1 = Wed 9 am Capacity constraints: atmost(3, [X 1, X 2. . ], Wed 9 am) Lecturer constraints: alldifferent([X 1, X 5, …])

Constraint optimization • CSP + objective function – E. g. objective is Profit = Income - Costs • Find assignment of vals to vars that: – Satisfies constraints – Maximizes (minimizes) objective • Often solved as sequence of satisfaction problems Profit > 0, Profit > Ans 1, Profit > Ans 2, …

Constraint programming v. Constraint logic programming • Constraints declaratively specify problem – Logic programming natural approach Assert constraints, call “labelling” strategy (backtracking search predicate) • But can also define constraint satisfaction or optimization within an imperative of functional language • Popular toolkits in C++, Java, CAML, …

Constraints • Constraints are tuples <S, R> where – S is the scope, [X 1, X 2, … Xm] • list of variables to which constraint applies – R is relation specifying allowed values (goods) • Subset of D_X 1 x D_X 2 x … x D_Xm • May be specified intensionally or extensionally

Constraints • Extensional specification – List of goods (or for tight constraints, nogoods) • Intensional specification – X 1 =/= X 2 – 5*X 1 + 6*X 2 < X 3 – alldifferent([X 1, X 2, X 3, X 4]), …

Constraint tightness • Informally, tight constraints admit few tuples – E. g. on 0/1 vars X 1 =/= X 2 is loose as half tuples satisfy X 1+X 2+X 3+X 4+X 5 <= 1 is tight as only 5 out of 32 tuples satisfy • More formal definition later

Binary v non-binary • Binary constraint – Scope covers 2 variables – E. g. not-equals constraint: X 1 =/= X 2. – E. g. ordering constraint: X 1 < X 2 • Non-binary constraint – Scope covers 3 or more variables – E. g. alldifferent(X 1, X 2, X 3). – E. g. tour(X 1, X 2, X 3, X 4). “Non-binary constraints” usually do not include unary constraints!

Constraint graph • Nodes = variables • Edge between 2 nodes iff constraint between 2 associated variables – Few constraints, sparse constraint graph – Lots of constraints, dense constraint graph

Some non-binary examples • Timetabling – Variables: Lecture 1, Lecture 2, … – Values: time 1, time 2, … – Constraint that lectures taught by same lecturer do not conflict: alldifferent(Lecture 1, Lecture 5, …).

Some non-binary examples • Scheduling – Variables: Job 1. Job 2, … – Values: machine 1, machine 2, … – Constraint on number of jobs on each machine: atmost(2, [Job 1, Job 2, …], machine 1), atmost(1, [Job 1, Job 2, …], machine 2).

Why use non-binary constraints? • Binary constraints are NP-complete – Any non-binary constraint can be represented using binary constraints – E. g. alldifferent(X 1, X 2, X 3) is “equivalent” to X 1 =/= X 2, X 1 =/= X 3, X 2 =/= X 3 • In theory therefore they’re not needed – But in practice, they are!

Modelling with non-binary constraints • Benefits include: – Compact, declarative specifications (discussed next) – Efficient constraint propagation (discussed second)

Modelling with non-binary constraints Consider writing your own alldifferent constraint: alldifferent([]). alldifferent([Head|Tail]): onediff(Head, Tail), alldifferent(Tail). onediff(El, []). onediff(El, [Head|Tail]): El #= Head, onediff(El, Tail).

Modelling with non-binary constraints • It’s possible but it’s not very pleasant! • Nor is it very compact – alldifferent([X 1, …Xn]) expands into n(n-1)/2 binary not-equals constraints, Xi #= Xj – one non-binary constraint or O(n^2) binary constraints? And there exist very efficient algorithms for reasoning efficiently with many specialized non-binary constraints

Constraint solvers • Two main approaches – Systematic, tree search algorithms – Local search or repair based procedures • Other more exotic possibilities – Hybrid algorithms – Quantum algorithms

Systematic solvers • Tree search – Assign value to variable – Deduce values that must be removed from future/unassigned variables • Propagation to ensure some level of consistency – If future variable has no values, backtrack else repeat • Number of choices – Variable to assign next, value to assign Some important refinements like nogood learning, nonchronological backtracking, …

Local search • Repair based methods – Generate complete assignment – Change value to some variable in a violated constraint • Number of choices – Violated constraint, variable within it, … Unable to exploit powerful constraint propagation techniques

Constraint propagation • Arc-consistency (AC) – A binary constraint r(X 1, X 2) is AC iff for every value for X 1, there is a consistent value (often called support) for X 2 and vice versa – A problem is AC iff every constraint is AC

Enforcing arc-consistency • Remove all values that are not AC (i. e. have no support) • May remove support from other values (often queue based algorithm) • Best AC algorithms (AC 7, AC-2000) run in O(ed^2) – Optimal if we know nothing else about the constraints

Enforcing arc-consistency • X 2 = X 3 is AC • X 1 = X 2 is not AC – X 2=1 has no support so can this value can be pruned • X 2 = X 3 is now not AC – No support for X 3=2 – This value can also be pruned Problem is now AC {1} X 1 = {1, 2} X 2 = {2, 3} X 3

Properties of AC • Unique maximal AC subproblem – Or problem is unsatisfiable • Enforcing AC can process constraints in any order – But order does affect (average-case) efficiency

Non-binary constraint propagation • Most popular is generalized arc-consistency (GAC) – A non-binary constraint is GAC iff for every value for a variable there are consistent values for all other variables in the constraint – We can again prune values that are not supported • GAC = AC on binary constraints

GAC on alldifferent • alldifferent(X 1, X 2, X 3) – Constraint is not GAC – X 1=2 cannot be extended X 1 {1, 2} • X 2 would have to be 3 • No value left then for X 3 – X 1={1} is GAC {2, 3} X 2 X 3

Enforcing GAC • Enforcing GAC is expensive in general – GAC schema is O(d^k) On k-ary constraint on vars with domains of size d • Trick is to exploit semantics of constraints – Regin’s all-different algorithm – Achieves GAC in just O(k^2 d^2) On k-ary all different constraint with domains of size d Based on finding matching in “value graph”

Other types of constraint propagation • (i, j)-consistency [due to Freuder, JACM 85] – Non-empty domains – Any consistent instantiation for i variables can be extended to j others • Describes many different consistency techniques

(i, j)-consistency • Generalization of arc-consistency – AC = (1, 1)-consistency – Path-consistency = (2, 1)-consistency • Strong path-consistency = AC + PC – Path inverse consistency = (1, 2)-consistency

Enforcing (i, j)-consistency • problem is (1, 1)-consistent (AC) • BUT is not (2, 1)-consistent (PC) – X 1=2, X 2=3 cannot be extended to X 3 – Need to add constraints: not(X 1=2 & X 2=3) not(X 1=2 & X 3=3) • Nor is it (1, 2)-consistent (PIC) – X 1=2 cannot be extended to X 2 & X 3 (so needs to be deleted) {1, 2} X 1 = {2, 3} X 2 = = {2, 3} X 3

Other types of constraint propagation • Singleton arc-consistency (SAC) – Problem resulting from instantiating any variable can be made AC • Restricted path-consistency (RPC) – AC + if a value has just one support then any third variable has a consistent value

Other types of consistency • problem is (1, 1)-consistent (AC) • BUT is not singleton AC (SAC) – Problem with X 1=2 cannot be made AC (so value should be deleted) • Nor is it restricted PC (RPC) – X 1=2 has only one support in X 2 (the value 3) but X 3 then has no consistent values – X 1=2 can therefore be deleted {1, 2} X 1 = {2, 3} X 2 = = {2, 3} X 3

Other types of constraint propagation • Neighbourhood inverse consistency (NIC) – For all vals for a var, there are consistent vals for all vars in the immediate neighbourhood • Bounds consistency (BC) – With ordered domains – Enforce AC just on max/min elements

Comparing local consistencies • Formal definition of tightness introduced by Debruyne & Bessiere [IJCAI-97] • A-consistency is tighter than B-consistency iff If a problem is A-consistent -> it is B-consistent We write A >= B

Properties • Partial ordering – reflexive – transitive A A A B & B C implies A C • Defined relations – tighter A > B iff A B & not B A – incomparable A @ B iff neither A B nor B A

Comparison of consistency techniques • Exercise for the reader, prove the following identities! Strong PC > SAC > PIC > RPC > AC > BC NIC > PIC NIC @ SAC NIC @ Strong PC NB gaps can reduce search exponentially!

Which to choose? • For binary constraints, AC is often chosen – Space efficient Just prune domains (cf PC) – Time efficient • For non-binary constraints GAC is often chosen – If we can exploit the constraint semantics to keep it cheap!

Why consider these other consistencies? • Promising experimental results – Useful pruning for their additional cost • Theoretical value – E. g. GAC on non-binary constraints may exceed SAC on equivalent binary model

Maintaining a local consistency property • Tree search – Assign value to variable – Enforce some level of local consistency • Remove values/add new constraints – If any future variable has no values, backtrack else repeat • Two popular algorithms – Maintaining arc-consistency (MAC) – Forward checking (very restricted form of AC maintained)

Forward checking • Binary constraints (FC) – Make constraints involving current variable and one future variable arc-consistent – No need to look at any other constraints! • Non-binary constraints – Several choices as to how to do forward checking

Forward checking with non -binary constraints n. FC 0 makes AC only those k-ary constraints with k-1 variables set n. FC 1 applies one pass of AC on constraints and projections involving current var and one future var n. FC 2 applies one pass of GAC on constraints involving current var and at least one future var n. FC 3 enforces GAC on this set n. FC 4 applies one pass of GAC on constraints involving at least one past and one future var n. FC 5 enforces GAC on this set

Summary • Constraint solving – Constraint propagation central part of many algorithms • Binary v non-binary constraints – Compact, more efficient representation