CS B 551 ELEMENTS OF ARTIFICIAL INTELLIGENCE 1

CS B 551: ELEMENTS OF ARTIFICIAL INTELLIGENCE 1 Instructor: Kris Hauser http: //cs. indiana. edu/~hauserk

CONSTRAINT PROPAGATION … … is the process of determining how the constraints and the possible values of one variable affect the possible values of other variables It is an important form of “least-commitment” reasoning 2 2

FORWARD CHECKING Whenever a pair (X v) is added to assignment A do: For each variable Y not in A do: For every constraint C relating Y to the variables in A do: Remove all values from Y’s domain that do not satisfy C § n = number of variables § d = size of initial domains § s = maximum number of constraints involving a given variable (s n-1) § Forward checking takes O(nsd) time 3

FORWARD CHECKING IN MAP COLORING Empty set: the current assignment {(WA R), (Q G), (V B)} does not lead to a solution WA NT Q NSW V SA T RGB RGB RGB RGB R GB G RGB GB R B G RB B B RGB 4 4

FORWARD CHECKING IN MAP COLORING NT WA Contradiction that forward checking did not detect Q T NSW SA V WA NT Q NSW V SA T RGB RGB RGB RGB R GB G RGB GB R B G RB B B RGB 5 5

FORWARD CHECKING IN MAP COLORING NT WA Contradiction that forward checking did not detect Q T NSW SA this contradiction Detecting requires a more V powerful constraint propagation technique WA NT Q NSW V SA T RGB RGB RGB RGB R GB G RGB GB R B G RB B B RGB 6 6

CONSTRAINT PROPAGATION FOR BINARY CONSTRAINTS REMOVE-VALUES(X, Y) 1. removed false 2. For every value v in the domain of Y do – If there is no value u in the domain of X such that the constraint on (X, Y) is satisfied then a. Remove v from Y‘s domain b. removed true 3. Return removed 7

CONSTRAINT PROPAGATION FOR BINARY CONSTRAINTS AC 3 1. Initialize queue Q with all variables (not yet instantiated) 2. While Q do a. X Remove(Q) b. For every (not yet instantiated) variable Y related to X by a (binary) constraint do – If REMOVE-VALUES(X, Y) then i. ii. If Y’s domain = then exit Insert(Y, Q) 8

EDGE LABELING We consider an image of a scene composed of polyhedral objects such that each vertex is the endpoint of exactly three edges 9 9

EDGE LABELING An “edge extractor” has accurately extracted all the visible edges in the image. The problem is to label each edge as convex (+), concave (-), or occluding ( ) such that the complete labeling is physically possible 10 10

Occluding edges Concave edges Convex edges 11 11

+ + + - + + The arrow is oriented such that the object is on the right of the occluding edge + 12 12

ONE POSSIBLE EDGE LABELING + + + - + + + 13 13

JUNCTION TYPES L Fork T Y 14 14

JUNCTION LABEL SETS + - + + - - - + + + 15 (Waltz, 1975; Mackworth, 1977) 15

EDGE LABELING AS A CSP A variable is associated with each junction The domain of a variable is the label set associated with the junction type Constraints: The values assigned to two adjacent junctions must give the same label to the joining edge 16 16

AC 3 APPLIED TO EDGE LABELING Q = (X 1, X 2, X 3, . . . ) X 5 X 3 X 12 X 8 X 1 X 2 X 4 17 17

AC 3 APPLIED TO EDGE LABELING Q = (X 1, . . . ) + + - -+ - + X 5 X 1 + - 18 18

COMPLEXITY ANALYSIS OF AC 3 n = number of variables d = size of initial domains s = maximum number of constraints involving a given variable (s n-1) Each variables is inserted in Q up to d times REMOVE-VALUES takes O(d 2) time AC 3 takes O(n d s d 2) = O(n s d 3) time Usually more expensive than forward checking AC 3 1. Initialize queue Q with all variables (not yet instantiated) 2. While Q do a. X Remove(Q) b. For every (not yet instantiated) variable Y related to X by a (binary) constraint do – If REMOVE-VALUES(X, Y) then i. If Y’s domain = then exit ii. Insert(Y, Q) REMOVE-VALUES(X, Y) 1. removed false 2. For every value v in the domain of Y do – If there is no value u in the domain of X such that the constraint on (X, Y) is satisfied then a. Remove v from Y‘s domain b. removed true 3. Return removed 28

IS AC 3 ALL THAT WE NEED? No !! AC 3 can’t detect all contradictions among binary constraints X Y {1, 2} Y Z X Z Z {1, 2} 29 29

IS AC 3 ALL THAT WE NEED? No !! AC 3 can’t detect all contradictions among binary constraints X Y {1, 2} REMOVE-VALUES(X, Y) X Z 1. removed false 2. For every value v in the domain of Y do – If there is no value u in the domain X Z of{1, such that the constraint on (X, Y) is satisfied then a. Remove v from Y‘s domain b. removed true 3. Return removed Y Z 2} 30 30

IS AC 3 ALL THAT WE NEED? No !! AC 3 can’t detect all contradictions among binary constraints X Y {1, 2} REMOVE-VALUES(X, Y, Z) removed false REMOVE-VALUES(X, Y)1. Y Z X Z 2. For every value w in the domain of Z do 1. removed false 2. For every value v in the domain of Y do – If there is no pair (u, v) of values in the domains of X – If there is no value u in the and domain of{1, X 2} YZverifying the constraint on (X, Y) such that the constraint on (X, Y) is constraints on (X, Z) and (Y, Z) are satisfied then a. Remove v from Y‘s domain a. Remove w from Z‘s domain b. removed true 31 3. Return removed 31

IS AC 3 ALL THAT WE NEED? No !! AC 3 can’t detect all contradictions among binary constraints {1, 2} X X Y {1, 2} Y Z X Z Z Y {1, 2} Not all constraints are binary 32 32

TRADEOFF Generalizing the constraint propagation algorithm increases its time complexity Ø Tradeoff between time spent in backtracking search and time spent in constraint propagation A good tradeoff when all or most constraints are binary is often to combine backtracking with forward checking and/or AC 3 (with REMOVE-VALUES for two variables) 33 33

MODIFIED BACKTRACKING ALGORITHM WITH AC 3 CSP-BACKTRACKING(A, var-domains) 1. 2. 3. 4. 5. 6. If assignment A is complete then return A Run AC 3 and update var-domains accordingly If a variable has an empty domain then return failure X select a variable not in A D select an ordering on the domain of X For each value v in D do a. Add (X v) to A b. var-domains forward checking(var-domains, X, v, A) c. If no variable has an empty domain then (i) result CSP-BACKTRACKING(A, var-domains) (ii) If result failure then return result d. Remove (X v) from A 7. Return failure 34

A COMPLETE EXAMPLE: 4 -QUEENS PROBLEM 1 2 3 4 X 1 {1, 2, 3, 4} X 2 {1, 2, 3, 4} X 3 {1, 2, 3, 4} X 4 {1, 2, 3, 4} 1 2 3 4 1) The modified backtracking algorithm starts by calling AC 3, which removes no value 35 35

4 -QUEENS PROBLEM 1 2 3 4 X 1 {1, 2, 3, 4} X 2 {1, 2, 3, 4} X 3 {1, 2, 3, 4} X 4 {1, 2, 3, 4} 1 2 3 4 2) The backtracking algorithm then selects a variable and a 36 value for this variable. No heuristic helps in this selection. 36 X 1 and the value 1 are arbitrarily selected

4 -QUEENS PROBLEM 1 2 3 4 X 1 {1, 2, 3, 4} X 2 {1, 2, 3, 4} X 3 {1, 2, 3, 4} X 4 {1, 2, 3, 4} 1 2 3 4 3) The algorithm performs forward checking, which eliminates 2 values in each other variable’s domain 37

4 -QUEENS PROBLEM 1 2 3 4 X 1 {1, 2, 3, 4} X 2 {1, 2, 3, 4} X 3 {1, 2, 3, 4} X 4 {1, 2, 3, 4} 1 2 3 4 4) The algorithm calls AC 3 38 38

REMOVE-VALUES(X, Y) 1. removed false 2. For every value v in the domain of Y do – If there is no value u in the domain of X such that the constraint on (X, Y) is satisfied then a. Remove v from Y‘s domain b. removed true 3. Return removed 4 -QUEENS PROBLEM 1 2 3 4 X 1 {1, 2, 3, 4} X 2 = 3 is incompatible with any of the remaining values of X 3 1 2 3 4 X 2 {1, 2, 3, 4} X 3 {1, 2, 3, 4} X 4 {1, 2, 3, 4} 4) The algorithm calls AC 3, which eliminates 3 from the domain of X 2 39 39

4 -QUEENS PROBLEM 1 2 3 4 X 1 {1, 2, 3, 4} X 2 {1, 2, 3, 4} X 3 {1, 2, 3, 4} X 4 {1, 2, 3, 4} 1 2 3 4 4) The algorithm calls AC 3, which eliminates 3 from the domain of X 2, and 2 from the domain of X 3 40 40

4 -QUEENS PROBLEM 1 2 3 4 X 1 {1, 2, 3, 4} X 2 {1, 2, 3, 4} X 3 {1, 2, 3, 4} X 4 {1, 2, 3, 4} 1 2 3 4 4) The algorithm calls AC 3, which eliminates 3 from the 41 domain of X 2, and 2 from the domain of X 3, and 4 from 41 the domain of X 3

4 -QUEENS PROBLEM 1 2 3 4 X 1 {1, 2, 3, 4} X 2 {1, 2, 3, 4} X 3 {1, 2, 3, 4} X 4 {1, 2, 3, 4} 1 2 3 4 5) The domain of X 3 is empty backtracking 42 42

4 -QUEENS PROBLEM 1 2 3 4 X 1 {1, 2, 3, 4} X 2 {1, 2, 3, 4} X 3 {1, 2, 3, 4} X 4 {1, 2, 3, 4} 1 2 3 4 6) The algorithm removes 1 from X 1’s domain and assign 2 43 to X 1 43

4 -QUEENS PROBLEM 1 2 3 4 X 1 {1, 2, 3, 4} X 2 {1, 2, 3, 4} X 3 {1, 2, 3, 4} X 4 {1, 2, 3, 4} 1 2 3 4 7) The algorithm performs forward checking 44 44

4 -QUEENS PROBLEM 1 2 3 4 X 1 {1, 2, 3, 4} X 2 {1, 2, 3, 4} X 3 {1, 2, 3, 4} X 4 {1, 2, 3, 4} 1 2 3 4 8) The algorithm calls AC 3 45 45

4 -QUEENS PROBLEM 1 2 3 4 X 1 {1, 2, 3, 4} X 2 {1, 2, 3, 4} X 3 {1, 2, 3, 4} X 4 {1, 2, 3, 4} 1 2 3 4 8) The algorithm calls AC 3, which reduces the domains of 46 X 3 and X 4 to a single value 46

EXPLOITING THE STRUCTURE OF CSP If the constraint graph contains several components, then solve one independent CSP per component NT WA Q NSW SA T V 47 47

EXPLOITING THE STRUCTURE OF CSP If the constraint graph is a tree, then : 1. Order the variables from the root to the leaves (X 1, X 2, …, Xn) Y 2. For j = n, n-1, …, 2 call REMOVE-VALUES(Xj, Xi) where Xi is the parent of Xj 3. Assign any valid value to X 1 4. For j = 2, …, n do Assign any value to Xj consistent with the value assigned to its parent Xi X Z U V (X, Y, Z, U, V, W) W 48 48

EXPLOITING THE STRUCTURE OF CSP Whenever a variable is assigned a value by the backtracking algorithm, propagate this value and remove the variable from the constraint graph NT WA Q NSW SA V 49 49

EXPLOITING THE STRUCTURE OF CSP Whenever a variable is assigned a value by the backtracking algorithm, propagate this value and remove the variable from the constraint graph NT WA Q NSW V If the graph becomes a tree, then proceed as shown in previous slide 50 50

8 Representation Variables Xij for i, j in {1, . . , 9} 2 2 5 8 7 Domains {1, …, 9} Constraints: 9 Xij Xik, for j k Xij Xkj, for i k 9 5 7 Xij Xmn, for (i, j), (m, n) in same cell 4 3 2 6 9 51

8 Representation Variables Xij for i, j in {1, . . , 9} 2 2 5 8 7 Domains {1, …, 9} Constraints: 9 Xij Xik, for j k Xij Xkj, for i k 9 5 7 Xij Xmn, for (i, j), (m, n) in same cell 4 3 2 6 9 52

8 Representation Variables Xij for i, j in {1, . . , 9} 2 2 5 8 7 Constraints: 9 Xij Xik, for j k Xij Xkj, for i k 9 5 7 4 3 2 Domains {1, …, 9} 6 9 Xij Xmn, for (i, j), (m, n) in same cell Can we detect this using constraint 53 propagation?

8 Representation Variables Xij for i, j in {1, . . , 9} 2 2 5 8 7 Constraints: 9 Xij Xik, for j k Xij Xkj, for i k 9 5 7 4 3 2 Domains {1, …, 9} 6 9 Xij Xmn, for (i, j), (m, n) in same cell Must detect 9 way interactions 54

C 39=5 C 15=4 C 33=8 C 12=9 C 28=4 C 38=1 … 8 R 18=3 Representation 2 R 22=8 2 R 32=6 R 37=9 … 2 5 8 7 Variables Rij in {1, …, 9} Cij in {1, …, 9} Xij in {(1, 1), …, (3, 3)} Constraints: 9 ? ? ? 9 5 7 4 3 2 6 9 55

C 39=5 C 15=4 C 33=8 C 12=9 C 28=4 C 38=1 … R 18=3 8 R 22=8 X 1 = (1, 3) Representation 2 2 8 R 32=6 R 37=9 … 2 5 8 7 Variables Rij in {1, …, 9} Cij in {1, …, 9} Xij in {(1, 1), …, (3, 3)} Constraints: 9 ? ? ? 9 5 7 4 3 2 6 9 56

C 39=5 C 15=4 C 33=8 C 12=9 C 28=4 C 38=1 … R 18=3 8 R 22=8 X 1 = (1, 3) 8 Representation 2 X 2 = (3, 3) R 32=6 R 37=9 … 2 2 2 5 8 7 Variables Rij in {1, …, 9} Cij in {1, …, 9} Xij in {(1, 1), …, (3, 3)} Constraints: 9 ? ? ? 9 5 7 4 3 2 6 9 57

C 39=5 C 15=4 C 33=8 C 12=9 C 28=4 C 38=1 … 8 R 18=3 R 2 2=8 X 1 = (1, 3) 8 Representation 2 X 2 = (3, 3) 2 R 32=6 R 37=9 … 2 5 8 X 32 = (2, 3) 2 X 37 = (3, 3) 7 Variables Rij in {1, …, 9} Cij in {1, …, 9} Xij in {(1, 1), …, (3, 3)} Constraints: 9 ? ? ? 9 5 7 4 3 2 6 9 58

C 39=5 C 15=4 C 33=8 C 12=9 C 28=4 C 38=1 … R 2 2=8 Representation 2 8 R 18=3 X 1 = (1, 3) 8 X 2 = (3, 3) 2 R 32=6 R 37=9 … 2 5 8 X 32 = (2, 3) 2 X 37 = (3, 3) 7 Constraints: 9 Rij=k Ckj=I 9 5 7 4 3 2 Variables Rij in {1, …, 9} Cij in {1, …, 9} Xij in {(1, 1), …, (3, 3)} 6 9 Similar constraints between X’s and R’s, X’s and C’s Rij Rik for j k Cij Cik for j k Xij Xik for j k 59

C 1 2 = 9 R 1 2 = Representation 2 8 {1 -9} 2 2 5 8 7 Constraints: 9 Rij=k Ckj=I 9 5 7 4 3 2 Variables Rij in {1, …, 9} Cij in {1, …, 9} Xij in {(1, 1), …, (3, 3)} 6 9 Similar constraints between X’s and R’s, X’s and C’s Rij Rik for j k Cij Cik for j k Xij Xik for j k 60

C 1 2 = 9 C 22 = {1 -9} R 1 2 = Representation 2 8 {2 -9} 2 2 5 8 7 Constraints: 9 Rij=k Ckj=i 9 5 7 4 3 2 Variables Rij in {1, …, 9} Cij in {1, …, 9} Xij in {(1, 1), …, (3, 3)} 6 9 Similar constraints between X’s and R’s, X’s and C’s Rij Rik for j k Cij Cik for j k Xij Xik for j k 61

C 1 = 9 X 32 = (2, 2) X 2 = (3, 3) 2 2 R 1 2 = Representation 2 8 {2 -9} 2 2 5 8 7 Constraints: 9 Rij=k Ckj=I 9 5 7 4 3 2 Variables Rij in {1, …, 9} Cij in {1, …, 9} Xij in {(1, 1), …, (3, 3)} 6 9 Similar constraints between X’s and R’s, X’s and C’s Rij Rik for j k Cij Cik for j k Xij Xik for j k 62

C 1 = 9 X 32 = (2, 2) X 2 = (3, 3) 2 2 R 12=2 Representation 2 8 2 2 5 8 7 Constraints: 9 Rij=k Ckj=I 9 5 7 4 3 2 Variables Rij in {1, …, 9} Cij in {1, …, 9} Xij in {(1, 1), …, (3, 3)} 6 9 Similar constraints between X’s and R’s, X’s and C’s Rij Rik for j k Cij Cik for j k Xij Xik for j k 63

C 1 = 9 R 12=2 X 32 = (2, 2) X 2 = (3, 3) 2 2 2 Representation 2 8 2 2 5 8 7 Constraints: 9 Rij=k Ckj=I 9 5 7 4 3 2 Variables Rij in {1, …, 9} Cij in {1, …, 9} Xij in {(1, 1), …, (3, 3)} 6 9 Similar constraints between X’s and R’s, X’s and C’s Rij Rik for j k Cij Cik for j k Xij Xik for j k 64

LOCAL SEARCH FOR CSPS Init: Make an arbitrary assignment Repeat: Modify some variable to reduce # of violated constraints 65

BOOLEAN SATISFIABILITY PROBLEMS n variables ui, …, un p constraints of form ui* uj* uk*= 1 where ui* local is either u or ui Highly successful searchi algorithms � Walk. SAT See R&N 7. 3 66

OBSERVATIONS… If a CSP has few constraints, local search solves it quickly � Random starting assignment not too far from a solution � Million-queens puzzles solved in < 1 min, c. 1990 If a CSP has many constraints, local search solves it quickly � Constraints “guide” solver to a solution (if one exists) 67

HARD SUDOKU’S 1. . |. . 2. 9. | 4. . |. 5. . . 6 |. . . | 7. . ------+-----. 5. | 9. 3 |. . . |. 7. |. . . | 8 5. |. 4. ------+-----7. . | 6. . . 3. |. . 9 |. 8. . . 2 |. . 1 Human solvers: Lot of logic (deep constraint propagation) Computer solvers: Lot of backtracking 68

HARD & EASY 3 -SAT PROBLEMS Let R = # of constraints / # of variables n , the fraction of hard problems reduces to 0 As 69

RECAP Constraint propagation, AC 3 Taking advantage of CSP structure Local search for CSPs 70

NEXT CLASS Intro to uncertainty R&N 4. 3 -4, 13. 1 -2 71