University of Science and Technology Faculty of Computer

University of Science and Technology Faculty of Computer Science and Information Technology Artificial Intelligence (AI) 4 th Year B. Sc : Information Technology Academic Year : 2017 -2018 Instructor : Diaa Eldin Mustafa Ahmed Problem Solving (Searching Techniques)-2/2

Informed (Heuristic Search) Searching Strategies in AI Informed (Heuristic Search) Un-informed (Blind Search) Depth-first (DFS) Depth. Limited (DLS) Breadth. First (BFS) Cost. First (CFS) Best-First Search (BFS) A* Search Iterative Deepening 2 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6) Hill Climbing Constraint Satisfaction

Informed (Heuristic Search) q. For complex problems, the traditional algorithms, presented above, are unable to find the solution within some practical time and space limits. q. Consequently, many special techniques are developed, using heuristic functions. Ø Blind search is not always possible, because they require too much time or Space (memory). Ø Heuristics are rules of thumb; they do not guarantee for a solution to a problem. Ø Heuristic Search is a weak techniques but can be effective if applied correctly; they require domain specific information. 3 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

Heuristic Search compared with other search The Heuristic search is compared with Brute force or Blind search techniques 4 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

Informed Search q. A search strategy which searches the most promising branches of the state-space first can: Ø Find a solution more quickly, Ø Find solutions even when there is limited time available, Ø Often find a better solution, since more profitable parts of the state-space can be examined, while ignoring the unprofitable parts. q A search strategy which is better than another at identifying the most promising branches of a search-space is said to be more informed. 5 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

Best-first search strategy q Combing depth-first search and breadth-first search. q Selecting the node with the best estimated cost among all nodes. q This method has a global view. e. g. 8 -puzzle problem evaluation function f(n) = d(n) + w(n) where : d(n) : is the depth of node n. w(n) : is # of misplaced tiles in node n. 6 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

Best-first search strategy 7 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

Hill climbing q Hill climbing is a mathematical optimization technique which belongs to the family of local search. q It is an iterative algorithm that starts with an arbitrary solution to a problem, then attempts to find a better solution by incrementally changing a single element of the solution. q If the change produces a better solution, an incremental change is made to the new solution, repeating until no further improvements can be found. q A variant of depth-first search. q The method selects the locally optimal node to expand. 8 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

Hill climbing q. An 8 -puzzle problem solved by a hill climbing method. 9 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

Exercise (1) 10 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

Exercise (1) From the previous search graph try to solve the following questions : 1 - Show the complete search tree. 2 - Find the path from the start state S to the goal state G if we implement : a) Best first search b) Hill Climbing search 3 - Which is the best solution from the above 11 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

Exercise (2) Consider the following map. (START) A 36 B 61 31 32 D 52 31 E 43 20 F 102 112 G 32 36 40 12 K 122 H I L 80 C 45 J AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6) M (END)/Goal

q. Straight Line Distance to M. 13 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6) A B C D E F G H I J K L M 223 222 166 192 165 136 122 111 100 60 32 102 0

1 - Using the A* algorithm work out a route from town A to town M. Use the following cost functions: G(n) = The cost of each move as the distance between each town (shown on map). H(n) = The Straight Line Distance between any town and town M. These distances are given in the table below. 2 - Provide the search tree for your solution and indicate the order in which you expanded the nodes. 3 - Finally, state the route you would take and the cost of that route. AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture 14 (6)

q. 15 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

Constraint Satisfaction Problems (CSPs) q Constraints arise in most areas of human endeavor. q Constraints are a natural medium for people to express problems in many fields. q Many real problems in AI can be modeled as Constraint Satisfaction Problems (CSPs) and are solved through search. q Examples of constraints : Ø The sum of three angles of a triangle is 180 degrees, Ø The sum of the currents flowing into a node must equal zero. q Constraint is a logical relation among variables. Ø The constraints relate objects without precisely specifying their positions ; moving any one, the relation is still maintained. Ø Example : “circle is inside the square”. 16 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

Constraint Satisfaction Problems (CSPs) q Constraint satisfaction ØThe Constraint satisfaction is a process of finding a solution to a set of constraints. ØThe constraints articulate allowed values for variables. ØFinding solution is evaluation of these variables that satisfies all constraints. q The CSPs are all around us while managing work, home life, budgeting expenses and so on; Ø where we do not get success in finding solution, there we run into problems. Ø we need to find solution to such problems satisfying all constraints. Ø the Constraint Satisfaction problems (CSPs) are solved through search. 17 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

Constraint Satisfaction Problems (CSPs) q. Many problems in AI can be considered as problems of constraint satisfaction, in which the goal state satisfies a given set of constraint. q. Constraint satisfaction problems can be solved by using any of the search strategies. q The general form of the constraint satisfaction procedure is as follows: 18 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

Constraint Satisfaction Problems (CSPs) q Until a complete solution is found or until all paths have led to lead ends, do: 1. select an unexpanded node of the search graph. 2. Apply the constraint inference rules to the selected node to generate all possible new constraints. 3. If the set of constraints contains a contradiction, then report that this path is a dead end. 4. If the set of constraints describes a complete solution then report success. 5. If neither a constraint nor a complete solution has been found then apply the rules to generate new partial solutions. 6. Insert these partial solutions into the search graph. AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture 19 (6)

(CSPs)- Cryptarithmetic Example: consider the following cryptarithmetic problem. S END + MORE -------MONEY -------q The aim is to assign each letter a unique integer in the range 0. . 9 so that the sum is correct. q Define the problem as a constraint satisfaction problem (CSP) in terms of : Ø Variables V, Ø Domains D and Ø Constraints C. q Show an analysis of the problem can be used to reduce AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture the domains of the variables and create additional constraints. 20 (6)

(CSPs)- Cryptarithmetic Example CONSTRAINTS: 1. No two digit can be assigned to same letter. 2. Only single digit number can be assign to a letter. 3. No two letters can be assigned same digit. 4. Assumption can be made at various levels such that they do not contradict each other. 5. The problem can be decomposed into secured constraints. A constraint satisfaction approach may be used. 6. Any of search techniques may be used. 7. Backtracking may be performed as applicable us applied search techniques. 8. Rule of arithmetic may be followed. 21 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

(CSPs)- Cryptarithmetic Example Initial Solution Initially the problem can be stated as follows : V = {S, E, N, D, M, O, R, Y} DS = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9} DE = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9} DN = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9} DD = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9} DM = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9} DO = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9} DR = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9} DY = {0, 1, 2, 3, 4, 5, 6, 7, 8, 9} 22 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

(CSPs)- Cryptarithmetic Example 1 -From Column 5, since it is only carry-over possible from sum of 2 single digit number in column 4 then. ØM=1 or (C 4=1) ØThen remove 1 from the all domains of the variables ØDomain of S will be DS ={1} only. DS DE DN DD DM DO DR DY 23 = {0, 2, 3, 4, 5, 6, 7, 8, 9} 5 4 3 2 1 = {0, 2, 3, 4, 5, 6, 7, 8, 9} C 4 C 3 C 2 C 1 = {0, 2, 3, 4, 5, 6, 7, 8, 9} S E N D = {0, 2, 3, 4, 5, 6, 7, 8, 9} + M O R E = {1} ----------= {0, 2, 3, 4, 5, 6, 7, 8, 9} MO N E Y = {0, 2, 3, 4, 5, 6, 7, 8, 9} -----------= {0, 2, 3, 4, 5, 6, 7, 8, 9} AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

(CSPs)- Cryptarithmetic Example 2 -To produce a carry from column 4 to column 5 then: Ø S + M is at least 9 so 'S=8 or 9' so Ø S+M=9 or 10 & so Ø 'O = 0 or 1'. But 'M=1', so 'O = 0‘. 5 4 3 2 1 C 4 C 3 C 2 C 1 S E N D + 1 0 R E = 1 0 N E Y 3 - If there is carry from Column 3 to 4 then: Ø 'E=9' & so 'N=0'. Ø But 'O = 0' so there is no carry & 'S=9' & ‘C 3=0'. 1 0 C 2 C 1 9 E N D AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture + 1 0 R E 24 (6) = 1 0 N E Y

(CSPs)- Cryptarithmetic Example 4 - If there is no carry from column 2 to 3 then ØE=N which is impossible, Ø therefore there is carry & N=E+1 & C 2=1. 5 - If there is carry from column 1 to 2 then: Ø N+R=E +10 & N=E+1 Ø so E+1+R=E +10, so 'R=9' but 'S=9', so there must be carry from column 1 to 2. Ø Therefore ‘C 1=1' & 'R=8'. 5 4 3 2 1 1 0 1 1 9 E N D + 1 0 8 E = 1 0 N E Y 25 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

(CSPs)- Cryptarithmetic Example 6 - To produce carry ‘C 1=1' from column 1 to 2, we must have: Ø 'D+E=10+Y‘ as Y cannot be 0/1 so D+E is at least 12. Ø D is at most 7 & E is at least 5 (D cannot be 8 or 9 as it is already assigned). Ø N is at most 7 & 'N=E+1' so 'E=5 or 6'. 7 - If E were 6 & D+E at least 12 then D would be 7, Ø but 'N=E+1' & N would also be 7 which is impossible. Ø Therefore 'E=5' & 'N=6'. 8. D+E is at least 12 for that we get Ø 'D=7 & Y=2 . 5 4 3 2 1 1 0 1 1 26 9 + 1 = 1 0 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6) 5 6 7 0 8 5 6 5 2

Step (1) M=1 O=0 M=C 4=1 S+M=S+1<=10 so S=8 or 9 E(2)+O(0)+C 2 (1 or 0)=N if C 2=1 if C 2=0 E(2)+O(0)+C 2 (1)=N(3) E(2)+O(0)+C 2 (0)=N(2) Contradiction N(3)+R+C 1 (1 or 0)=E(2) if C 1=1 R =8 S =9 C 3=0 D+E(2)=Y D>7 to generate a carry Contradiction 27 if C 1=0 R=9 S=8 C 3=1 D=4 D=5 Solution Not Satisfied AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6) D>7 to satisfy Y should generate a carry Contradiction

Step (2) M=1 O=0 S=8 , S=9 , C 3=1 , C 3=0 E=3 E(3)+O(0)+C 2 (0 or 1)=M if C 2=1 if C 2=0 E(3)+O(0)+C 2 (1 or 0)=N(4) E(3)+O(0)+C 2 (0)=N(3) Contradiction N(4)+R+C 1 (1 or 0)=E(3) if C 1=1 R=8 S=9 D + E(3)=Y 28 if C 1=0 Contradiction Solution Not Satisfied AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6) D>7 to satisfy Y should generate a carry Contradiction

(CSPs)- Cryptarithmetic Example And our problem now looks like this S E N D + 1 0 R E = 10 N E Y We can now turn our attention to the thousands column. C 3 could be zero or one (representing a carry or not). That is, S + M + C 3 = 10 + O We’ll consider both cases of carry below 29 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

(CSPs)- Cryptarithmetic Example Simplify Subtract 2 (or 1) from both sides Conclusions C 3 = 1 C 3 = 0 S + 1 = 10 + O S + 1 + 0 = 10 + O S + 2 = 10 + O S = 8 + O S = 9 + O If O is 0, S = 8 If O is 1, S = 9 Therefore, S must equal 8 or 9. Let’s try to prove that S = 8. 8 E N D + 1 0 R E = 1 0 N E Y 30 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6) If O is 0, S = 9

(CSPs)- Cryptarithmetic Example From the above, we showed that O=0 if S=8. If this is the correct answer we can see that a carry is required in the hundreds column (C 2) as E + 0 = N is not valid as E and N would take the same value, which is not allowed. Or, to show it is invalid another way E + 0 + C 2(0) = 10 + N Simplify E = N + 10 Which is invalid, as N would have to be zero, which is already the case with O So, assuming there is a carry, we have E + 0 + 1 = 10 + N Simplifying gives E + 1 = 10 + N Subtract 1 from both sides E = 9 + N 31 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

(CSPs)- Cryptarithmetic Example The only possible value for N is zero (else E would be outside its domain) but O is already zero, so this answer is invalid. Therefore, S cannot equal 8, so it S = 9, giving 9 E N D + 1 0 R E = 10 N E Y Now, consider the thousands column. We have either 9 + 1+ C 3(0) = O or 9 + 1 + C 3(1) = O 32 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

(CSPs)- Cryptarithmetic Example If the carry, C 3, is zero then M = 1 and O = 0. If the carry, C 3, is one then M = 1 and O = 1. As O 1 (as M=1), then O = 0, giving 9 E N D + 10 RE = 10 NEY Turning our attention to the tens column, we have N + R = E. We can see that we must also have a carry as the hundreds column (E + 0 = N) needs to have a carry from the previous column else E = N, which is invalid. We can also state that N = E + 1 by definition of the hundreds column and the carry, C 2. From the above we can derive N + R + C 1 = 10 + E. In the table below we will consider at this formula AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture 33 (6)

(CSPs)- Cryptarithmetic Example C 1 = 1 N + R + 1 = 10 + E Subtract 1 N + R = 9 + E Substitute N = E + 1 + R = 9 + E Subtract E 1 + R = 9 Subtract 1 R = 8 Possible Yes Therefore, we can set R to 8, giving 9 EN D + 1 0 8 E = 1 0 NE Y 34 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6) C 1 = 0 N + R + 0 = 10 + E E + 1 + R = 10 + E 1 + R = 10 R = 9 No, as S = 9

(CSPs)- Cryptarithmetic Example We could continue with this algebraic analysis and solve the puzzle completely but if we assume that we could not go any further then we have the following to give our CSP search. V = {S, E, N, D, M, O, R, Y} DS = { 9} DE = {2, 3, 4, 5, 6} DN = {2, 3, 4, 5, 6, 7} DD = {2, 3, 4, 5, 6, 7} DM = {1} DO = {0} DR = {8} DY = {2, 3, 4, 5, 6, 7} C 1 = N = E + 1 (note, how this constraint also further limits the domain of E). AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture 35 (6)

(CSPs)- Cryptarithmetic Example In Summary V = {S, E, N, D, M, O, R, Y} DS = { 9} DE = {2, 3, 4, 5, 6} DN = {2, 3, 4, 5, 6, 7} DD = {2, 3, 4, 5, 6, 7} DM = {1} DO = {0} DR = {8} DY = {2, 3, 4, 5, 6, 7} C 1 = If the same letter occurs more than once, it must be assigned the same value C 2 = No two different letters may be assigned the same digit C 3 = N = E + 1 36 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

(CSPs)- Cryptarithmetic Example q. 37 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

(CSPs)- Cryptarithmetic Example q. 38 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

(CSPs)-Color Map q Using the most-constrained-variable CSP heuristic colour the following map using the colours Blue, Red and Green. Show you reasoning at each step of the algorithm. 39 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

(CSPs)-Color Map Now, let us consider this in the context of a CSP problem. We start with the following variables and their domains A = {Red, Blue, Green} B = {Red, Blue, Green} C = {Red, Blue, Green} D = {Red, Blue, Green} E = {Red, Blue, Green} F = {Red, Blue, Green} 40 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

(CSPs)-Color Map After making the first two assignments, and using forward tracking to reduce the domains of the other variables, we have the following assignments (variables marked * have been instantiated) A* = {Green} B* = {Red} C = {Red, Blue} D = {Red, Blue, Green} E = {Blue} F = {Blue, Green} 41 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

(CSPs)-Color Map To decide which variable to instantiate we choose the variable with the smallest domain. In this case E. After instantiating E the assignments look like this (again forward tracking has been used to reduce the domain of the remaining variables) 42 A* = {Green} B* C D = = = {Red} {Red, Blue, Green} E* F = = {Blue} {Green} AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

(CSPs)-Color Map We can now choose C (or F) and then F (or C). This results in the following 43 A* = {Green} B* = {Red} C* D = = {Red} {Red, Blue} E* = {Blue} F* = {Green} AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

(CSPs)-Color Map Lastly, we can assign D Red or Blue. We choose (arbitrarily) Red, leading to the final solution 44 A* = {Green} B* = {Red} C* = {Red} D* = {Red} E* = {Blue} F* = {Green} AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

(CSPs)-Color Map A B C D Search Tree 45 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6) E F

(CSPs)-Color Map C A B E F Graph Search 46 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6) D

A 47 B C D E F R G B B R G B R G B R G B AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

q. 48 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

(CSPs)- Latin Square Problem q. Examples of CSPs: Some popular puzzles like, the Latin Square, the Eight Queens, and Sudoku are stated below. q. Latin Square Problem : How can one fill an n × n table with n different symbols such that each symbol occurs exactly once in each row and each column ? Solutions : The Latin squares for n = 1, 2, 3 and 4 are : 49 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

Constraint Satisfaction Problems (CSPs) q. Eight Queens Puzzle Problem : How can one put 8 queens on a (8 x 8) chess board such that no queen can attack any other queen ? Solutions: The puzzle has 92 distinct solutions. If rotations and reflections of the board are counted as one, the puzzle has 12 unique solutions. 50 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

Best-first search q To implement an informed search strategy, we need to slightly modify the skeleton for agenda-based search that we've already seen. q Again, the crucial part of the skeleton is where we update the agenda. q Rather than simply adding the new agenda items to the beginning (depth-first) or end (breadth-first) of the existing agenda, we add them to the existing agenda in order according to some measure of how promising we think a state is, with the most promising ones first. This gives us best-first search. q Best-first search isn't so much a search strategy, as a mechanism for implementing many different types of informed search 51 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

q. 52 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

q. 53 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)

Thank You End 54 AI - (2017 -2018) -Diaa Eldein Mustafa - Lecture (6)