Oregon State University CS 430 Intro to AI

Oregon State University – CS 430 Intro to AI Search-Based Agents w Appropriate in Static Environments where a model of the agent is known and the environment allows n n prediction of the effects of actions evaluation of goals or utilities of predicted states w Environment can be partially-observable, stochastic, sequential, continuous, and even multi-agent, but it must be static! w We will first study the deterministic, discrete, single-agent case. (c) 2003 Thomas G. Dietterich and Devika Subramanian 1

Oregon State University – CS 430 Intro to AI Computing Driving Directions Oradea 71 You are here 75 Neamt Zerind 87 151 Iasi Arad 140 Sibiu 92 99 Fagaras 118 Vasini 80 Rimnicu Vilcea Timisoara 111 Lugoj 97 142 211 Pitesti 70 Mehadia 146 75 Dobreta 101 138 120 85 Bucharest 98 Hirsova Urzicenl 86 90 Craiova Giurgia Eforie You want to be here (c) 2003 Thomas G. Dietterich and Devika Subramanian 2

Oregon State University – CS 430 Intro to AI Search Algorithms w Breadth-First w Depth-First w Uniform Cost w A* w Dijkstra’s Algorithm (c) 2003 Thomas G. Dietterich and Devika Subramanian 3

Oregon State University – CS 430 Intro to AI Breadth-First Detect duplicate path (291) Oradea (146) 71 75 Detect duplicate path (197) Zerind (75) Arad (0) 151 140 Sibiu (140) 99 118 80 Timisoara (118) 111 Detect new shorter path (418) Fagara s (239) Rimnicu Vilcea (220) Lugoj (229) Pitesti (317) 97 211 70 Mehadia (299) 75 Dobreta (486) 146 101 138 Bucharest (450) 120 Detect new shorter path (374) Detect duplicate path (494) Craiova (366) Detect duplicate path (504) Detect duplicate path (455) (c) 2003 Thomas G. Dietterich and Devika Subramanian 4

Oradea (146) 71 75 Zerind (75) Arad (0) Breadth-First 151 140 Sibiu (140) Fagaras (239) 99 118 80 Timisoara (118) 111 Rimnicu Vilcea (220) Lugoj (229) Pitesti (317) 97 211 70 Mehadia (299) 75 Dobreta (486) 146 101 138 Bucharest (450) 120 Craiova (366) Arad (0) Zerind (75) Arad (150) Oradea (146) Zerind (217) Sibiu (197) Sibiu (140) Arad (280) Sibiu (338) Timisoara (118) Fagaras (239) Oradea (291) Rimnicu Vilcea (220) Bucharest (450) Sibiu (300) Arad (236) Pitesti (317) Craiova (366) Lugoj (229) Timisoara (340) Mehadia (299) Lugoj (369) Rimnicu Vilcea (317) Bucharest (418) Craiova (455) Rimnicu Vilcea (482) Dobreta (486) Medhadia (449) (c) 2003 Thomas G. Dietterich and Devika Subramanian Dobreta (374) Pitesti (504) Craiova (494) 5

Oregon State University – CS 430 Intro to AI Formal Statement of Search Problems w State Space: set of possible “mental” states n cities in Romania w Initial State: state from which search begins n Arad w Operators: simulated actions that take the agent from one mental state to another n traverse highway between two cities w Goal Test: n Is current state Bucharest? (c) 2003 Thomas G. Dietterich and Devika Subramanian 6

Oregon State University – CS 430 Intro to AI General Search Algorithm w Strategy: first-in first-out queue (expand oldest leaf first) (c) 2003 Thomas G. Dietterich and Devika Subramanian 7

Oregon State University – CS 430 Intro to AI Leaf Selection Strategies w Breadth-First Search: oldest leaf (FIFO) w Depth-First Search: youngest leaf (LIFO) w Uniform Cost Search: cheapest leaf (Priority Queue) w A* search: leaf with estimated shortest total path length g(x) + h(x) = f(x) n n n where g(x) is length so far and h(x) is estimate of remaining length (Priority Queue) (c) 2003 Thomas G. Dietterich and Devika Subramanian 8

Oregon State University – CS 430 Intro to AI A* Search w Let h(x) be a “heuristic function” that gives an underestimate of the true distance between x and the goal state n Example: Euclidean distance w Let g(x) be the distance from the start to x, then g(x) + h(x) is an lower bound on the length of the optimal path (c) 2003 Thomas G. Dietterich and Devika Subramanian 9

Oregon State University – CS 430 Intro to AI Euclidean Distance Table Arad Mehadia 241 0 Neamt 234 Craiova 160 Oradea 380 Dobreta 242 Pitesti 100 Bucharest Eforie 366 161 Rimnicu Vilcea 193 Fagaras 176 Sibiu 253 Giurgiu 77 Timisoara 329 Hirsova 151 Urziceni 80 Iasi 226 Vaslui 199 Lugoj 244 Zerind 374 (c) 2003 Thomas G. Dietterich and Devika Subramanian 10

Oregon State University – CS 430 Intro to AI A* Search Arad (0+366=366) Zerind (75+374=449) Sibiu (140+253=393) Fagaras (239+176=415) Oradea (291+380=671) Bucharest (450+0=450) Bucharest (418+0=418) Timisoara (118+329=447) Rimnicu Vilcea (220+193=413) Pitesti (317+100=417) Craiova (366+160=526) Craiova (455+160=615) All remaining leaves have f(x) ¸ 418, so we know they cannot have shorter paths to Bucharest (c) 2003 Thomas G. Dietterich and Devika Subramanian 11

Oregon State University – CS 430 Intro to AI Dijkstra’s Algorithm w Works backwards from the goal w Each node keeps track of the shortest known path (and its length) to the goal w Equivalent to uniform cost search starting at the goal w No early stopping: finds shortest path from all nodes to the goal (c) 2003 Thomas G. Dietterich and Devika Subramanian 12

Oregon State University – CS 430 Intro to AI Local Search Algorithms w Keep a single current state x w Repeat n n n Apply one or more operators to x Evaluate the resulting states according to an Objective Function J(x) Choose one of them to replace x (or decide not to replace x at all) w Until time limit or stopping criterion (c) 2003 Thomas G. Dietterich and Devika Subramanian 13

Oregon State University – CS 430 Intro to AI Hill Climbing w Simple hill climbing: apply a randomly-chosen operator to the current state w If resulting state is better, replace current state w Steepest-Ascent Hill Climbing: w Apply all operators to current state, keep state with the best value w Stop when no successors state is better than current state (c) 2003 Thomas G. Dietterich and Devika Subramanian 14

Oregon State University – CS 430 Intro to AI Gradient Ascent w In continuous state spaces, x = (x 1, x 2, …, xn) is a vector of real values w Continuous operator: x : = x+ x for any arbitrary vector x (infinitely many operators!) w Suppose J(x) is differentiable. Then we can compute the direction of steepest increase of J by the first derivative with respect to x, the gradient: (c) 2003 Thomas G. Dietterich and Devika Subramanian 15

Oregon State University – CS 430 Intro to AI Gradient Descent Search w Repeat n n Compute Gradient r. J Update x : = x + r. J w Until r. J ¼ 0 w is the “step size”, and it must be chosen carefully w Methods such as conjugate gradient and Newton’s method choose automatically (c) 2003 Thomas G. Dietterich and Devika Subramanian 16

Oregon State University – CS 430 Intro to AI Visualizing Gradient Ascent If is too large, search may overshoot and miss the maximum or oscillate forever (c) 2003 Thomas G. Dietterich and Devika Subramanian 17

Oregon State University – CS 430 Intro to AI Problems with Hill Climbing w Local optima w Flat regions w Random restarts can give good results (c) 2003 Thomas G. Dietterich and Devika Subramanian 18

Oregon State University – CS 430 Intro to AI Simulated Annealing w T = 100 (or some large value) w Repeat n n Apply randomly-chosen operator to x to obtain x′. Let E = J(x′) – J(x) If E > 0, /* J(x′) is better */ switch to x′ Else /* J(x′) is worse */ switch to x′ with probability l l exp [ E/T] /* large negative steps are less likely */ T : = 0. 99 * T */ “cool” T */ w Slowly decrease T (“anneal”) to zero w Stop when no changes have been accepted for many moves w Idea: Accept “down hill” steps with some probability to help escape from local minima. As T ! 0 this probability goes to zero. (c) 2003 Thomas G. Dietterich and Devika Subramanian 19