PAGE of designing an agent Problem types Problem

• PAGE of designing an agent • Problem types • Problem formulation • Search Strategies 1

PAGE: Percepts, Actions, Goals, Environment Sensors: eyes, camera, … percepts environment agent actions Effectors: hands, motors, … Fig 1. A generic agent diagram e. g. Percepts cameras, speedometer, GPS, sonar Actions steer, accelerate, brake Goals safely to destination Environment traffic light, other traffic, pedestrians, in Japan Fig 1. The taxi driver agent and its PAGE description 2

More e. g. : Vacuum-clearning Agent Percepts Actions Goals Environment touch sensor, photo sensor, infrared sensor go forward, turn right by 90 o，turn left by 90 o, such up dirt, and turn off clean up the room A grid of squares that may have obstacles (wall or furniture), dirt, or may be open space (having nothing) Fig 2. The Vacuum-cleaning agent and its PAGE description Notes: Touch sensor is to check if the machine has bumped into something. Photo sensor is to check if there is dirt there. Infrared sensor is to check if the agent is in its home location. 3

Problem types Deterministic, accessible => single-state problem more The agent sensors give it enough information to tell exactly which state it is in. technique search Deterministic, inaccessible => multiple-state problem The agent knows all the effects of its actions but has limited access to the world knowledge state so that it must reason about sets of states that it might get to. technique search Nondeterministic, inaccessible => Contingency problem There is no fixed action sequence that guarantees a solution to the problem. It must sensors during execution. solution is a tree or policy, not a single sequence of actions. techniques: interleave search + execution Unknown state space => exploration problem less The agent must experiment, gradually discovering what its actions do and what sorts of states exist. techniques: experiment + discovery 4

Define a problem A problem is defined by four items: initial state: The world initial state. operators: A set of possible actions. goal-test: To check if it reaches goal state or states. path-cost-fucntion: It is the sum of cost of the individual actions along the path, which is from one state to another. Notes: State space is the set of all states reachable from the initial state by any sequence of actions. Solution is a path (a sequence of operators) from the initial state to a state that satisfies the goal test. How to find a solution? 5 search

Some examples - e. g. 1 The 8 -puzzle 5 4 1 6 1 8 8 7 3 2 7 Start state 2 3 4 6 5 Goal state: the location of each of the eight tiles in one of the nine squares. operators: blank tile moves left, right, up, or down. goal-test: state matches the goal state. path-cost-fucntion: each step costs 1 so the total cost is the length of the path. Can you write a Java program that can find a solution, i. e. path from the initial state to the goal state? 6

Some examples – e. g. 2 The vacuum world: 2 squares vacuum 1 3 5 7 2 4 6 8 dirt state: one of the eight states above. operators: move left, move right, suck. goal-test: no dirt left in any square. path-cost-fucntion: each action costs 1. Can you write a Java program that can find a solution, i. e. path from any initial state to the goal state? 7

Breath-first search The breadth-first search algorithm searches a state space by constructing a hierarchical tree structure consisting of a set of nodes and links. The algorithm defines a way to move through the tree structure, examining the value at nodes. From the start, it looks at each node one edge away. Then it moves out from those nodes to all two edges away from the start. This continues until either the goal node is found or the entire tree is searched. The algorithm follows: 1. Create a queue and add the first node to it. 2. Loop: - If the queue is empty, quit. - Remove the first node from the queue. - If the node contains the goal state, then exit with the node as the solution. - For each child of the current node: add the new state to the back of the queue. 8

An example of breadth-first search: Rochester Wausau International. Falls Grand. Forks Bemidji Duluth Fargo Green. Bay St. Cloud Minneapolis Wausau La. Crosse Madison Milwaukee Sioux Rochester Dubuque Rockford Chicago [Rochester]->[Sioux Falls, Minneapolis, La. Crosse, Dubuque]->[Minneapolis, La. Crosse, Dubuque, Frago, Rochester]-> [La. Crosse, Dubuque, Frago, Rochester, St. Cloud, Duluth, Wausau, La. Crosse, Rochester]->……(after 5 goal test and expansion) -> [Wausau, La. Crosse, Rochester, Minneapolis, Green. Bay, ……] 9 Finally, we get to Wausau, the goal test succeeds.

Source code for your reference A demo public Search. Node breadth. First. Search(Search. Node initial. Node, Object goal. State) { Vector queue = new Vector() ; queue. add. Element(initial. Node) ; initial. Node. set. Tested(true) ; // test each node once while (queue. size()> 0) { Search. Node test. Node = (Search. Node)queue. first. Element() ; queue. remove. Element. At(0) ; test. Node. trace() ; if (test. Node. state. equals(goal. State)) return test. Node ; // found it if (!test. Node. expanded) { test. Node. expand(queue, Search. Node. BACK) ; } } return null ; } 10

Depth-first search The depth-first algorithm follows a single branch of the tree down as many levels as possible until we either reach a solution or a dead end. It searches from the start or root node all the way down to a leaf node. If it does not find the goal node, it backtracks up the tree and searches down the next untested path until it reaches the next leaf. The algorithm follows: 1. Create a queue and add the first node to it. 2. Loop: - If the queue is empty, quit. - Remove the first node from the queue. - If the node contains the goal state, then exit with the node as the solution. - For each child of the current node: add the new state to the front of the queue. 11

An example of depth-first search: Rochester Wausau International. Falls Grand. Forks Bemidji Duluth Fargo Green. Bay St. Cloud Minneapolis Wausau La. Crosse Madison Milwaukee Sioux Rochester Dubuque Rockford Chicago [Rochester]->[Dubuque, La. Crosse, Minneapolis, Sioux Falls]->[Rockford, La. Crosse, Rochester, La. Crosse, Minneapolis, Sioux Falls]-> …… (after 3 goal test and expansion) -> [Green. Bay, Madison, Chicago, Rockford, Chicago, Madison, Dubuque, La. Crosse, Rochester, La. Crosse, Minneapolis, Sioux Falls] We remove Green. Bay and add Milwaukee, La. Crosse, and Wausau to the queue in that order. Finally, 12 we get to Wausau, at the front of the queue and the goal test succeeds.

Source code for your reference A demo public Search. Node depth. First. Search(Search. Node initial. Node, Object goal. State) { Vector queue = new Vector() ; queue. add. Element(initial. Node) ; initial. Node. set. Tested(true) ; // test each node once while (queue. size()> 0) { Search. Node test. Node = (Search. Node)queue. first. Element() ; queue. remove. Element. At(0) ; test. Node. trace() ; // display trace information if (test. Node. state. equals(goal. State)) return test. Node ; // found it if (!test. Node. expanded) { test. Node. expand(queue, Search. Node. FRONT); } } return null ; } 13