Planning Machine Learning UNIT4 Syllabus n Basic Plan

Planning & Machine Learning UNIT-4

Syllabus n Basic Plan Generation systems n STRIPS n Advanced Plan Generation systems n K-STRIPS n Strategic Explanations n Why, Why not and how explanation n Learning n Machine learning n Adaptive learning

Advanced Problem Solving Approaches n In order to solve nontrivial problems, it is necessary to combine n Basic problem solving strategies n Knowledge representation mechanisms n Partial solutions and at the end combine into complete problem solution (decomposition) n Planning is useful as a problem solving technique for non decomposable problem.

Planning • Definition : Planning is arranging a sequence of actions to achieve a goal. • Uses core areas of AI like searching and reasoning & • Is the core for areas like NLP, Computer Vision. Kinematics (ME) • Robotics Planning (CSE) • Examples : Navigation , Language Processing (Generation)

Components of a Planning System n In any general problem solving systems, elementary techniques to perform following functions are required n Choose the best rule (based on heuristics) to be applied n Apply the chosen rule to get new problem state n Detect when a solution has been found n Detect dead ends so that new directions are explored. n Repairing an almost correct solution

1. Choose Rules to apply n Most widely used technique for selecting appropriate rules is to n first isolate a set of differences between the desired goal state and current state, n identify those rules that are relevant to reducing these difference n if more rules are found then apply heuristic information to choose out of them.

Grocery Shopping example April 3, 2006 AI: Chapter 11: Planning 7

2. Apply Rules n In simple problem solving system, n applying rules was easy as each rule specifies the problem state that would result from its application. n In complex problem we deal with rules that specify only a small part of the complete problem state.

Example: Block World Problem n Block world problem assumptions Square blocks of same size n Blocks can be stacked one upon another. n Flat surface (table) on which blocks can be placed. n Robot arm that can manipulate the blocks. It can hold only one block at a time. n In block world problem, the state is described by a set of predicates representing the facts that were true in that state. n One must describe for every action, each of the changes it makes to the state description. n In addition, some statements that everything else remains unchanged is also necessary. n

Actions (Operations) done by Robot n UNSTACK (X, Y) : [US (X, Y)] n Pick up X from its current position on block Y. The arm must be empty and X has no block on top of it. n STACK (X, Y): [S (X, Y)] n Place block X on block Y. Arm must holding X and the top of Y is clear. n PICKUP (X): [PU (X) ] n Pick up X from the table and hold it. Initially the arm must be empty and top of X is clear. n PUTDOWN (X): [PD (X)] n Put block X down on the table. The arm must have been holding block X.

Contd. . n Predicates used to describe the state ON(X, Y) Block X on block Y. n ONTABLE(X) Block X on the table. n CLEAR(X) Top of X clear. n HOLD(X) Robot-Arm holding X. n ARM EMPTY Robot-arm empty. n Logical statements true in this block world. n Holding X means, arm is not empty n ( X) : HOLD (X) ~ ARM EMPTY n X is on a table means that X is not on the top of any block ( X) : ONTABLE (X) ~ ( Y) ON (X, Y) n Any block with no block on has clear top ( X) : (~ ( Y): ON (Y, X)) CLEAR (X)

Predicate logic n “→” Material Implication n “~” Not n “˅” Or n “˄” AND n “ ” For all n “Ǝ” There exists

Effect of Unstack operation n The effect of UNSTACK(X, Y) is described by the following axiom [CLEAR(X, State) ON(X, Y, State)] [HOLD(X, DO(UNSTACK (X, Y), State)) CLEAR(Y, DO(UNSTACK(X, Y), State)) ] n DO is a function that generates a new state as a result of given action and a state. n For each operator, set of rules (called frame axioms) are defined where the components of the state are n affected by an operator n n If UNSTACK(A, B) is executed in state S 0, then we can infer that HOLD (A, S 1) CLEAR (B, S 1) holds true, where S 1 is new state after Unstack operation is executed. not affected by an operator n If UNSTACK(A, B) is executed in state S 0, B in S 1 is still on the table but we can’t derive it. So frame rule stating this fact is defined as ONTABLE(Z, S) ONTABLE(Z, DO(US (A, B), S))

Contd. . n Advantage of this approach is that simple mechanism of resolution can perform all the operations that are required on the state descriptions. n Disadvantage is that n number of axioms becomes very large for complex problem such as COLOR of block also does not change. n So we have to specify rule for each attribute. n COLOR(X, red, S) COLOR(X, red, DO(US(Y, Z), s)) n To handle complex problem domain, there is a need of mechanism that does not require large number of explicit frame axioms.

STRIPS Mechanism n One such mechanism was used in early robot problem solving system named STRIPS(Stanford Research Institute Problem Solver) (developed by Fikes, 1971). n In this approach, each operation is described by three lists. n n n Pre_Cond list contains predicates which have to be true before operation. ADD list contains those predicates which will be true after operation DELETE list contain those predicates which are no longer true after operation n Predicates not included on either of these lists are assumed to be unaffected by the operation. n Frame axioms are specified implicitly in STRIPS which greatly reduces amount of information stored.

STRIPS – Style Operators n S (X, Y) Pre: n Del: n Add: n US (X, Y) n Pre: n Del: n Add: n PU (X) n Pre: n Del: n Add: n PD (X) n Pre: n Del: n Add: n CL (Y) HOLD (X) AE ON (X, Y) CL (X) AE ON (X, Y) AE HOLD (X) CL (Y) ONT (X) CL (X) AE ONT (X) AE HOLD (X) ONT (X) AE

STRIPS – Style Operators n STACK (X, Y) Pre: CL (Y) HOLD (X) n Del: CL (Y) HOLD (X) n Add: AE ON (X, Y) n UN STACK (X, Y) n Pre: ON (X, Y) CL (X) AE n Del: ON (X, Y) AE n Add: HOLD (X) CL (Y) n PICK UP (X) n Pre: ONT (X) CL (X) AE n Del: ONT (X) AE n Add: HOLD (X) n PUT DOWN(X) n Pre: HOLD (X) n Del: HOLD (X) n Add: ONT (X) AE n

3. Detecting a solution n A planning system find solution to a problem when it has found a sequence of operators that transforms the initial state into the goal state. n Predicate logic is an representation technique that serves as the basis for many of the planning systems.

4. Detecting dead ends n Planning system searching for operators must able to detect when it is exploring path that can never lead to solution(dead end). n If the search process is reasoning forward from initial state, it can prune any path that leads to a state from which the goal state cannat be reached. n If the search process is reasoning backward from initial state, it can terminate any path that leads to a state from which the goal state cannat be reached.

5. Repairing an almost correct solution n When the sequence of operations corresponding to the proposed solution is executed and to compare that situation to the desired goal.

Simple Planning using a Goal Stack (Goal Stack Planning) n One of the earliest techniques is planning using goal stack. n Problem solver uses single stack that contains sub goals and operators both n sub goals are solved linearly and then finally the conjoined sub goal is solved. n Plans generated by this method will contain n complete sequence of operations for solving one goal followed by complete sequence of operations for the next etc. n Problem solver also relies on n A database that describes the current situation. n Set of operators with precondition, add and delete lists. n

Algorithm n Let us assume that the goal to be satisfied is: GOAL = G 1 G 2 … Gn n Sub-goals G 1, G 2, … Gn are stacked with compound goal G 1 G 2 … Gn at the bottom. Top Bottom G 1 G 2 : Gn G 1 G 2 … G 4 n At each step of problem solving process, the top goal on the stack is pursued.

Algorithm - Contd… n Find an operator that satisfies sub goal G 1 (makes it true) and replace G 1 by the operator. n If more than one operator satisfies the sub goal then apply some heuristic to choose one. n In order to execute the top most operation, its preconditions are added onto the stack. n Once preconditions of an operator are satisfied, then we are guaranteed that operator can be applied to produce a new state. n New state is obtained by using ADD and DELETE lists of an operator to the existing database. n Problem solver keeps tract of operators applied. n This process is continued till the goal stack is empty and problem solver returns the plan of the problem.

Goal stack method - Example n Logical representation of Initial and Goal states: n n Initial State: ON(B, A) ONT(C) ONT(A) ONT(D) CL(B) CL(C) CL(D) AE Goal State: ON(C, A) ON(B, D) ONT(A) ONT(D) CL(C) CL(B) AE

Cont. . n We notice that following sub-goals in goal state are also true in initial state. ONT(A) ONT(D) CL(C) CL(B) AE n Represent for the sake of simplicity - TSUBG. n Only sub-goals ON(C, A) & ON(B, D) are to be satisfied and finally make sure that TSUBG remains true. n Either start solving first ON(C, A) or ON(B, D). Let us solve first ON(C, A). Goal Stack: ON(C, A) ON(B, D) ON(C, A) ON(B, D) TSUBG

Cont. . . n To solve ON(C, A), operation S(C, A) could only be applied. n So replace ON(C, A) with S(C, A) in goal stack. Goal Stack: S (C, A) ON(B, D) ON(C, A) ON(B, D) TSUBG n S(C, A) can be applied if its preconditions are true. So add its preconditions on the stack. Goal Stack: CL(A) HOLD(C) Preconditions of STACK CL(A) HOLD(C) S (C, A) Operator ON(B, D) ON(C, A) ON(B, D) TSUBG

Cont… n Next check if CL(A) is true in State_0. n Since it is not true in State_0, only operator that could make it true is US(B, A). n So replace CL(A) with US(B, A) and add its preconditions. Goal Stack: ON(B, A) CL(B) Preconditions of UNSTACK AE ON(B, A) CL(B) AE US(B, A) Operator HOLD(C) CL(A) ) HOLD(C) S (C, A) Operator ON(B, D) ON(C, A) ON(B, D) TSUBG

Contd… n ON(B, A), CL(B) and AE are all true in initial state, so pop these along with its compound goal. n Next pop top operator US(B, A) and produce new state by using its ADD and DELETE lists. n Add US(B, A) in a queue of sequence ofoperators. SQUEUE = US (B, A) State_1: ONT(A) ONT(C) ONT(D) HOLD(B) CL(A) CL(C) CL(D) Goal Stack: HOLD(C) CL(A) ) HOLD(C) S (C, A) Operator ON(B, D) ON(C, A) ON(B, D) TSUBG

Cont… To satisfy the goal HOLD(C), two operators can be used e. g. , PU(C ) or US(C, X), where X could be any block. Let us choose PU(C ) and proceed further. n Repeat the process. Change in states is shown below. State_1: ONT(A) ONT(C) ONT(D) HOLD(B) CL(A) CL(C) CL(D) SQUEUE = US (B, A) n Next operator to be popped of is S(B, D). So State_2: ONT(A) ONT(C) ONT(D) ON(B, D) CL(A) CL(C) CL(B) AE SQUEUE = US (B, A), S(B, D) State_3: ONT(A) HOLD(C) ONT(D) ON(B, D) CL(A) CL(B) SQUEUE = US (B, A), S(B, D), PU(C ) State_4: ONT(A) ON(C, A) ONT(D) ON(B, D) CL(C) CL(B) AE SQUEUE = US (B, A), S(B, D), PU(C ), S(C, A) n

Difficult Problem n The Goal stack method is not efficient for difficult problems such as Sussman anomaly problem. n It fails to find good solution. n Let us consider the Sussman anomaly problem

Cont… Initial State: ON(C, A) ONT(B) Goal State: ON(A, B) ON(B, C) n Remove CL and AE predicates for the sake of simplicity. n To satisfy ON(A, B), following operators are applied US(C, A) , PD(C), PU(A) and S(A, B)

Cont… State_1: ON(B, A) ONT(C) n To satisfy ON(B, C), following operators are applied US(A, B) , PD(A), PU(B) and S(B, C) State_2: ON(B, C) ONT(A)

Cont… n Finally satisfy combined goal ON(A, B) ON(B, C). n Combined goal fails as while satisfying ON(B, C), we have undone ON(A, B). n Difference in goal and current state is ON(A, B). n Operations required are PU(A) and S(A, B)

Final Solution n The complete plan for solution is as follows: 1. US(C, A) 2. PD (C) 3. PU(A) 4. S(A, B) 5. US(A, B) 6. PD(A) 7. PU(B) 8. S(B, C) 9. PU(A) 10. S(A, B) n Although this plan will achieve the desired goal, but it is not efficient.

Cont… n In order to get efficient plan, either repair this plan or use some other method. n Repairing is done by looking at places where operations are done and undone immediately, such as S(A, B) and US(A, B). n By removing them, we get 1. 2. 3. 4. 5. 6. US(C, A) PD (C) PU(B) S(B, C) PU(A) S(A, B)

STRATEGIC EXPLANATIONS n The expert systems are the computer applications developed to solve complex problems , at the level of extra-ordinary human intelligence and expertise. n expert systems have two capabilities to perform its task i)explain its reasoning - reasoning process proceed in understandable steps and enough meta knowledge be available so the explanations of those steps can be generated. ii) Acquire new knowledge and modifications of old knowledge

why, why not and how explanation n The explanation module provides the user with an explanation of the reasoning process when requested. n This is done in response to a how query or a why query. To respond to a how query n explanation module traces the chain of rules fired during a consultation with the user. n Chain of rules are printed for the user in an easy to understand humanlanguage style. n Reasoning process are viewed by user n If user not agree with the reasoning steps presented, they may be changed using the editor.

To respond to a why query n explanation module must able to explain why certain information is needed by the inference engine to complete a step in the reasoning process before it can proceed.

Learning

What is Learning? n “Learning denotes changes in a system that. . . enable a system to do the same task … more efficiently the next time. ” - Herbert Simon n “Learning is constructing or modifying representations of what is being experienced. ” - Ryszard Michalski n “Learning is making useful changes in our minds. ” Marvin Minsky “Machine learning refers to a system capable of the autonomous acquisition and integration of knowledge. ”

n Two features of learning are 1. skill refinement 2. knowledge acquisition

example

Machine Learning n Machine collects data and recognizes patterns in the data n Algorithms – sequence of instructions to transform the input into output n Intelligent systems have the ability to learn in a changing environment Alpaydin (2010)

Why Machine Learning? n No human experts n n industrial/manufacturing control mass spectrometer analysis, drug design, astronomic discovery n Black-box human expertise n n face/handwriting/speech recognition driving a car, flying a plane n Rapidly changing phenomena n n credit scoring, financial modeling diagnosis, fraud detection n Need for customization/personalization n n personalized news reader movie/book recommendation

Related Fields data mining control theory statistics information theory machine learning decision theory cognitive science databases psychological models evolutionary neuroscience models Machine learning is primarily concerned with the accuracy and effectiveness of the computer system.

Machine Learning Paradigms n rote learning n learning by being told (advice-taking) n learning from examples (induction) n learning by analogy n speed-up learning n concept learning n clustering n discovery n …

Adaptive Learning

Adaptive Learning • A system which collects user information and behavioral data to customize a learning experience for an individual • Encourages active participation rather than passive receptacle • Moves away from static hypermedia (same page content and links for all users) • Artificial Intelligence movement Brusilovsky (2001)

This…

Not this…

Adaptation Process Brusilovsky & Maybury (2002)

Modeling Jacko (2009)

Important Question n n n Explain STRIPS concept or mechanism in detail with example? **** (16 marks or 8 marks) Explain Goal based planning or goal stack planning with example? ** (8 marks) Explain various types or classification of learning with example? (8 marks) Explain in detail Machine learning? *** (16 marks or 8 marks) Machine learning algorithm? (8 marks) Explain in detail Adaptive learning ? *** (16 marks or 8 marks) Block world problem using Goal based planning ** (8 marks) Sussman anomaly problem (8 marks) What is meant by strategic explanations? Explain in detail? * (8 marks) Explain some basic plan generation system? (STRIP, goal stack planning) (16 marks or 8 marks) Explain some advanced plan generation system? (non-linear planning, hierarchical planning) (16 marks or 8 marks) Explain K- STRIPS ?

n n n n n n Define planning? * What are the components of planning? List out various planning techniques? Define STRIP mechanism? What are three list of STRIP used in operation? Define goal stack planning? What is meant by non-linear planning? Name the heuristics for planning using constraint posting (TWEAK)? Define declobbering? Define hierarchical planning? Define strategic explanation? Define learning? Name the two features of learning? List out the types or classification of learning? Define rote learning or base learning or learning by memorization? What is meant by induction learning? Define learning by analogy? Define learning by deduction? Define machine learning? ** Name the machine learning algorithms/ Define supervised learning? Define unsupervised learning? Define reinforcement learning? List out the application of machine learning?

n Define clustering? n Define adaptive learning? ** n What are advantage of adaptive learning?

n STRIP n Machine learning n Adaptive learning n Goal stack planning n Block world problem