An Agent Oriented Programming Language integrating Temporal Planning

An Agent Oriented Programming Language integrating Temporal Planning and the Plan Coordination Mechanisms Dr. Muhammad Adnan Hashmi

Outline n Introduction Ø Ø n Plan Coordination Mechanisms Ø Ø n Coordinated Planning Problem Proactive-Reactive Coordination Problem P-CLAIM: AOP Language supporting Temporal Planning Ø Ø n Background Problem Overview Language Definition Planning Mechanism Conclusion 2

Outline n Introduction Ø Ø n Plan Coordination Mechanisms Ø Ø n Coordinated Planning Problem Proactive-Reactive Coordination Problem P-CLAIM: AOP Language supporting Temporal Planning Ø Ø n Background Problem Overview Language Definition Planning Mechanism Conclusion 3

What is an Agent? n An agent is a computer system capable of autonomous action in some environment in order to meet its design objectives (goals). n Achievement of goals is important for agents 4

Planning n Finding a sequence of actions that takes agent from Initial State to Goal State Initial state: clear(c), on(c, a), ontable(a), clear(b), ontable(b), handempty c a b c c a a Goal: on(a, b), b on(b, c) b a c b c a b c 5

Multi-Agent Planning n Multiple agents sharing the same environment Ø Ø n Independent planning is not sufficient Coordination of plans is vital Coordination of plans Ø Ø Removing conflicts (negative interactions) Utilizing help relations (positive interactions) A DA B DB 6

How to Program Agents? n Agent Oriented Programming (AOP) Languages allow to program intelligent and autonomous agents n Main Characteristics Ø Ø n Mental State: Beliefs, Goals, Commitments Reasoning Mechanism Capabilities, Services Communication Some Languages Ø Agent-0 [Shoham 1993], 2 APL [Dastani 2008], Agent. Speak (L) [Rao 1996] 7

Outline n Introduction Ø Ø n Plan Coordination Mechanisms Ø Ø n Coordinated Planning Problem Proactive-Reactive Coordination Problem P-CLAIM: AOP Language supporting Temporal Planning Ø Ø n Background Problem Overview Language Definition Planning Mechanism Conclusion 8

Problem Statement n Current AOP languages Ø Ø n Follow a reactive (PRS based) approach Do not support temporal planning n Only a few support planning Problems Ø Ø Ø Execution without planning may result in the goal failures Ø Agent can reach a dead end Conflicts can arise among different agents Actions’ duration is important n Real world actions take place over a timespan 9

Objectives n Propose a programming language that endows agents with planning skills n n Has temporal planning Deals with uncertainty of the environment Incorporate reactivity by dealing with on the fly goals having different priorities Propose coordination mechanisms for the plans having different priorities 10

Outline n Introduction Ø Ø n Plan Coordination Mechanisms Ø Ø n Coordinated Planning Problem Proactive-Reactive Coordination Problem P-CLAIM: AOP Language supporting Temporal Planning Ø Ø n Background Problem Overview Language Definition Planning Mechanism Conclusion 11

Assumptions n Two agents α and β sharing the same environment Ø Ø n Actions have: Ø Ø Ø n Agent α having higher priority (reactive) goals Agent β having normal priority (proactive) goals Preconditions Add effects Delete effects Two possible conflicts among plans Ø Ø Causal link threat Parallel actions interference 12

Two Possible Conflicts n Causal Link (A 1, A 2, p) Ø Ø Ø n Action A 1 adds an effect p Action A 2 needs this effect No action between A 1 and A 2 adding p Causal Link Threat Ø If an action A deletes p and lies between A 1 and A 2, then A threatens the causal link (A 1, A 2, p) p A 1 A 2 Threat A ¬p 13

Two Possible Conflicts n Causal Link (A 1, A 2, p) Ø Ø Ø n Action A 1 adds an effect p Action A 2 needs this effect No action between A 1 and A 2 adding p Parallel Actions Interference Ø Ø Actions A 1 and A 2 lie in parallel Either one of them deletes the preconditions or add effects of the other A 1 p ¬p A 2 A 1 p A 2 ¬p 14

Outline n Introduction Ø Ø n Plan Coordination Mechanisms Ø Ø n Coordinated Planning Problem Proactive-Reactive Coordination Problem P-CLAIM: AOP Language supporting Temporal Planning Ø Ø n Background Problem Overview Language Definition Planning Mechanism Conclusion 15

Coordinated Planning Problem n Prerequisite: Ø n Plan Pα of Agent α Our Aim: Ø Compute a Plan Pβ for Agent β n n n Has no conflict with Pα Avails the cooperative opportunities offered by Pα Solution: Ø Ø Non Temporal Domains µ-SATPLAN Temporal Domains Coordinated-Sapa 16

SATPLAN [Kautz and Selman 2006] A Classical Planner that Finds Optimal Plans in Non-Temporal Domains

SATPLAN We Use Graph. Plan Encoding Planning Problem • Init State • Goal • Actions Plan Propositional formula in conjunctive normal form (CNF) Compiler (encoding) Simplifier (polynomial inference) CNF Increment plan length If unsatisfiable Decoder satisfying model CNF Solver (SAT engine/s) 18

Constructing the planning graph n n Level P 1: All literals from the initial state Add an action in level Ai if all its preconditions are present in level Pi Add a proposition in level Pi if it is the effect of some action in level Ai-1 Maintain a set of exclusion relations to eliminate incompatible propositions and actions 19

Graph. Plan Encoding Pre 1 Act 1 Fact Pre 2 Act 2 Fact Act 1 Act 2 Act 1 Pre 2 ¬Act 1 ¬Act 2 n n Can create such constraints for every node in the planning graph Only involves facts and actions in the graph 20

µ-SATPLAN An Extension of SATPLAN that Computes Coordinated Plans in Non. Temporal Domains

Handling Causal Link Threat n While constructing the planning graph for Agent β, don’t add an action O at level Ai if Ø Ø Ø It has an effect ¬p, and There is a causal link (Aj, Ak, p) in plan Pα, and j≤i≤k Action O threatens Causal Link (Aj, Ak, p) 22

Handling Positive Interactions & Parallel Actions Interference n For each time step i in the plan of Agent α, create an action No. Name(i) Ø Add(No. Name(i)) All the effects added by Pα at time step i Ø Del(No. Name(i)) All the effects deleted by Pα at time step i Ø n Pre(No. Name(i)) All the preconditions of actions of Pα at time step i Explicitly add all the No. Name actions in the planning graph of Agent β 23

Handling Positive Interactions n Pα = {α 1(0), α 2(0), α 3(1)} Eff(α 1) = a 0, Eff(α 2) = a 1, Eff(α 3) = a 2 n Eff( No. Name(0) ) = {a 0, a 1}, Eff( No. Name(1) ) = {a 2} n Planning Graph of Agent β Level 0 Level 1 a 0 No. Name(0) No. Name(1) a 1 Level 2 a 0 a 2 a 1 β 2 a 4 β 1 a 6 β 4 a 8 a 4 a 4 a 5 a 5 a 7 a 624 β 3 a 6

Handling Positive Interactions n Partial CNF Sentence Ø Ø Ø a 8 β 4 a 0 a 7 a 0 β 2 No. Name(0) a 7 β 3 β 2 a 5 … Level 0 n Add No. Name actions as unary clauses to CNF Ø Ø No. Name(0) No. Name(1) Solution Problem Level 1 a 0 No. Name(0) No. Name(1) a 1 Level 2 a 0 a 2 a 1 β 2 a 4 β 1 a 6 β 4 a 8 a 4 a 4 a 5 a 5 a 7 a 625 β 3 a 6

Example Plan Generated (Logistics) Pα Pβ Time Action Performed 0 DRIVE-TRUCK (TRUCK 1, LOC 1 -2, CITY 1) 0 LOAD-TRUCK (PACKAGE 4, TRUCK 2, LOC 2 -2) 1 LOAD-TRUCK (PACKAGE 1, TRUCK 1, LOC 1 -2) 1 1 DRIVE-TRUCK (TRUCK 2, LOC 2 -1, CITY 2) 1 LOAD-PLANE (PACKAGE 2, PLANE 1, LOC 1 -1) 2 DRIVE-TRUCK (TRUCK 1, LOC 1 -2, LOC 1 -1, CITY 1) 2 FLY-PLANE (PLANE 1, LOC 1 -1, LOC 2 -1) 3 UNLOAD-PLANE (PACKAGE 2, PLANE 1, LOC 2 -1) 3 LOAD-PLANE (PACKAGE 3, PLANE 1, LOC 2 -1) 3 UNLOAD-TRUCK (PACKAGE 1, TRUCK 1, LOC 1 -1) 4 FLY-PLANE (PLANE 1, LOC 2 -1, LOC 1 -1) 4 LOAD-PLANE (PACKAGE 1, PLANE 2, LOC 1 -1) 4 UNLOAD-TRUCK (PACKAGE 4, TRUCK 2, LOC 2 -1) 4 LOAD-TRUCK (PACKAGE 2, TRUCK 2, LOC 2 -1) 5 UNLOAD-PLANE (PACKAGE 3, PLANE 1, LOC 1 -1) 5 FLY-PLANE (PLANE 2, LOC 1 -1, LOC 2 -1) 6 LOAD-PLANE (PACKAGE 4, PLANE 2, LOC 2 -1) 5 DRIVE-TRUCK (TRUCK 2, LOC 2 -1, LOC 2 -2, CITY 2) 6 LOAD-TRUCK (PACKAGE 3, TRUCK 1, LOC 1 -1) 6 UNLOAD-PLANE (PACKAGE 1, PLANE 2, LOC 2 -1) 7 FLY-PLANE (PLANE 2, LOC 2 -1, LOC 1 -1) 6 UNLOAD-TRUCK (PACKAGE 2, TRUCK 2, LOC 2 -2) 7 DRIVE-TRUCK (TRUCK 1, LOC 1 -2, CITY 1) 8 UNLOAD-PLANE (PACKAGE 4, PLANE 2, LOC 1 -1) 8 UNLOAD-TRUCK (PACKAGE 3, TRUCK 1, LOC 1 -2) 26 2

Outline n Introduction Ø Ø n Plan Coordination Mechanisms Ø Ø n Coordinated Planning Problem Proactive-Reactive Coordination Problem P-CLAIM: AOP Language supporting Temporal Planning Ø Ø n Background Problem Overview Language Definition Planning Mechanism Conclusion 27

Proactive-Reactive Coordination n Prerequisite: Ø Ø n Reactive plan Pα of Agent α Proactive plan Pβ of Agent β Our Aim: Ø Modify plan Pβ such that: n n n It has no conflict with Pα Avails the cooperative opportunities offered by Pα Solution: Ø Plan Merging Algorithm 28

Case Study n Ø n Rescue the Victims Tasks of Analyzer Agent Ø Ø n Analyzer Agent : β Tasks of Rescue Agent : α Analyze the goal cells Call the central agent Constraints Ø Ø One agent in a cell Hyper energy cells n Ø Needs fuel or energy to enter Agent should have key to open door 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 A B C D 29

Conflict Resolution n Threat-Repair Link (A 1, A 2, p) Ø Ø Action A 1 deletes p A 2 is a subsequent action and adds p A 1 is called Threat Action A 2 is called Repair Action p B 1 B 2 Threat A 1 -p A 2 p Repair 30

Valid and Possibly Valid Time Slot for an action A Stamps All preconditions are met n Ø Ø a No parallel actions interference P[1] b h b P[2] c -d c b n P[3] e P[4] e f f P[5] g g P[6] i i h P[7] g d -h P[1] Valid Time Slot for an action A Ø Ø Ø All preconditions are met No parallel actions interference Either: n n a P[1] No causal link threat Repair Action exist before the deadline b h b P[2] c -d c b P[3] e P[1] e P[4] d -h d P[2] k k P[3] h 31

Plan Merging Algorithm n Fix all the actions of Reactive Plan Pα on timeline n For every action CA of Proactive Plan Ø Search for the first Possibly Valid Time Slot T on timeline Ø Reason about the time slot T n There could be 5 cases at T 32

Plan Merging Algorithm Case 1: No causal link threat by CA at T Ø Assign Time Slot T to CA EXAMPLE n Current Action: Move(A 1, A 2) Ø Ø Ø Returned Time Slot: 0 - 5 Any Threat? : No Assign Time Slot 0 – 5 to CA 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 A B C D 33

Plan Merging Algorithm Case 2: CA threatens a Causal Link but Repair Action exist Ø Ø Assign Time Slot T to CA Save a Possible Threat <Threat. Action, Repair. Action, Deadline> EXAMPLE n Current Action: Move(A 4, A 5) Ø Ø Ø Time Slot: 20 - 25 Any Threat? : Yes (Agent α needs A 5 at time 40 -45) Repair Action: Move(A 5, A 6) Assign Time Slot 20 - 25 to Move (A 4, A 5) Save <Move(A 4, A 5), Move(A 5, A 6), 40> 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 A B C D 34

Plan Merging Algorithm Case 3: It is a Repair Action but can not meet a deadline of some Threat Action Ø Backtrack to the Threat Action, find another time stamp EXAMPLE n Current Action: Move (A 8, A 9) Ø Ø n Returned Time Slot: 50 - 55 Any Threat? : Yes (Agent α needs A 9 at 85 -110) Repair Action : Move (A 9, B 9) Save <Move(A 8, A 9), Move(A 9, B 9), 85> Next Action: Analyze. Cell (A 9) Ø Time Slot Assigned: 55 - 70 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 A B C D 35

Plan Merging Algorithm n Next Action: Call. Central (A 9) Ø n Time Slot Assigned: 80 – 90 Attention Next Action: Move (A 9, B 9) Ø Ø Ø Is it a Repair Action? : Yes Meet all deadlines? : No (Agent α needs A 9 at 85) Backtrack to action Move(A 8, A 9) n Find another Time Slot n New Time Slot: 110 – 115 (Valid Time Slot) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 A B C D 36

Plan Merging Algorithm Case 4: All the effects of CA are already achieved WHAT TO DO? n Mark CA as redundant POST PROCESSING n n Remove all redundant actions from the plan Recursively remove all actions which achieve only the preconditions of removed action 37

Plan Merging Algorithm EXAMPLE n Current Action: Open. Door (C 11) Ø Ø Ø Returned Time Slot: 172 - 175 Redundant(Open. Door(C 11)) true n Because opened. Door(C 11) is true at time 172 When the plan is returned n Remove Open. Door(C 11) from plan n Also remove Take. Key(C 11, key 1) from plan 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 A B C D 38

Plan Merging Algorithm Case 5: Action CA’s preconditions can not be achieved Ø Remove action CA from the plan and compute a plan to achieve effects of CA n n Ø I = State just before CA G = Effects (CA) Use Coordinated-Sapa Plan should have no conflict with Reactive Plan Pβ and if CA is a repair action, repair effects must meet their deadline n Replacement. Plan = Coordinated-Sapa (I, G, Pβ) Ø If a plan is returned, replace the removed actions with the plan Ø If a deadline is violated, backtrack to the threat action Ø If no plan possible, then remove another action CA + 1 n G = G U Effects (CA + 1) Pre (CA + 1) 39

Plan Merging Algorithm EXAMPLE n n Current Action: Take. Energy(B 13, energy 1) Preconditions can not be achieved Ø Repair the plan 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 A B C D 40

Plan Repair Algorithm n Create a CPP by removing Take. Energy(B 13, energy 1) Ø Ø n I = { at(β, B 13), at(energy 1, B 13), at(energy 2, B 13) } G = { has. Energy(β, energy), at(β, B 13)} Call Coordinated-Sapa to solve this CPP Ø Ø Coordinated-Sapa returns fail Why? energy 2 is also needed by Agent α 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 A B C D 41

Plan Repair Algorithm n Create another CPP by removing Move(B 13, A 12) Ø Ø n I = { at(β, B 13), at(energy 1, B 13), at(energy 2, C 15) } G = { at(β, A 12) } Call Coordinated-Sapa to solve this CPP Ø A plan is returned to enter A 12 by taking the fuel from D 14 POST PROCESSING n This plan will become a replacement for both Take. Energy(B 13, energy 1) and Move(B 13, A 12) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 A B C D 42

Outline n Introduction Ø Ø n Plan Coordination Mechanisms Ø Ø n Coordinated Planning Problem Proactive-Reactive Coordination Problem P-CLAIM: AOP Language supporting Temporal Planning Ø Ø n Background Problem Overview Language Definition Planning Mechanism Conclusion 43

P-CLAIM n An AOP language having: Ø Ø n Cognitive aspects specific to intelligent agents Communication primitives Mobility primitives Temporal planning capability P-CLAIM Agent: Ø Ø Ø Is autonomous, intelligent and mobile Has a mental state containing knowledge, goals, and capabilities Is able to communicate with other agents Entails a planning based behaviour Achieves goals based on their priorities Maintains the stability of the plan in the dynamic environments 44

Defining P-CLAIM Agent define. Agent agent. Name{ parent = null | agent. Name ; knowledge = null ; | {knowledge 1; …; knowledgem} goals = null ; | {goal 1; … ; goaln} activities = null ; {activity 1 … activityo} actions = {action 1 … actionp} agents = null ; | {ag. Name 1, ag. Name 2, …, ag. Nameq} } 45

Activities define. Agent agent. Name{ parent = null | agent. Name ; knowledge = null ; | {knowledge 1; …; knowledgem} goals = null ; | {goal 1; … ; goaln} activities = null ; {activity 1 … activityo} actions = {action 1 … actionp} agents = null ; | {ag. Name 1, ag. Name 2, …, ag. Nameq} } activityity = name { message = null | message ; conditions = null | condition ; do { process } effects = null ; | { effect 1 ; …; effectf } } 46

Actions define. Agent agent. Name{ parent = null | agent. Name ; knowledge = null ; | {knowledge 1; …; knowledgem} goals = null ; | {goal 1; … ; goaln} activities = null ; {activity 1 … activityo} actions = {action 1 … actionp} agents = null ; | {ag. Name 1, ag. Name 2, …, ag. Nameq} } action = name { message = null | message ; conditions = null | condition ; do { process } duration = dur ; } 47

Outline n Introduction Ø Ø n Plan Coordination Mechanisms Ø Ø n Coordinated Planning Problem Proactive-Reactive Coordination Problem P-CLAIM: AOP Language supporting Temporal Planning Ø Ø n Background Problem Overview Language Definition Planning Mechanism Conclusion 48

Agent Definition to Planning (Translator) Agent Description File Knowledge Goals Activities Actions Translator (Java. CC) Initial State Goals Methods Problem File Operators Domain File Planner 49

Agent Life Cycle 50

Messages Handler 51

Messages Handler n Waits for messages from other agents n Message is a request to achieve a goal? Ø Ø Ø n Assigns priority to the goal Reactive Goal puts it in Global Reactive Goals (GRG) queue Proactive Goal puts it in Global Proactive Goals (GPG) priority queue Message is an information? Ø Store the information in the knowledge base of the agent 52

Planner 53

2 - GPG Goals are accessed only when GRG is empty Planner GRG GPG Main Algorithm 3 - Sends a suspension signal to Executor if the goal is reactive i. e. from GRG Compute Plan 1 - Fetch goals one by one from GRG and GPG and calls Compute_Plan to compute a plan for the goal Temporal Converter 54

Plan Computation n JSHOP 2 algorithm [Nau et el. 2003] is used to compute a totally ordered plan for each goal Ø Ø Ø T 1 M 1 An HTN planning algorithm T 11 T 12 Decomposes the task into sub-tasks by applying methods M 11 M 12 Recursively applies the same procedure on every composite sub-task until there are only primitive tasks 55

Temporal Converter n Input to procedure Ø Ø n A totally ordered plan Actions’ information (Add, Del, Pre, Durations) Output of procedure Ø A position constrained parallel plan n n Every action is assigned a time stamp Multiple actions can possibly lie in parallel 56

Temporal Converter (Example) Input Plan -a -f +d -h a P[1] f P[2] +g e P[3] +i h P[4] +j +e +h +f g P[5] +k i k P[6] +l l P[7] +m m P[8] +c -k c P[9] -c e P[10] -c m +h Makespan =150 Output Plan -a -f +d -h a P[1] f P[2] +g h P[4] +j +e +h +f e P[3] +i g P[5] +k i k P[6] +l l P[7] +m m P[8] c +n m P[9] -c +c n -k e P[10] -c +h Makespan =105 Makespan Gain: 30% 57

Merging the New Plan to Global Plan 58

Merging the New Plan to Global Planner Reactive Goal Merge at the start of Pexec Proactive Goal Append at the end of Pexec Plan Under Execution (Pexec) 59

Schedule Handler and Executor 60

Schedule Handler and Executor 61

Plan Mender 62

Plan Mender pa a pag S a S sin g in n U s U Pla n laute SW P e p Repplua. Ct om m cemen Plan n o o i t t a P C lan No G ntinu o C ) ] 4 Pla [ c SP(Pexe n c[3]) e x e P ( P S ) ] 2 (Pexec[ P S 1]) [ c e x e SP(P Pexecnew = Replacement. Plan + Continuation. Plan 63

Outline n Introduction Ø Ø n Plan Coordination Mechanisms Ø Ø n Coordinated Planning Problem Proactive-Reactive Coordination Problem P-CLAIM: AOP Language supporting Temporal Planning Ø Ø n Background Problem Overview Language Definition Planning Mechanism Conclusion 64

Conclusion n An agent oriented programming language supporting: Ø Ø Ø n Coordinated Planning Problem Ø n Temporal Planning Plan Repairing Dealing with different priority goals Computing plan while coordinating it with another plan n SATPLAN µ-SATPLAN n Sapa Coordinated-Sapa Proactive-Reactive Coordination Problem Ø Modifying a plan to remove conflicts with a higher priority plan n Plan Merging Algorithm 65

Publications 1. A. El Fallah Seghrouchni, M. A. Hashmi, “Multi-Agent Planning”, Book chapter in Software Agents, Agent Systems and Their Applications. M. Essaaidi et al. (Eds. ), 2012, IOS Press. 2. Y. Dimopoulos, M. A. Hashmi, P. Moraitis, “µ-SATPLAN: Multi-Agent Planning as Satisfiability”, Knowledge Based Systems Journal, 2011 3. M. A. Hashmi, A. El Fallah Seghrouchni, “Merging of Temporal Plans supported by Plan Repairing”, Proceedings of 22 nd IEEE International Conference on Tools with Artificial Intelligence (ICTAI 2010), Aras, France 4. M. A. Hashmi, A. El Fallah Seghrouchni, “Coordination of Temporal Plans for the Reactive and Proactive Goals”, Proceedings of IEEE/WIC/ACM International Conference on Intelligent Agent Technology (IAT’ 10), Toronto, Canada 5. Y. Dimopoulos, M. A. Hashmi, P. Moraitis, “Extending SATPLAN to Multiple Agents”, Proceedings of 30 th SGAI International Conference on Artificial Intelligence 2010, Cambridge, UK. 6. M. A. Hashmi, A. El Fallah Seghrouchni, “Temporal Planning in Dynamic Environments for P-CLAIM Agents”, In proceedings of Languages, Methodologies and Development Tools for Multi-Agent Systems (LADS’ 09), Torino, Italy, Springer-Verlag. 66

Publications 7. M. A. Hashmi, A. El Fallah Seghrouchni, “Temporal Planning in Dynamic Environments for Mobile Agents”, Proceedings of International Conference on Frontiers of Information Technology (FIT’ 09), Abottabad, Pakistan, Publisher ACM press. 8. M. A. Hashmi, “A Planning Component for CLAIM Agents”, In the proceedings of 17 th International Conference on Control Systems and Computer Science, Bucharest, Romania, Volume 2, pages 485 -492, Politehnica Press 67

Thanks for your attention!!! 68