Gheorghe Tecuci with Mihai Boicu Dorin Marcu Bogdan

Gheorghe Tecuci with Mihai Boicu, Dorin Marcu, Bogdan Stanescu, Cristina Boicu, Marcel Barbulescu Learning Agents Center George Mason University Symposium on Reasoning and Learning in Cognitive Systems Stanford, CA, 20 -21 May 2004

Overview Research Problem, Approach, and Application Problem Solving Method: Task Reduction Learnable Knowledge Representation: Plausible Version Spaces Multistrategy Learning during Problem Solving Agent Development Experiments Teaching and Learning Demo Acknowledgements

Research Problem and Approach Research Problem: Elaborate a theory, methodology and family of systems for the development of knowledge-base agents by subject matter experts, with limited assistance from knowledge engineers. Approach: Develop a learning agent that can be taught directly by a subject matter expert while solving problems in cooperation. 2. Teaching and learning 3. Multistrategy learning Interface 1. Mixed-initiative problem solving The expert teaches the agent to perform various tasks in a way that resembles how the expert would teach a person. The agent learns from the expert, building, verifying and improving its knowledge base Problem Solving Ontology + Rules Learning

Sample Domain: Center of Gravity Analysis Centers of Gravity: Primary sources of moral or physical strength, power or resistance of the opposing forces in a conflict. Application to current war scenarios (e. g. War on terror, Iraq) with state and non-state actors (e. g. Al Qaeda). Identify COG candidates Test COG candidates Identify potential primary sources of moral or physical strength, power and resistance from: Test each identified COG candidate to determine whether it has all the necessary critical capabilities: Government Military Which are the critical capabilities? People Are the critical requirements of these capabilities satisfied? Economy If not, eliminate the candidate. Alliances If yes, do these capabilities have any vulnerability? Etc.

Overview Research Problem, Approach, and Application Problem Solving Method: Task Reduction Learnable Knowledge Representation: Plausible Version Spaces Multistrategy Learning during Problem Solving Agent Development Experiments Teaching and Learning Demo Acknowledgements

Problem Solving: Task Reduction T 1 Q 1 S 1 A complex problem solving task is performed by: • successively reducing it to simpler tasks; • finding the solutions of the simplest tasks; • successively composing these solutions until the solution to the initial task is obtained. A 11 S 11 … T 11 a S 11 a T 11 b. S 11 b … Q 11 b S 11 b A 1 n S 1 n T 1 n … A 11 b 1 S 11 b 1… A 11 bm S 11 bm T 11 b 1 T 11 bm Let T 1 be the problem solving task to be performed. Finding a solution is an iterative process where, at each step, we consider some relevant information that leads us to reduce the current task to a simpler task or to several simpler tasks. The question Q associated with the current task identifies the type of information to be considered. The answer A identifies that piece of information and leads us to the reduction of the current task.

We need to Identify and test a strategic COG candidate for Sicily_1943 Which is an opposing_force in the Sicily_1943 scenario? Allied_Forces_1943 COG Analysis: World War II at the time of Sicily 1943 Therefore we need to Identify and test a strategic COG candidate for Allied_Forces_1943 Is Allied_Forces_1943 a single_member_force or a multi_member_force? Allied_Forces_1943 is a multi_member_force Therefore we need to Identify and test a strategic COG candidate for Allied_Forces_1943 which is a multi_member_force What type of strategic COG candidate should I consider for this multi_member_force? I consider a candidate corresponding to a member of the multi_member_force Therefore we need to Identify and test a strategic COG candidate corresponding to a member of the Allied_Forces_1943 Which is a member of Allied_Forces_1943? US_1943 Therefore we need to Identify and test a strategic COG candidate for US_1943

Overview Research Problem, Approach, and Application Problem Solving Method: Task Reduction Learnable Knowledge Representation: Plausible Version Spaces Multistrategy Learning during Problem Solving Agent Development Experiments Teaching and Learning Demo Acknowledgements

Knowledge Base: Object Ontology + Rules Object Ontology A hierarchical representation of the objects and types of objects. A hierarchical representation of the types of features.

Knowledge Base: Object Ontology + Rules We need to Identify and test a strategic COG candidate corresponding to a member of the Allied_Forces_1943 Which is a member of Allied_Forces_1943? US_1943 EXAMPLE OF REASONING STEP Therefore we need to Identify and test a strategic COG candidate for US_1943 LEARNED RULE IF Identify and test a strategic COG candidate corresponding to a member of the ? O 1 Question Which is a member of ? O 1 ? Answer ? O 2 THEN Identify and test a strategic COG candidate for ? O 2 INFORMAL STRUCTURE IF Identify and test a strategic COG candidate corresponding to a member of a force The force is ? O 1 Plausible Upper Bound Condition ? O 1 is multi_member_force has_as_member ? O 2 is force Plausible Lower Bound Condition ? O 1 is equal_partners_multi_state_alliance has_as_member ? O 2 is single_state_force THEN Identify and test a strategic COG candidate for a force The force is ? O 2

Learnable knowledge representation Use of the object ontology as an incomplete and evolving generalization language. Use of plausible version spaces to represent and use partially learned knowledge: • Rules with PVS conditions • Tasks with PVS conditions • Object features with PVS concept • Task features with PVS concept Plausible version space (PVS) Universe of Instances Plausible Upper Bound Concept Plausible Lower Bound Feature Domain: PVS concept Range: PVS concept

Overview Research Problem, Approach, and Application Problem Solving Method: Task Reduction Learnable Knowledge Representation: Plausible Version Spaces Multistrategy Learning during Problem Solving Agent Development Experiments Teaching and Learning Demo Acknowledgements

Control of modeling, learning and problem solving Input Task Mixed. Initiative Problem Solving Ontology + Rules Generated Reduction New Reduction Solution Modeling Learning Accept Reduction Reject Reduction Task Refinement Rule Refinement

Disciple uses the learned rules in problem solving, and refines them based on expert’s feedback. We need to Identify and test a strategic COG candidate corresponding to a member of the Allied_Forces_1943 1 Which is a member of Allied_Forces_1943? Provides an example Learns Rule_15 US_1943 Therefore we need to Identify and test a strategic COG candidate for US_1943 Modeling 2 Learning … Problem Solving Refining 3 We need to Identify and test a strategic COG candidate corresponding to a member of the European_Axis_1943 Applies Rule_15 4 Accepts the example Which is a member of European_Axis_1943? ? Germany_1943 Therefore we need to Identify and test a strategic COG candidate for Germany_1943 5 Refines Rule_15

Rule learning method Analogy and Hint Guided Explanation Analogy-based Generalization Plausible version space rule PUB Example of a task reduction step PLB Incomplete justification analogy Knowledge Base

Find an explanation of why the example is correct Which is a member of Allied_Forces_1943? US_1943 I need to Identify and test a strategic COG candidate corresponding to a member of the Allied_Forces_1943 Therefore I need to Identify and test a strategic COG candidate for US_1943 The explanation is the best possible approximation of the question and the answer, in the object ontology. Allied_Forces_1943 has_as_member US_1943

Generate the PVS rule Allied_Forces_1943 has_as_member US_1943 IF Identify and test a strategic COG candidate corresponding to a member of a force The force is ? O 1 Rewrite as Most generalization Condition ? O 1 is Allied_Forces_1943 has_as_member ? O 2 is US_1943 Most specific generalization explanation ? O 1 has_as_member ? O 2 Plausible Upper Bound Condition ? O 1 is multi_member_force has_as_member ? O 2 is force Plausible Lower Bound Condition ? O 1 is equal_partners_multi_state_alliance has_as_member ? O 2 is single_state_force THEN Identify and test a strategic COG candidate for a force The force is ? O 2

Rule refinement method Knowledge Base Learning by Analogy And Experimentation Condition <condition 1> PVS Rule Failure explanation Example of task reductions generated by the agent Incorrect example IF <task> Correct example Learning from Explanations Learning from Examples Except when condition <condition 2> … Except when condition <condition n> THEN <subtask 1> … <subtask m>

Overview Research Problem, Approach, and Application Problem Solving Method: Task Reduction Learnable Knowledge Representation: Plausible Version Spaces Multistrategy Learning during Problem Solving Agent Development Experiments Teaching and Learning Demo Acknowledgements

Agent Development Methodology Modeling the problem solving process of the subject matter expert and development of the object ontology of the agent. Teaching of the agent by the subject matter expert.

Use of Disciple at the US Army War College 319 jw Case Studies in Center of Gravity Analysis Disciple helps the students to perform a Disciple was taught based on the center of gravity analysis of an expertise of Prof. Comello in center of assigned war scenario. gravity analysis. Teaching Problem Disciple Learning Agent KB solving Global evaluations of Disciple by officers from the Spring 03 course The use of Disciple is an assignment that is well suited to the course's learning objectives Disciple helped me to learn to perform a strategic COG analysis of a scenario Disciple should be used in future versions of this course

Use of Disciple at the US Army War College 589 jw Military Applications of Artificial Intelligence course Students teach test the Disciple their trained COG analysis Disciple expertise, using agent based sample scenarios on a new (Iraq 2003, War on scenario terror 2003, Arab(North Israeli 1973) Korea 2003) Global evaluations of Disciple by officers during three experiments I think that a subject matter expert can use Disciple to build an agent, with limited assistance from a knowledge engineer Spring 2001 COG identification Spring 2002 COG identification and testing Spring 2003 COG testing based on critical capabilities

Parallel development and merging of KBs Initial KB Domain analysis and ontology development (KE+SME) 37 acquired concepts Extended KB and features for COG testing DISCIPLE-COG informed be irreplaceable 5 features 10 tasks rules KB 10 merging Knowledge Engineer (KE) All subject matter experts (SME) Parallel KB development (SME assisted by KE) Team 1 stay 432 concepts and features, 29 tasks, 18 rules For COG identification for leaders DISCIPLE-COG Team 2 communicate 14 tasks 14 rules DISCIPLE-COG Team 3 be influential 2 features 19 tasks 19 rules (KE) Unified 2 features Deleted 4 rules Refined 12 rules Final KB: +9 features 478 concepts and features +105 tasks 134 tasks +95 rules 113 rules Correctness = 98. 15% Integrated KB DISCIPLE-COG Training scenarios: Iraq 2003 Arab-Israeli 1973 War on Terror 2003 DISCIPLE-COG Team 4 DISCIPLE-COG be protected be driving force 35 tasks 3 33 rules features 24 tasks 23 rules Learned features, tasks, rules have support Team 5 5 h 28 min average training time / team 3. 53 average rule learning rate / team COG identification and testing (leaders) Testing scenario: North Korea 2003

Other Disciple agents Disciple-WA (1997 -1998): Estimates the best plan of working around damage to a transportation infrastructure, such as a damaged bridge or road. Demonstrated that a knowledge engineer can use Disciple to rapidly build and update a knowledge base capturing knowledge from military engineering manuals and a set of sample solutions provided by a subject matter expert. Disciple-COA (1998 -1999): Identifies strengths and weaknesses in a Course of Action, based on the principles of war and the tenets of army operations. Demonstrated the generality of its learning methods that used an object ontology created by another group (TFS/Cycorp). Demonstrated that a knowledge engineer and a subject matter expert can jointly teach Disciple.

Overview Research Problem, Approach, and Application Problem Solving Method: Task Reduction Learnable Knowledge Representation: Plausible Version Spaces Multistrategy Learning during Problem Solving Agent Development Experiments Teaching and Learning Demo Acknowledgements

Acknowledgements This research was sponsored by the Defense Advanced Research Projects Agency, Air Force Research Laboratory, Air Force Material Command, USAF under agreement number F 30602 -00 -2 -0546, by the Air Force Office of Scientific Research under grant number F 4962000 -1 -0072 and by the US Army War College.