Gheorghe Tecuci Learning Agents Center and Computer Science

Gheorghe Tecuci Learning Agents Center and Computer Science Department The Volgenau School of IT & E, George Mason University tecuci@gmu. edu, http: //lac. gmu. edu Apprentissage Artificiel : la Carte, le Territoire et l‘Horizon Paris, May 23 rd 2008 1

Overview Research Vision: From PC to LA Disciple Approach to Agents Development Disciple Principles for Learning Assistants Applications of Disciple Cognitive Assistants Final Remarks and Discussion 2

How Did it Start: Yves Kodratoff and Initial Milestones October 1982: Met Yves at AI Conference in Bratislava. Encouraged to do Machine Learning research. January 1986: First time in France and the first paper to use the name “Disciple”: Kodratoff Y. and Tecuci G. , "Rule Learning in Disciple, " in Proceedings of the First European Working Session on Learning, Orsay, 31 Jan - 2 Feb (Yves: Chair). July 1988: Thèse de Docteur en Science Disciple: A Theory, Methodology and System for Learning Expert Knowledge, Université de Paris-Sud (Yves: Director) February 1990: Joined Yves and Ryszard Michalski at George Mason University. 3

Learning Agents Center: Mission • Conducts basic and applied research on the development of knowledge-based learning and problem solving agents. Basic Research Applied Research Tools Applications Transitions • Supports teaching in the areas of intelligent agents, machine learning, knowledge acquisition, artificial intelligence and its applications. • Develops and applies the Disciple theory, methodology and agent shells for building agents that can be taught how to solve problems by subject matter experts. Main Collaborators: Mihai Boicu, David Schum, Dorin Marcu, Vu Le, Cristina Boicu, Marcel Barbulescu, Jerome Comello, Cindy Ayers, Tom Dybala, Michael Bowman, … 4

Teaching as Alternative to Programming Building an intelligent machine by programming is too difficult. “Instead of trying to produce a program to simulate the adult mind, why not rather try to produce one which simulates the child's? If this were then subjected to an appropriate course of education one would obtain the adult brain. ” Alan Turing Computing Machinery and Intelligence Mind, 59, 433 -460, 1950. 5

How Agents Are Built and Why It is Hard Traditional Approach: Agent Development by Knowledge Engineer Ed Feigenbaum, Expert Agent Subject Matter Expert Knowledge Engineer Inference Engine Dialog Programming Results Knowledge Base AAAI-93: Rarely does a technology arise that offers such a wide range of important benefits. Yet … Another Approach: Agent Training by Subject Matter Expert Learning Agent Subject Matter Expert Inference Engine Dialog Learning Engine Knowledge Base Bill Gates (NYT, 1 March 2004): If you invent a breakthrough in artificial intelligence, so machines can learn, that is worth 10 Microsofts. 6

Disciple Approach to Agent Development Develop learning and problem solving agents that can be taught by subject matter experts to become knowledge-based assistants. The expert teaches the agent how to solve problems in a way that resembles how the expert would teach a student, an apprentice or a collaborator. DISCIP L E The agent continuously develops and refines its knowledge base to capture and better represent expert’s knowledge and problem solving strategies. There is no longer a clear distinction between knowledge base development and its maintenance. 7

Multidisciplinarity and Integration “The multidisciplinary approach to problems has become common practice. However, this will not be sufficient if we will not find the integrative factors to melt the multidisciplinarity into a unity, be it the case of understanding a complex reality, or that of achieving goals serving the people and society. ” Mihai Drăgănescu The Ring of the Material World, 1989. 8

Multidisciplinarity and Integration in Disciple Working closely with subject matter experts to model their reasoning DISCIPLE & on ati ns uc tio Ed era Op Development of systematic approach to expert problem solving Mo Ex deli Re pe ng as rt’s on ing Intelligence analysis, Center of gravity determination, Course of action critiquing, Emergency response planning, Workaround reasoning, Ph. D advisor selection, Teaching higher order thinking skills. Disciple Learning Agents Research Army War College Air War College George Mason University Development and application of Disciple agents Working closely with end users to receive crucial and timely feedback Development of the Disciple theory for agents teaching by noncomputer experts 9

Lifecycle of a Disciple agent Development of an agent shell Disciple Knowledge engineer 1 Agent teaching by the expert 2 Disciple Knowledge base integration and optimization Disciple 6 Disciple Knowledge engineer and subject matter expert Subject matter expert and knowledge engineer User teaching by the agent 3 5 Disciple User After action review and agent Disciple personalization User 4 Agent use and non-disruptive learning Users 10

Vision 1: Evolution of Software Development and Use Mainframe Computers Personal Computers Learning Assistants Software systems developed and used by persons who are not computer experts Software systems developed and used by computer experts Software systems developed by computer experts and used by persons who are not computer experts DISCIPLE 11

Vision 2: Use of Disciple in Education Disciple Agent KB The subject matter expert teaches Disciple similarly to how she or he would teach a student. … teaches Disciple Agent KB teaches Disciple teaches the students similarly to how it was taught. Personalized Learning: Grand Challenge for the 21 st Century US National Academy of Engineering, February 15 th, 2008 12

Overview Research Vision: From PC to LA Disciple Approach to Agents Development Disciple Principles for Learning Assistants Applications of Disciple Cognitive Assistants Final Remarks and Discussion 13

Disciple-LTA: Analyst’s Cognitive Assistant Analytic Assistance Empowers the analysts through mixed-initiative reasoning for hypotheses analysis, collaboration with other analysts and experts, and sharing of information. Learning Rapid acquisition and maintenance of subject matter expertise in intelligence analysis which currently takes years to establish, is lost when experts separate from service, and is costly to replace. Tutoring Helps new intelligence analysts learn the reasoning processes involved in making intelligence judgments and solving intelligence analysis problems. 14

Knowledge Base = Object Ontology + Rules The ontology is a hierarchical description of the domain objects. 15

Knowledge Base = Ontology + Rules specify general problem reduction or solution synthesis steps Analysis Tree P 1 S 1 P 11 S 11 … P 1 n S 1 n P 111 S 111 … P 11 m S 11 m Partially learned rule To assess whethere are states that may be willing to sell nuclear weapons to an actor, one has to consider each nuclear state and assess whether that state may be willing to sell nuclear weapons to that actor, except for the case in which the nuclear state is an enemy of that actor and also except for the case when the nuclear state opposes the proliferation of nuclear weapons. 16

Disciple Agent Development Approach Model the reasoni ng of SME Create object ontolog y KE SME Define reasoni ng rules Verify and update rules Traditionally With Disciple Instruct SME to explicitate reasoning KE SME Import and develop initial ontology KE SME Agent Provide and explain solutions SME Agent Develop reasoning trees SME Agent Specify Learn instances ontological and elements features SME Agent Learn reasoning rules Agent Analyze Agent’s solutions SME Explain errors SME Agent Refine rules Agent 17

Overview Research Vision: From PC to LA Disciple Approach to Agents Development Disciple Principles for Learning Assistants Applications of Disciple Cognitive Assistants Final Remarks and Discussion 18

Disciple Principles for Learning Assistants 1. Problem Solving Paradigm for Expert-Agent Collaboration 2. Multi-Agent and Multi-Domain Problem Solving 3. User Tutoring in Problem Solving 4. Integrated Mixed-Initiative Reasoning 5. Integrated Teaching and Learning 6. Multistrategy Learning 7. Learning within Evolving Representation Space 8. Plausible Reasoning with Partially Learned Knowledge 9. Agent Architecture for Generality-Power Tradeoff 10. Knowledge Base Structuring for Knowledge Reuse 19

P 1: Problem Solving Paradigm for Expert-Agent Collaboration Use a general problem solving paradigm which is: ○ natural for the human user; ○ appropriate for the automated agent. The reductions and synthesis operations are guided by introspective questions and answers. Assess whether Al Qaeda has nuclear weapons. P 1 S 1 It is likely that Al Qaeda has nuclear weapons. P 11 S 11 … P 1 n S 1 n "I Keep Six Honest. . . " I keep six honest serving-men (They taught me all I knew); Their names are What and Why and When And How and Where and Who. Rudyard Kipling P 111 S 111 … P 11 m S 11 m Pa 11 m Sa 11 m … Pd 11 m Sd 11 m 20

Problem Reduction: Illustration Main problems Abstract tree Reduction of a main problem to its main subproblems Detailed tree 21 21

Problem Reduction and Solution Synthesis Detailed evidence and source analysis EVD-Dawn-Mir-0101 c 22

Solution Synthesis: Illustration Analyzed problem Synthesized solution This is only an example — not to be taken as a current analytic estimate Disciple-LTA makes very clear: The analysis logic; What evidence was used and how; What assumptions have been made; What is not known. 23

Assumptions-based Analysis 2. Revised, assumptionbased, solution Disciple-LTA allows for: Assumptions checking; Rapid updating of large analysis trees based on new intelligence data and assumptions. Over 1700 reasoning nodes 1. Analyst’s assumption challenged by Disciple -LTA 24

P 2: Multi-Agent and Multi-Domain Hypothesis Analysis Use a general problem solving paradigm which facilitates: ○ collaboration between users assisted by their agents; ○ solving problems requiring multi-domain expertise. P 1 S 1 P 11 S 11 … P 1 n S 1 n P 111 S 111 … P 11 m S 11 m P 11 S 11 P 11 m S 11 m Pa 11 m Sa 11 m … Pd 11 m Sd 11 m 25

Service-Oriented Disciple Systems Problem: Assess whether Al Qaeda has nuclear weapons. Perform analysis 1 Subproblem: Assess whether other countries within the global community Hypothesis believe that Al Qaeda has nuclear Analysis weapons. Subproblem: Assess whether Al Qaeda Broker makes credible claims to have nuclear weapons. 2 6 Disciple-LTA Client Upload reasonin g tree in Catalyst Check UDDI for registered competence 3 Ask broker for solutions Integrate Solutions 7 UDDI Receive solutions 5 Subproblem: Assess whether other countries within the global community believe that Al Qaeda has nuclear weapons. Solution: It is likely that other countries within the global community believe that Al Qaeda has nuclear weapons. 4 Solicit / receive solutions Disciple-LTA Clients Disciple LTA Servers Hypothesis Analysis Web Service Subproblem: Assess whether Al Qaeda makes credible claims to have nuclear weapons. Solution: It is almost certain that the Al Qaeda claims of having nuclear weapons

P 3: User Tutoring in Problem Solving Abstract reduction strategy Lesson on Evidence Automaticall y generated illustration of the abstract strategy 27

Lesson Fragment: Hypothesis support from a piece of evidence Abstract synthesi s strategy Automatica lly generated illustration of the abstract strategy 28

P 4: Mixed-Initiative Integrated Reasoning Creative solution Context for creative solution Modeling Problem Solving Generated example Modeling of Reasoning, Learning, and Problem Solving Mixed Initiative Refined Reasoning rule Expert example Learning Rule-based guidance 29

P 5: Integrated Teaching and Learning Analogy and Hint Guided Explanation Analogy-based Generalization Example of problem reduction step Incomplete explanation analogy NLP Knowledge Base 30

Reasoning Rules Learned from Analyst’s Solution 2. Disciple 1. The analyst extends the analysis logic Learned Rule learns reasoning rules Learned Rule To assess whethere are states that may be willing to sell nuclear weapons to an actor, one has to consider each nuclear state and assess whether that state may be willing to sell nuclear weapons to that actor. 31

P 6: Multistrategy Learning Knowledge Base Learning by Analogy and Experimentation IF we have to solve <Problem> Main PVS Condition Except-When PVS Condition Failure explanation Example of problem reductions generated by the agent Incorrect example Correct example Learning from Explanations Learning from Examples Except-When PVS Condition THEN solve <Subproblem 1> … <Subproblem m> 32

Rules Refined based on Analyst’s Critique 2. The analyst critiques the reasoning 1. Disciple applies the learned rule 3. Disciple-LTA refines the rule with an exceptwhen condition Refined Rule This is wrong! France will not sell nuclear weapons to Al Qaeda because it perceive s it as an enemy. To assess whethere are states that may be willing to sell nuclear weapons to an actor, one has to consider each nuclear state and assess whether that state may be willing to sell nuclear weapons to that actor, except for the case in which the nuclear state is an enemy of that actor. 33

Rules Refined based on Analyst’s Critique 2. The analyst critiques the reasoning 1. Disciple applies the refined rule 3. Disciple refines the rule with a new except-when condition Refined Rule This is wrong! Russia will not sell nuclear weapons to Al Qaeda because it opposes the proliferati on of To assess whethere are states that may be willing to sell nuclear weapons to an actor, one has to consider each nuclear state and assess whether that state may be willing to sell nuclear weapons to that actor, except for the case in which the nuclear state is an enemy of that actor and also except for the case when the nuclear state opposes the proliferation of nuclear weapons. 34

P 7: Use of Partially Learned Knowledge Use confidence level-based reasoning methods that allow efficient use of partially learned rules for modeling expert’s reasoning, learning and problem solving. PVS Condition Except-When PVS Condition Rule’s conclusion is (most likely) incorrect Rule’s conclusion is not plausible Rule’s conclusion is (most likely) incorrect Rule’s conclusion is plausible Rule’s conclusion is (most likely) correct 35

P 8: Learning within Evolving Representation Space Continuous adaptation of the previously learned rules to the evolution of the ontology. direct test ev IF <Problem> PVS Condition Except-When PVS Condition THEN <Subproblem 1> … <Subproblem m> 36

P 9: Architecture for Generality-Power Tradeoff Structure the architecture of the agent into two parts: o a reusable domain-independent learning agent shell; o domain specific modules. Disciple Agent Disciple Learning Agent Shell Graphical User Interface Knowledge Repository Learner Problem Solver Customized User Interface Customized Problem Solver Knowledge Base Manager Knowledge Base Domain Independent Modules Domain Dependent Modules 37

P 10: KB Structuring for Knowledge Reuse Structure the knowledge base into two parts: o its more general and reusable components; o its more specific components. Knowledge Base Disciple: Knowledge Base Structuring n n The object ontology which may be reused from existing knowledge repositories; The problem solving rules which are learned from the subject matter expert. Object Ontology Rules 38

Overview Research Vision: From PC to LA Disciple Approach to Agents Development Disciple Principles for Learning Assistants Applications of Disciple Cognitive Assistants Final Remarks and Discussion 39

Sample Applications of the Disciple Agents Intelligence Analysis Emergency Response Planning Course of Action Critiquing Workaround Reasoning Ph. D Advisor Selection Higher Order Thinking Skills in History and Statistics Regulatory KBs for Financial Services Industry Breast Cancer Treatment Innovations 40

Center of Gravity (COG) of a Force The center of gravity of an entity (state, alliance, coalition, or group) is the foundation of capability, the hub of all power and movement, upon which everything depends, the point against which all the energies should be directed. Carl Von Clausewitz, On War, 1832. The center of gravity of an entity is its primary source of moral or physical strength, power or resistance. Joe Strange, Centers of Gravity & Critical Vulnerabilities, 1996. If a combatant eliminates or influences the enemy’s strategic center of gravity, then the enemy will lose control of its power and resources and will eventually fall to defeat. If the combatant fails to adequately protect his own strategic center of gravity, he invites disaster. Giles and Galvin, USAWC 1996. 41

The CG-CC-CR-CV Task Reduction Analysis Model 1) Analyzing the strategic COG of a force is reduced to the tasks of analyzing the COG candidates corresponding to its main elements of power (government, people, economy, military, etc. ). 2) Analyzing a COG candidate is reduced to the tasks of analyzing its Critical Capabilities that may make it a COG. 3) Analyzing a Critical Capability is reduced to analyzing its Critical Requirements (i. e. the essential conditions, resources and means needed by the critical capability to be fully operative). 4) Analyzing a Critical Requirement is reduced to determining whether it has any Critical Vulnerability (i. e. Analyze the strategic COG candidates for US 1943 T 11 S 11 Analysis of President Roosevelt, military of US, industrial … T S capacity, 1 n 1 n … S 1 T 111 S 111 … T 11 m S 11 m Ta 11 m Sa 11 m … Td 11 m Sd 11 m Joe Strange, Centers of Gravity & Critical Vulnerabilities, 1996 (first printing), 2002 (third printing). 42

Disciple-COG A learning and decision-making assistant for center of gravity analysis based on the CG-CC-CR-CV model, used at the Army War College and the Air War College 43

Disciple-COG at Army War College and Air War College Center of Gravity Analysis Course Disciple-COG was taught based on the expertise of the course’s instructor, who used the CG-CC-CR-CV model. Disciple helps military personnel perform center of gravity analyses of scenarios of interest. Teaching Problem Disciple Learning Agent KB solving Sample Evaluations by Officers (Spring 2007) Tecuci G. , Boicu M. , Comello J, Agent-Assisted Center of Gravity Analysis, GMU 44

Disciple-VPT: Collaborating Virtual Experts for Emergency Response Planning Emergency situation Tanker truck leaking redfuming nitric acid near a student residential area of George Mason University. Virtual experts needed • emergency management; • police operations; • fire department operations; • hazardous materials handling; • health and emergency medical services; • sheltering, public works VE Library Disciple-VE Disciple-VE Disciple-VE User VE Team VE Assistant Disciple-VE Disciple-VE 45

Disciple-COA: Course of Action Critiquing Identifies strengths and weaknesses in a course of action, based on the principles of war and the tenets of Army operations. 46

DARPA HPKB Program Evaluation To what extent does COA 411 conform to the Principle of Surprise? Sample Critique: There is a strength with respect to surprise in COA 411 because the enemy is unlikely to be prepared for the heavy concentration of combat power applied by BLUE-TASKFORCE 1 as MAIN-EFFORT 1 in action PENETRATE 1. In this action, MAIN-EFFORT 1 is applying a force ratio of 10. 6 which is more than double the recommended force ratio 3. 0. Applying this much combat power for this action is likely to surprise the 100 -5 enemy Reference: FM pg 2 -5, KFand 118. 1, is KF indicative theis achieved proper 118. 2, KF 118. 3 of - Surprise by striking/engagingofthethe enemyprinciple in a time, place application of or manner for which he is unprepared. The surprise. " enemy can be surprised by the tempo of the operation, the size of the force, the direction or location of the main effort, and timing. Factors contributing to surprise include speed, effective intelligence, deception, application of unexpected combat power, operations security, and variations in tactics 100% • High knowledge acquisition rate. • Better performance and coverage than the other evaluated systems. • Better performance than the evaluating experts (many unanticipated solutions). 47

Disciple-WA Workaround Agent Estimates the best plan of working around damage to a transportation infrastructure, such as a damaged bridge or road. Disciple-WA showed that a knowledge engineer can use Disciple to rapidly build and update a knowledge base capturing knowledge from military engineering manuals and a set of sample solutions provided by a subject matter expert. Evolution of KB coverage and performance from the pre-repair phase to the post-repair phase. Development of Disciple’s KB during evaluation. 72% increase of KB size in 17 days • High knowledge acquisition rate; • High problem solving performance (including unanticipated solutions). • Demonstrated at EFX by Alphatech. 48

Overview Research Vision: From PC to LA Disciple Approach to Agents Development Disciple Principles for Learning Assistants Applications of Disciple Cognitive Assistants Final Remarks and Discussion 49

Research Vision for the Disciple Learning Assistants Personal Computers Mainframe Computers 50

Questions 51

Acknowledgements The research performed in the Learning Agents Center was sponsored by several United States government agencies, including the Air Force Research Laboratory, the Air Force Office of Scientific Research, the Defense Advanced Research Projects Agency, the National Science Foundation, and the Army War College. 52

PVS Condition Except-When PVS Condition Rule with Plaus Version Space Con Upper Lower 53

54

Problem Mixed-Initiative Problem Solving Reasoning Tree Ontology + Rules Extend Reasoning Tree Accept Reasoning Steps Reject Reasoning Steps Explain Examples Learned Rules Explain Examples Refined Rules Refined Ontology Explain Examples Rule Refinement Rule Learning 55