Sim Student Building a Cognitive Tutor by Teaching

Sim. Student: Building a Cognitive Tutor by Teaching a Simulated Student Noboru Matsuda Human-Computer Interaction Institute Carnegie Mellon University

CTAT: Cognitive Tutor Authoring Tools • Example-Tracing Tutor with zero programming – A cognitive model specific to a particular problem – Some generalization by modifying a behavior graph Sim. Student Project • Model-Tracing Tutor requires a cognitive model – Cognitive task analysis is challenging – Writing production rules is even more challenging – Performing the task is much easier… 2 PSLC Summer School 2009 : : Sim. Student Demo : : Noboru Matsuda (CMU)

Next Generation Authoring Build a tutor GUI Teaching a solution Sim. Student Project Sim. St. learning Production Rules Rule simplify-LHS: IF is-equation( Eq ), IFRule simplify-LHS: is-equation( Eq ), is-lhs( Eq, Lhs ), is-lhs( Eq, IF polynomial( is-equation( Lhs ), ), Eq ), polynomial( Lhs all-var-terms( is-lhs( Lhs Eq, )), Lhs ), all-var-terms( Lhs Then simplify( polynomial( Lhs, S-lhs Lhs ), ) ), Then simplify( ), ) enter( S-lhs all-var-terms( )Lhs, S-lhs. Lhs enter( S-lhs ) Then simplify( Lhs, S-lhs ), enter( S-lhs ) 3 PSLC Summer School 2009 : : Sim. Student Demo : : Noboru Matsuda (CMU)

Sim. Student • Machine learning agent – Learns problem-solving steps by … – Observes model solutions / solving problems, and … Sim. Student Project – Outputs a set of production rules • Fundamental technology – Programming by Demonstration – Inductive Logic Programming Lau, T. A. , & Weld, D. S. (1998). Programming by demonstration Blessing, S. B. (1997). A programming by demonstration authoring tool for model-tracing tutors 4 PSLC Summer School 2009 : : Sim. Student Demo : : Noboru Matsuda (CMU)

Authoring Strategies • Authoring by demonstration Sim. Student Project – Learning from worked-out examples – Demonstrate whole solutions – Learning by generalizing examples (when it can’t “selfexplain”) • Authoring by tutoring – Learning by doing (with tutor feedback) – Interactively tutor with immediate feedback and hint – Learning by generalizing hint with taking feedback into account 5 PSLC Summer School 2009 : : Sim. Student Demo : : Noboru Matsuda (CMU)

Demo! Sim. Student Project 6 PSLC Summer School 2009 : : Sim. Student Demo : : Noboru Matsuda (CMU)

Learning Production Rules in 3 parts: What-When-How If such and such constraints hold When Sim. Student Project among this and that GUI elements What Then do actions with the GUI elements How PSLC Summer School 2009 : : Sim. Student Demo : : Noboru Matsuda (CMU)

Production Rule in JESS GUI elements Sim. Student Project Constraints Actions PSLC Summer School 2009 : : Sim. Student Demo : : Noboru Matsuda (CMU)

Background Knowledge • Domain concepts to “explain” demonstrations – Operators – Feature predicates • External Jess function written in Java Sim. Student Project (defrule multi-lhs … ? var 22140 <- (column (cells ? ? ? var 22143 ? ? ? )) ? var 22143 <- (cell (value ? val 0&~nil)) => ) 9 (test (fraction-term ? val 0 )) (bind ? val 2 (denominator ? val 0)) (bind ? input (mul-term-by ? val 0 ? val 2)) … PSLC Summer School 2009 : : Sim. Student Demo : : Noboru Matsuda (CMU)

Example： Algebra domain • 16 Feature predicates & 28 operators Sim. Student Project 10 Feature Predicates for LHS conditions Operators for RHS actions Has. Coefficient Has. Const. Term Has. Var. Term Homogeneous Is. Fraction. Term Is. Constant Is. Denominator. Of Is. Numerator. Of Is. Polynomial Monomial Not. Null Var. Term Is. Skill. Add Is. Skill. Subtract Is. Skill. Divide Is. Skill. Multiply Add. Term. By Coefficient Copy. Term Denominator Div. Term. By Eval. Arithmetic First. Term First. Var. Term Get. Operand Inverse. Term Last. Const. Term Last. Var. Term Mul. Term. By Numerator Reverse. Sign Rip. Coefficient Skill. Add Skill. Clt Skill. Divide Skill. Multiply Skill. Rf Skill. Mt Skill. Subtract Var. Name PSLC Summer School 2009 : : Sim. Student Demo : : Noboru Matsuda (CMU)

Learning Results Sim. Student Project % Correct rule firings (10 test tasks) Authoring by tutoring Better than Authoring by demonstration # of training tasks 11 PSLC Summer School 2009 : : Sim. Student Demo : : Noboru Matsuda (CMU)

Authoring Time Authoring by Tutoring • Authoring by tutoring took 86 minutes • Authoring by demonstration took 238 minutes Sim. Student Project • A 2. 8 x speed-up! Authoring by Demonstration 12 PSLC Summer School 2009 : : Sim. Student Demo : : Noboru Matsuda (CMU)

Example: Stoichiometry Tutor Sim. Student Project 13 PSLC Summer School 2009 : : Sim. Student Demo : : Noboru Matsuda (CMU)

Learn more about Sim. Students • Project Web – www. cs. cmu. edu/~mazda/Sim. Student • Download & Tutorial Sim. Student Project – http: //ctat. pact. cs. cmu. edu (linked from project web) • Contact Noboru – mazda@cs. cmu. edu 14 PSLC Summer School 2009 : : Sim. Student Demo : : Noboru Matsuda (CMU)

Sim. Student Projects • Intelligent Authoring – Building a Cognitive Tutor as a CTAT Plug-in • Student Modeling and Simulation Sim. Student Project – Controlled educational studies – Error formation study – Prerequisite conceptual knowledge study • Teachable Peer Learner – Learning by teaching 15 PSLC Summer School 2009 : : Sim. Student Demo : : Noboru Matsuda (CMU)

Learning by Teaching Sim. Student Project PSLC Summer School 2009 : : Sim. Student Demo : : Noboru Matsuda (CMU)