Dialogue in Intelligent Tutoring Systems Dialogs on Dialogs

Dialogue in Intelligent Tutoring Systems Dialogs on Dialogs Reading Group CMU, November 2002

ITS? • Goal: help a human learn to perform a task • Task Model: models how an expert would perform the goal task • Student Model: models how the student currently performs the task + prior knowledge on how students usual perform • Teacher Model: usually none…

Auto. Tutor (U. of Memphis) • Textual conversation with an animated agent tutor • Originally for Computer Literacy, also for Newtonian Physics and Research Methods • Goal: get (long) answers to general, concrete questions and elicit/correct student knowledge e. g. : Suppose a runner is running in a straight line at constant speed, and the runner throws a pumpkin straight up. Where will the pumpkin land? Explain.

Autotutor: Dialogue • Selects dialogue moves from: – – Positive/negative feedback Prompts Hints … • Students can ask Wh- and Yes/No-questions • Based on a “Dialogue Advancer Network”: FSM that selects the next move according to student’s last utterance • Latent Semantic Analysis to match student answers with expectations

Autotutor: Comments • “super form filling” where the system knows the value of the slots beforehand. • Mostly system-initiative, no memory (if the student asks a question, the system forgets what it is doing) • Global strategy fixed (by system architecture)

ATLAS/ANDES (Pitt) • ANDES: ITS for physics, no natural language • ATLAS: “add-on” to ANDES, provides Knowledge Construction Dialogues for hints (main task/evaluation is left to ANDES) • KCD: recursive FSM – Reactive planner to pick next KCD – Can insert subdialogues (clarification, rectification…) and go back to original topic

ATLAS/ANDES

BEETLE (U. of Edinburgh) • Fully plan-based tutorial dialogue: – Top tier: global strategy/repair when failure – Middle tier: handles specific tasks according to the situation – Bottom tier: performs primitive dialogue actions • Teaches basic electricity and electronics • Very dialogue-oriented but not completely implemented yet

PACO: Pedagogical Agent for Collagen (USC, Mitsubishi, MITRE) • Simulation-based training • Domain-independent: adapts to any simulator (e. g. Gas Turbine Engine) • Collaborative Discourse Theory-based: – Rules describe interactions between three agents: student, tutor, simulator – Discourse acts: both utterances and domain actions

Stanford’s CSLI System • Also simulation-based (Shipboard Damage Control) • Complex dialogue management: – tree based activity model (similar to CMU Communicator) built dynamically • Separation between dialogue management and tutoring strategy: – Tutoring Module constructs the activity tree using recipes while the Dialogue Manager uses the tree to conduct the dialogue

Simulation-based Systems Comments • Combine advanced dialogue architectures with ITS • Mixed-initiative dialogue management • Really multimodal (click-based simulator, speech-based tutorial dialogue)

CALL System (U. Le Mans) • Actional approach to Language Learning: – User must perform a task (cooking) • Communicative approach: – Interaction with a partner agent • Tutor agent to give instructions/help • Specificity of CALL: language is the domain taught, not only a means of teaching

CALL System • Based on a theory of Human Computer Dialogue • Partner agent: usual dialogue system • Tutor agent: must monitor language and dialogue issues • Evaluation in terms of efficacy/efficiency: similar to evaluation of dialogue systems (number of turns taken) but for the human!

What can dialogue bring to ITS? • Human tutor-like instruction: – Qualitative, natural (cf science) – Helps the student construct knowledge instead of just “telling” him/her deeper understanding (? ) • Realistic simulation of certain tasks when teaching communicative skills

What characterizes tutorial dialogue? • Tutor has expectations about student’s utterances • Student must be able to: – Exhibit knowledge – Ask questions (although this does not happen so often…) • Open-ended: no specific goal (except teaching) • ?

Are standard dialogue systems suitable for tutorial dialogues? • Possibilities of state-of-the-art dialogue systems underexploited? – Strategies for repair/elicitation – Confidence measures (integrated with student model? ) – Real mixed-initiative: its meaning in the context of tutoring

What can dialogue research learn from ITS? • Task modeling • Student/User modeling • Multi-level planning (long-term strategies, mid-term tactics, short-term actions) more necessary in ITS than anywhere else (pedagogical goals)

Other issues… • General lack of cooperation between language technology and ITS researchers? – Natural Language Understanding – Dialogues • Both dialogue systems and ITS require heavy human work to create: – how can we derive a task model automatically? • Anything else?

Any other comment…