The Tactical Language Training Project W Lewis Johnson

The Tactical Language Training Project W. Lewis Johnson, Sunhee Choi, Stacy Marsella, Nicolaus Mote, Shrikanth Narayanan, Hannes Vilhjálmsson, Shumin Wu University of Southern California Micro Analysis & Design US Military Academy UCLA CRESST Center for Advanced Research in Technology for Education, USC / ISI

Major Research Themes n Study role of simulation games and intelligent tutoring in promoting communicative skill ä Simulation motivates, builds fluency · Intelligent scaffolding built into simulation ä n n n Tutoring provides form feedback Tailor speech recognition to learner language Emulate free dialog within narrative structure Manage learner motivation ä ä Strong task orientation Promote early communicative success Employ motivational tutorial tactics Adjust level of challenge to maximize engagement Center for Advanced Research in Technology for Education, USC / ISI

Scenario n n Your special ops unit is deployed to a town to assist with postwar recovery The objectives: ä ä ä n Establish rapport with locals, describe mission Ask for directions to local leader Meet leader and discuss recovery needs Initiate mission (organize reconstruction) Overcome barriers to mission (factions, sabotage, etc. ) Impact of communication failure: ä ä Failure of mission Hostile encounters with locals Center for Advanced Research in Technology for Education, USC / ISI

Demo Center for Advanced Research in Technology for Education, USC / ISI

Overall Architecture Center for Advanced Research in Technology for Education, USC / ISI

Interaction with Mission Skill Builder Center for Advanced Research in Technology for Education, USC / ISI

Authoring Issues n Story-oriented authoring poses particular challenges: ä Authoring is a multidisciplinary team activity · A mixture of top-down and bottom-up design Curriculum content is introduced incrementally and opportunistically ä Needed: ä · Tools to facilitate incremental construction of materials · Tools to track curriculum coverage · Tools to check for validity and completeness Scene script - HTML Center for Advanced Research in Technology for Education, USC / ISI Scene script - XML

Speech Processing n n Dual goals of robust recognition and spoken language skill evaluation Recognition subsystem goals: ä ä n Robustness to non native speech Lightweight operation Pronunciation evaluation system goals: ä ä Multilevel evaluation ranging from utterance level prosody to segmental cues Feedback requirements differ for learner and game AI Center for Advanced Research in Technology for Education, USC / ISI

Speech Recognition- System Design n HTK based system ä n Recognition networks loaded dynamically ä n Networks optimized for different stages in schoolhouse and mission training Mission training mode ä n Bootstrapped from modern standard Arabic and adapted to Levantine speech and lexicon Large vocabulary (LVCSR) like operation Schoolhouse mode for skill acquisition ä Limited vocabulary with pronunciation variants and hypothesis rejection Center for Advanced Research in Technology for Education, USC / ISI

Natural Language Processing n Functions of NLP: Validate authored learning materials ä Link dialog examples to authored explanations, translations ä Predict & recognize common learner errors ä Center for Advanced Research in Technology for Education, USC / ISI

Managing Pedagogical Drama Want an engaging pedagogical drama n n n Unfold based on trainee's language & cultural skills Give trainee a sense of control and responsibility Provides pedagogically appropriate consequences Respond robustly to learner communicative actions Avoid computationally expensive approaches Center for Advanced Research in Technology for Education, USC / ISI

MPE Architecture Center for Advanced Research in Technology for Education, USC / ISI

Technical Details Director uses POMDPs to model trainee and characters n ä Student model informs trainee capabilities POMDPs used to predict possible story paths Fit parameters of character POMDPs to increase likelihood of desirable story path n n ä Example: Adjust importance of being hospitable Fitting sensitive to cultural & situational factors n ä Example: hospitality for strangers, support for US involvement important goals to adjust Center for Advanced Research in Technology for Education, USC / ISI

Evaluation Sessions, May 2004 n n 7 learners 6 some prior knowledge of TLTS All had prior L 2 language experience All experienced gamers n n n Practice w/ SB ~ 1 hour MPE session Post session interview Center for Advanced Research in Technology for Education, USC / ISI

General comments n n TLTS viewed as potentially much better than class instruction learners readily identify best features (one-on-one, ME), can anticipate benefits n n no major interface or navigation concerns many problems identified are already being solved Center for Advanced Research in Technology for Education, USC / ISI ä ä MSB · speech recognition · learner feedback MPE · avoidance

Skill Builder Conclusions n Pronunciation accuracy threshold too high for beginners ä Since has been lowered n Tutor ä Feedback selection algorithm adjusted n Lack ä feedback sometimes repetitive of transfer from MSB to MPE New exercises added to MSB to emulate MPE dialog Center for Advanced Research in Technology for Education, USC / ISI

Practice Environment Conclusions n Learners ä reluctant to enter MPE Learners now explicitly encouraged at multiple points n Learners not sure what to do Provide detailed mission briefings ä Aide helps provide guidance ä Center for Advanced Research in Technology for Education, USC / ISI

Design for Next Evaluation n ~25 subjects, non-Arabic speakers n Four experimental conditions: Complete system ä MPE only ä MSB without pronunciation feedback ä Center for Advanced Research in Technology for Education, USC / ISI

Backup Slides Center for Advanced Research in Technology for Education, USC / ISI

Role of Gesture and Culture n Animate gestures in virtual locals: To promote learner understanding ä To expose learner to unfamiliar gestures ä n Allow ä learner to select gestures: To learn appropriate gestures for different social situations · E. g. , to build rapport, manage crowds n Culture training involves: Training in face threats, face threat mitigation strategies ä Expectations (etiquette) in different social situations ä Center for Advanced Research in Technology for Education, USC / ISI