AMBILEARN Ambient Intelligent Multimodal Learning Environment for Children

AMBILEARN Ambient Intelligent Multimodal Learning Environment for Children 100 day review December 2008 1 Jennifer Hyndman Supervisors: Dr. Tom Lunney, Prof. Paul Mc Kevitt School of Computing and Intelligent systems Faculty of Computing and Engineering University of Ulster, Magee.

OVERVIEW Research Objectives Multimodal Interfaces HCI ~ Child Computer Interaction Games in Education Project Proposal Software Analysis & Prospective Tools Relation to other work Project Plan Conclusion 2

RESEARCH OBJECTIVES Design and Develop Ambi. Learn Means to merge multiple modalities Mobile Devices e. g. Smart Phones (Nokia) /Personal Digital Assistant (PDA) Interface Design /Usability Ambi. Learn Educational game e. g. Treasure hunt / puzzels Demonstrate and Test Ambi. Learn 3

MULTIMODAL INTERFACES Natural communication methods Bolt’s Media Room (Bolt, 1980) Smart. Kom (Wahlster et al. 2001) Spoken dialogue systems Speech Embodied Agents Gandalf (Thórisson, 1997) Steve (Johnson et al. 1997) REA (Cassell et al. 1999) 4

HCI ~ CHILD COMPUTER INTERACTION Child centred design Dexterity/speech/interaction style/background/reading (Bruckman et al. 2007) GAMES IN EDUCATION Adaptive and appropriate feedback / cognitive strategies (Rosas et al. 2003) Augmented Reality Story Telling Collaboration / Educational Networking e. g. Web 2. 0 5

PROJECT PROPOSAL Speech Ambi. Learn PDA based Multimodal input Graphics 2 D Gesture Embodied Agent e. g. Speech / 2 D gesture (touch screen) Multimodal Output e. g. Graphics / Embodied Agent / speech Educational Game e. g. Treasure hunt / puzzels Designed for children Collaborative learning / problem solving 6

SOFTWARE ANALYSIS & PROSPECTIVE TOOLS Platforms Psyclone COLLAGEN Spoken dialogue systems CSLU Toolkit Microsoft Speech Embodied Agents X-Face Microsoft Agent Languages C++ Java Visual Basic Web Tops Glide Nivido eye. OS PDA e. g. Compaq i. PAQ 7

RELATION TO OTHER WORK System Year Device Input Modality Output modality Desktop PC PDA Speech Touch visual Speech Graphics Text Embodied Agent Support collaboration Educational Domain Gandalf 1997 Solar system Steve 1997 Virtual Procedural Training Kids. Room 1998 Imaginative Story Telling Storyteller 1999 Story Telling TICLE 2002 Tangram Puzzle Read-It 2004 Reading and Language NICE 2005 Life and Works of Hans Christian Anderson Virtuoso 2006 Art History Augmented Knights Castle 2007 History (Middle Ages) Ambi. Learn 2011 Treasure Hunt / Puzzles 8

RESEARCH PLAN Research Activities 2008 Oct-Dec Activities Submissions Deliverables 2009 Jan-Mar Apr-Jun 2010 Jul-Sep Oct-Dec Jan-Mar Apr-Jun 2011 Jul-Sep Oct-Dec Jan-Mar Apr-Jun Jul-Sep Perform literature review 100 Day Review and VIVA Confirmation report 2 nd Year Poster 3 rd Year Presentation ISEA 2009 conference AICS 2009 conference Develop Ambi. Learn Test and Evaluate Thesis write up Submit Thesis 9

CONCLUSION Advance knowledge Multimodal mobile games in education Multimodal interfaces on mobile devices Collaborative learning with mobile devices Ambi. Learn Ambient intelligent Multimodal Learning Environment for children Multimodal communication supporting collaborative learning using mobile devices 10

THANK YOU! QUESTIONS? 11