Programming by Voice with Scratch Teaching the Cat

Programming by Voice with Scratch: Teaching the Cat to Obey a Bird Ramaraju Rudraraju, Srinivasa Datla, Avishek Banerjee, Mandar Sudame Univ. of Alabama at Birmingham, Department of Computer and Information Sciences Jeff Gray University of Alabama, Department of Computer Science Gary Edwards United Cerebral Palsy of Greater Birmingham This research sponsored by a Google Research Award

Overview of Presentation Scratch Script Video Motivation Myna Features Architecture & Working of Myna Dynamic Grammars

Scratch Script Get your popcorn, its Movie Time!!

Motivation The WIMP Metaphor Icons Menu Window Pointing Device

Motivation The WIMP assumes dexterity The MYNA Solution for Scratch

Features Types of Navigation Drag and Drop Navigation Continuous Navigation using Transparent Frames

Features Macro Commands (An extension of Transparent Frames) Drop After Drop In Drop Before 7

Myna Architecture Component Mappings Cloud Garden Model Sprite State Storage Speech Recognizer Component Hierarchy MYNA View Grammar Controller Command Executor Java Robot Scratch UI Transparent Frames

Under the Hood of Myna Move Right Controller Grammar Speech Recognizer Model Command Executor Mappings Java Robot Component Hierarchy

Dynamic Grammar Resume Pause File Navigation Grammar Default Grammar Speech Pause Grammar File Grammar Control Grammar Looks Grammar

Looking Ahead Limitations Scratch needs to be in maximized mode Not compatible with different screen resolutions Myna is not scroll-enabled Large vocabulary and still growing! Future Work Integration with UCP - Deep human subjects testing with United Cerebral Palsy of Greater Birmingham Portability on other platforms Configurable screen resolutions

Summary Myna is our preliminary attempt towards voice-driven programming applied to Initial Learning Environments (ILEs) to overcome the physical challenges of using the traditional WIMP metaphor Our long-term goal is to assist children with disabilities by giving them a learning tool that enables them to encounter the joys and excitement of programming, while also improving their logic skills

Thank you Additional questions? teammyna@googlegroups. com Questions and comments? https: //sites. google. com/site/projectmyna 2010/