Making a Computer Speak Algebra However You Want

Making a Computer Speak Algebra However You Want CEC 2013 Convention & Expo SAN ANTONIO, TEXAS April 5, 2013 1

Presented by Susan A. Osterhaus Texas School for the Blind and Visually Impaired Outreach Programs 1100 West 45 th Street Austin, TX 78756 susanosterhaus@tsbvi. edu www. tsbvi. edu/math/ 2

Presentation Overview �Project Motivation, Goals, and Background �Development to date: discussion and demonstration �Research Studies �Next Steps 3

Project Motivation �Text-to-speech (TTS) vs. Math-to-speech �Limited choice of syntax/semantics �Limited screen-reader support �Limited or no within-expression navigation �Difficult to author �Works in limited environments �ETS Experience 4

Project Goals �High school algebra �Classroom-like synthesized speech “style”: Clear. Speak �Speech is flexible so computer will speak the way the teacher wants �Can be used in any math document – including both instruction and assessment �Interactive navigation �Author in Word with easy playback 5

Not Part of Project �Braille �Support for other formats �Authoring by people with visual impairment �Solving math problems �Tutoring/Curriculum �Integration with standardized testing 6

What is Math. ML? �Mathematical Markup Language (Math. ML) is an application of XML for describing mathematical notations and capturing both its structure and content. It aims at integrating mathematical formulae into World Wide Web pages and other documents. It is a recommendation of the W 3 C math working group. From Wikipedia, the free encyclopedia �Math. ML = Math Accessibility 7

Behind the Scene: Math. ML (What is Math. ML? ) <math> <mroot> <mi>x</mi> <mn>3</mn> </mroot> </math> 8

Project Software for Math �Math. ML Tools: �Authoring: Math. Type+Word �Audio rendering: Math. Player �Screen reader �Window-Eyes (Word & IE) 9

Many ways to Speak Math �How do you say… 10

Many ways to Speak Math �How do you say… 11

Many ways to Speak Math �How do you say… 12

Many ways to Speak Math �How do you say… 13

Many ways to Speak Math �How do you say… 14

Many ways to Speak Math �How do you say… 15

How Math Speech Works �Rules �Preferences �Exact Speech 16

Clear. Speak �Many ways to speak math �Clear. Speak design philosophy �Other design philosophies �Math. Speak �Simple. Speech 17

Demonstration �Authoring �Using Math. Type in Word to enter math and use preferences �Entering exact speech in Word �Playback �Word with Window-Eyes 18

Project Goals Recap �Clear. Speak �Author in Word �Allow setting preference and exact speech �Playback in Word and IE �Interactive navigation 19

Creating Expressions and Questions for Studies �Level �Degree of complexity �Relevance of speech variations �Provide useful data �Test comprehension, not computation �Guide decisions about speech rules, preferences 18

Feedback Study #1 �Compared Clear. Speak, Math. Speak, Simple. Speech �Fractions, exponents, parentheses �Focus: was expression understood? �Research/Statistical consultation � 16 HS students: blind or low vision 21

Results Summary �Clear. Speak more understandable than Simple Speech or Math. Speak �Students overall preferred Clear. Speak �Students got more answers correct with Clear. Speak than with Simple Speech or Math. Speak 22

One Expression’s Results P+ Very Somewhat Very Familiar Unfamiliar Simple Speech 3 0 8 1 7 Math. Speak 3 1 5 4 6 Clear. Speak* 6 6 8 1 0 23

Familiarity and Understanding Very familiar Somewhat familiar Not very familiar Very unfamiliar Clear. Speak Simple. Speak Math. Speak 48% 24% 50% 31% 2% 9% 23% 0% 16% 22% Very easy to understand Somewhat hard to understand Very hard to understand 52% 33% 13% 3% 17% 37% 24% 22% 16% 38% 23% 22% Very sure understood Somewhat sure understood Not sure understood Definitely did not understand 71% 24% 5% 0% 42% 37% 7% 14% 39% 33% 11% 17% 24

Feedback Study #2 �Can Prosody help understanding? �Pauses �Pitch �Rate Changes �Volume Changes �Rejected pitch and volume changes �non-speech sounds not an option �Focus on pauses, rate-change vs. start/end 25

Feedback #2 Example vs. �With Pauses / rate change �Expression 1 �Expression 2 �With “end root” �Expression 1 �Expression 2 26

Feedback #2 Another Example Nested Parentheses: �Uniform Pauses �Non-uniform pauses and “First Paren” 27

Navigation �Simple char by char mode for beginners �Powerful navigation features for experts �Read �Describe �Multiple ways of moving �Tree �Character, Placemarker, Semantic �After tree move: read or describe � 10 Placemarkers, 2 Cursors, Where am I? 28

Participation Opportunities �Blind or Visually Impaired Students �Teachers 29

Contact Information �Lois Frankel: lfrankel@ets. org �Neil Soiffer: neils@dessci. com �Beth Brownstein: bbrownstein@ets. org �Susan Osterhaus: susanosterhaus@tsbvi. edu �Math. Player: http: //www. dessci. com/en/products/mathplayer/ The research reported here was supported by the Institute of Education Sciences, U. S. Department of Education, through Grant R 324 A 110355 to the Educational Testing Service. The opinions expressed are those of the authors and do not represent views of the Institute or the U. S. Department of Education. 30

Thank you for attending this session. 31