Tactile Graphics with a Voice 2015 Richard Ladner

Tactile Graphics with a Voice 2015 Richard Ladner University of Washington 1

Text Book Translation Problem images math text 2

Tactile Perception Somatosensory system 3

Tactile Perception Limitations • Resolution of human fingertip: 25 dpi • Tactual field of perception is no bigger than the size of the fingertips of two hands • Color information is replaced by texture information • Visual bandwidth is 1, 000 bits per second, tactile is 100 bits per second 4

Resolution Study. 35 mm 1 mm 2 mm • On average, blind participants could sense 1 mm separation but no smaller. • On average, sighted participant could sense 1. 5 mm separation but no smaller. 5

Blind Spatial Perception • Tactile perception can lead to a “spatial” model in the brain.

Tactile Graphic 7

Tactile Graphics Kit 8

Tactile Graphics Printing • Embossing Printer • Thermoform • Swell Paper 9

Tiger Embosser • 20 dpi (raised dots per inch) • 7 height levels (only 3 or 4 are distinguishable) • Prints Braille text and graphics • Prints dot patterns for texture • Invented by a blind physicist, John Gardner 10

Refreshable Tactile Display 11

Graphics 12

Graphic Translation location file text extract preprocess original scanned image clean image <Location. Information> <Num. Labels>16</Num. Labels> <Resolution>100. 000000</Resolut ion> <Scale. X>1. 923077</Scale. X> <Scale. Y>1. 953125</Scale. Y> <sd> <x 1>121</x 1> <y 1>45</y 1> <x 2>140</x 2> <y 2>69</y 2> <Alignment>0</Alignment> <Angle>3. 141593</Angle> </Label> pure graphic text image 13

Graphic Translation location file pure graphic text image <Location. Information> <Num. Labels>16</Num. Labels> <Resolution>100. 000000</Resolution> <Scale. X>1. 923077</Scale. X> <Scale. Y>1. 953125</Scale. Y> <Label> <x 1>121</x 1> <y 1>45</y 1> <x 2>140</x 2> <y 2>69</y 2> <Alignment>0</Alignment> <Angle>3. 141593</Angle> </Label> text y (0, 20) x=15 15 10 5 O x 5 10 15 20 20 x+y=20 (15, 0) (15, 5) Braille y (#0, #20) x. k#15 #10 #5 O x #5 #10 #15 #20 x+y. k#20 (#15, #0) (#15, #5) 14

15

Available Books • Computer Architecture: A Quantitative Approach, 3 rd Edition 25 minutes per figure (230 figures) • Advanced Mathematical Concepts, Precalculus with Applications 6. 3 minutes per figure (1, 080 figures) • An Intoduction to Modern Astrophysics 10. 2 minutes per figure (467 figures) • Discrete Mathematical Structures 8. 8 minutes per figure (598 figures) • Introduction to the Theory of Computation, 2 nd Edition 13. 3 minutes per figure (180 figures) 16

TGA Workflow • Advantages – Much faster production – Batch processing instead of one figure at a time – Much tedious work is avoided • Disadvantages – May be of lower quality than custom translation – A lot of technology needs to be mastered 17

One-offs vs. Mass Production 1916 Woods Dual Power 1906 Reo Model T 18

Audio-Tactile Graphics Talking Touch Tablet Digital Pen audio-tactile graphic 19

Tactile Graphics with a Voice: using QR codes to access text in tactile graphics • C. M. Baker, L. R. Milne, J. Scofield, C. L. Bennett, RE Ladner. Proceedings of the 16 th international ACM SIGACCESS conference on Computers. ASSETS 2014. 20

Tactile Graphics with a Voice 21

System Use 22

Modes p o T Silent Verbal Finger Pointing 23

Study • • • 10 blind users 4/10 non-Braille 4 tasks 3 modes 6 sessions 24

Tasks Line Triangle Parabola Bar Chart 25

Time F 2, 640=0. 619, p=. 5391 F 5, 640=2. 268, p<. 05 26

Time outs 27

Accuracy Silent Verbal Finger Pointing Line 97% 93% Parabola 93% 95% Triangle 88% 90% Bar Chart 73% 70% 79% All Tasks 88% 89% 28

Preference • 4 preferred silent – To be honest, I use screen readers every day and I am so sick of electronic noise. • 4 preferred finger pointing – I like the concept of the finger pointing. I feel more confident that since it looks for a finger it’s getting the right QR code if you have multiple on the same page. • 2 preferred verbal – The other thing I like about Verbal mode is that every time I hear a zero I think so I need to move it a little bit 29

Braille comparison In a study of 82 images from a pre-calculus textbook, on average Braille took up more space – Braille was larger in 326 out of 626 labels • Braille labels: 327 sq. mm • QR codes: 225 sq. mm 30

Braille - Time 31

Braille - Accuracy 32

Questions 33