Invisible Interfaces Ken Fishkin Anuj Gujar Beverly Harrison

Invisible Interfaces Ken Fishkin, Anuj Gujar, Beverly Harrison, Roy Want Ken Fishkin, Nov. 2000 1

Trend z. Exponential growth in computer speed; Moore’s Law, but z. Limited growth of bandwidth and naturalness of our physical interaction with the computer Ken Fishkin, Nov. 2000 2

Emerging UI Paradigm AR, VR Phicons Interaction Bandwidth “Invisible Interfaces” GUI command line switches & knobs 50’s 60’s 70’s 80’s Time Ken Fishkin, Nov. 2000 90’s 98 3

Towards “Invisible Interfaces” Goal: to seamlessly blend the affordances and strengths of physically manipulatable objects with virtual environments, devices and artifacts Invisible Interfaces Physical Ken Fishkin, Nov. 2000 Virtual 4

Our Invisible Interface Efforts z. Manipulative UI y. PDAs ypressure sensors, tilt sensors z. Proximal UI ymobile computing y. RF sensors ywireless networking Ken Fishkin, Nov. 2000 5

Manipulative UI Approach interaction through transparently mimicking familiar physical manipulations + via augmenting handheld devices invisible interfaces Ken Fishkin, Nov. 2000 6

Video. . . z 3 tasks ynavigation through an ordered list ynavigation within a document yannotation optimization (handedness detection) Ken Fishkin, Nov. 2000 7

Key Design Considerations z. Match properties ye. g. , book - unit pages, sequential ordering preserved but thickness or extent is lost z. Match manipulations ye. g. , book - flicking corners consistent but moving by “chunks” is not z. Match feedback yvisual, auditory, kinesthetic (tactile) Ken Fishkin, Nov. 2000 8

Navigation - Rolodex Physical Rolodex manipulation Virtual Rolodex manipulation Ken Fishkin, Nov. 2000 9

Navigation - Rolodex Behind the scenes. . . Custom circuitry on back Side view - Palm Pilot Pressure sensors Ken Fishkin, Nov. 2000 10

Navigation Within a Book Physical page turning Page turning via strokes Ken Fishkin, Nov. 2000 11

Navigation Within a Book Moving by “chunks” relative to beginning/ending of book Moving relative to beginning/ending of document Ken Fishkin, Nov. 2000 12

Navigation Within a Book Behind the scenes. . . degree of pressure Ken Fishkin, Nov. 2000 spatial location 13

Next Iteration Ken Fishkin, Nov. 2000 14

Annotation Optimization Physical annotation Virtual annotation (handedness) Ken Fishkin, Nov. 2000 15

Annotation Optimization Behind the scenes. . . Pressure pad Ken Fishkin, Nov. 2000 16

Annotation Optimization The result. . . Ken Fishkin, Nov. 2000 17

Lessons Learned z No prior model that physical interaction has an effect (unexpected, learning needed) z Fidelty of matching is crucial (“interface fusion”), based on real world expectations z Inadvertent action is problematic z Careful kinesthetics design needed (range of motion, precision, JNDs) Ken Fishkin, Nov. 2000 18

Lessons Learned z User comments: “intuitive”, “cool”, “pretty obvious in terms of what’s going on” z Users explored the range and space of manipulations (as in novel GUIs) z Passive interactions perceived as “magical” z Enhanced interaction experience “beyond the laptop” Ken Fishkin, Nov. 2000 19

Related Work z Physical input devices: “phicons” [Ishii et al], “bricks” [Fitzmaurice] z Unique input devices: “doll’s head” [Hinkley et al] z Augmented devices [Rekimoto] [Small & Ishii] z Ubicomp, PARC tabs [Weiser & Want PARC] Ken Fishkin, Nov. 2000 20