Sphere Multitouch Interactions on a Spherical Display Hrvoje

Sphere Multi-touch Interactions on a Spherical Display Hrvoje Benko, Andrew D. Wilson, and Ravin Balakrishnan Microsoft Research – University of Toronto

UIST '08 2

Spherical Displays Not directly interactive ARC Science – Omni. Globe Global Imagination - Magic. Planet NOAA – Science on a Sphere Physical rotation only Pufferfish – Puffer. Sphere Kettner et al. ‘ 04 – Vi. Ball Companje et al. ‘ 07 – Globe 4 D UIST '08 3

Interaction Metaphor Shift UIST '08 4

Sphere Multi-touch sensitive � Built on Global Imagination’s Magic. Planet � Internal projection and vision-based sensing � GOAL: Explore interactive possibilities of spherical displays. UIST '08 5

Video: Sphere UIST '08 6

Sphere Video UIST '08 7

Sphere Hardware 24” 16” UIST '08 8

UIST '08 9

UIST '08 10

Reusing the Optical Path UIST '08 11

Video: Touch-Sensing UIST '08 12

Touch-Sensing Software Pipeline UIST '08 13

Sensing and Projection Distortions 3 D Vertex shader: � Pre-distorts objects into 2 D disk image � Objects appear undistorted on the spherical surface 2 D UIST '08 14

Video: Projection Distortions UIST '08 15

Projection Distortions UIST '08 16

Choosing a Coordinate System 2 D (Disk) Flat (Cylindrical) 3 D (Spherical) Hybrid (Tangential Plane) UIST '08 17

Application Prototypes � Photo & Video Browser � Omni-Directional Data Visualization �Globe �Omni-camera �Panoramic Walkthrough � Sphere Painting � Sphere Pong UIST '08 18

Design Space of Spherical Displays � Non-visible hemisphere � Visibility changes with position �“Pseudo-private” and “public” areas � No master user position / orientation � Smooth transitions in depth and orientations �Near – far �Horizontal – vertical � Borderless, but finite display � Few natural landmarks: poles UIST '08 19

Multi-Touch Interaction Techniques � Multi-touch basics �Drag, Rotation, Scale � Extend user’s reach �Flicking, Send-to-Dark-Side � Minimize orientation difficulties �Auto-rotation, Tether � Universal circular menu �Orb-like invocation gesture UIST '08 20

Dragging A 1 V 2 1 θ 2 Q Flat Surface Spherical Surface UIST '08 21

Send-to-Dark-Side � Open-palm gesture � Control the location where the object will appear UIST '08 22

Tether � Allow unconstrained manipulation � Preserve “natural” orientation of omnidirectional data upon release UIST '08 24

Universal Circular Menu � Orb-like Invocation �Easy to perform – Hard to do by accident UIST '08 25

Video: Sphere Interactions UIST '08 26

Sphere Multi-Touch Interactions UIST '08 27

Video: Contact Area Interactions UIST '08 28

Sphere Paint & Pong UIST '08 29

User Observations � 3 high-traffic demos � >1000 people � Feedback: � “Magical”, “Like a crystal ball” � “What is it good for? ” � “Can I zoom? ” � Interaction zones: � Top – shared � Middle – personal/working � Bottom – storage � Logs show no position preference UIST '08 30

Future Work Zooming � Sphere as a large controller � Sphere as a room projector (instant CAVE) � Handheld Sphere � UIST '08 31

UIST '08 32

Try our demo at the reception tonight! benko@microsoft. com http: //research. microsoft. com/~benko/projects/sphere UIST '08 33

Extra Slides UIST '08 34

Applications � Visualizing spherical data � Planets, stars � � Public information booths Remote omni-directional visualization � Sphere + Round. Table � 360 deg. video-conferencing � Robot operation � � Ambient Displays Large displayable input device � Trackball � Space Mouse / Joystick � Planetarium Control � Games UIST '08 35

Conclusions � The first multi-touch sensitive spherical display �Self-enclosed solution without occlusion and shadowing problems �A set of multi-touch interactions that facilitate data sharing around the display � Omni-directional room projection possibility �Instant CAVE UIST '08 36

Uses UIST '08 37

Come to our demo at the reception tonight! benko@microsoft. com http: //research. microsoft. com/~benko/projects/sphere UIST '08 38

benko@microsoft. com http: //research. microsoft. com/~benko M. C. Escher UIST '08 39

Putting it all together… UIST '08 40

Two Spheres (of Interest) UIST '08 41

Sphere Pong UIST '08 42

Sphere Painting UIST '08 43

Extending User’s Reach UIST '08 44

Auto-rotation & Menu UIST '08 45

Tether Behavior UIST '08 46