HumanCentered Computing Telepresence AR tangible interfaces John Canny

Human-Centered Computing Telepresence, AR, tangible interfaces John Canny UC Berkeley

Telepresence, AR, tangible interfaces John Canny • PRo. Ps (Personal Roving Presences) and gesturing avatars. • UPM - a universal planar manipulator. • Bearable computers - turning laptops into augmented reality systems and wearable computers. HCC overview

PRo. Ps: Personal Roving Presences Space Browser + Eric Paulos Surface cruiser HCC overview

PRo. Ps: Toward being there • A PRo. P is a Personal Roving Presence. • PRo. Ps seek to bridge the gap between telepresence and • being there. There are two essential aspects of remote presence that only telerobotics can provide: u u Non-verbal communication, i. e. gaze, proxemics, gesture etc. which are highly spatialized. A gamut of human behaviors, such as chance encounters, informal meetings, showing or being shown, browsing, exploring etc, which are affordances of a physical presence. HCC overview

Terrestrial PRo. Ps: Carts • Carts are a good match to human capabilities. • Our strategy was to list important physical communication skills, and to implement as many as possible. HCC overview

The rest of the story: A taxonomy of interaction behaviors • Verbal: symbolic and prosodic • Nonverbal communication theory (psychology, linguistics, anthropology). Main cues are: u u u Gaze Proxemics Facial expression Body and hand gesture Posture Touch • Example: A Handshake depends on all the cues. HCC overview

Differences between verbal and nonverbal communication • Adam Kendon (83) summarizes the difference very well: . . . gesture has properties different from speech. In particular, it employs space as well as time in the creation of expressive forms, whereas speech can use only time. We find, therefore, that the way information may be preserved in speech, as compared to gesture, tends to be very different. HCC overview

Why nonverbal skills are spatial • Gaze - very sensitive to mutual direction. • • • Center-right-left. Proxemics - Depends on distance to others and the surroundings. Facial expression - the least spatial, directed by gaze. Gesture - deictic gesture is completely spatial, so are other pictographic forms. Posture - Directional. Touch - Cant fake it without space. HCC overview

You need social context as well • Communication. • Persuasion. • Trust-building. • These are radically different contexts that rely on different non-verbal cues. We need to study telepresence in representative scenarios. HCC overview

The two-way street: • PRo. Ps provide selective control over particular cues. They can be turned on and off at will. • So they provide a potentially interesting testbed for finegrained study of non-verbal cues. • We can create a rich variety of media and study their influence on the overall perception of the other person. HCC overview

The input side • How do you achieve rich, expressive input for • PRo. Ps/avatars with today’s technology? Cross-modal mapping from pen gesture to body gesture. Both have symbolic and expressive character: HCC overview

Future research? • Gaze cue repair: Face to face is not perfect. There are common breakdowns in face-to-face communication, such as gaze aversion. • Beyond Being There: It may be possible to rectify gaze aversion using image processing and/or a synthetic gaze cue. HCC overview

Project 2: UPM: A universal planar manipulator • A smart desk that can move a large number of objects placed on it independently. • Provides a tangible interface in both directions for 2 D layouts: floor plans, landscapes, stage directions etc. • Uses a small numbers of motors (4) and the non-linearity of friction. HCC overview

UPM: A universal planar manipulator • First prototype used linear voice-coils from ancient disk drives, capable of about 2 g acceleration. New prototype should manage about 50 gs. HCC overview

Bearable Computing (new project) • An exploration of issues in personal, persistent computing (augmented reality, worn interfaces) using ordinary laptop computers. • Avoid head-mounted displays (expensive and low-res) headtracking, and cables. • The approach: use optics to overlay computer images on reality, but use laptop or pocket-mounted displays. • Testbed: Grad course in HCC next semester. HCC overview

An augmented reality classroom Virtual image with notes, questions, private chat space Glasses to rotate the laptop image Physical space HCC overview

An AR classroom • Students work in groups of 5 -7, communicating silently via pen or keyboard chat. • Each group has one main note-taker, the others add their own comments or questions to the transcript. • Students can mark up the group transcript, or the lecturer’s notes. There is non-archived chat also. • One student per group works as facilitator or TA, posing questions to the others, and testing understanding. HCC overview

Bearable computers on the go • The laptop can serve as a “bearable” computer, attached wirelessly to its owner. Heads-up virtual image Double-mirror element in glasses Pocket-worn portable LCD TV Wireless chordal keyboard or Palmpilot in pocket for input. Laptop in briefcase with wireless TV and keyboard TXs. HCC overview

Goals of this research • Can we enhance attention in class with live group chat/notetaking? • Do we get a richer transcript with collaborative note-taking? • Does the head-up display make note-taking less distracting? • Will people do more tasks with a mobile bearable computer because threshold for computer use is lower? HCC overview