Continuous control of avatar gesture with dirty details

Continuous control of avatar gesture (with dirty details) Francesca Barrientos GUIR Meeting 22 October 1999 1

Goals To develop an interface for continuous control of avatar gesture To gain some understanding of the range of gesture necessary for communication and self expression 2

Gesticulation is gesture that cooccurs with speech Meaning of the utterance is divided up between the words and gestures Gestures derive meaning from timing with respect to speech 3

Issues Building the continuous control system Design kinematic mappings between the user’s movements and the avatars movements Measuring user’s satisfaction with expressive capabilities 4

System components User input Glove Tracker Voice Output Graphics Audio Virtual world and user management 5

Dirty details Graphics format renderer Humanoid avatar format Input devices Multi-user virtual world manager Audio conferencing technologies Hassles of using 3 rd party software and emerging/submerging technologies 6

Graphics VRML format VRML browser Web browser to support the VMRL browser H-anim Definition for humanoid model in VRML Joints, joint hierarchies, etc. Nancy is an h-anim model from Y 3 D 7

Picture of Nancy 8

Animation and control Java EAI Java libraries to allows a java applet to manipulate the VRML model within a VRML browser 9

Input devices 5 th Glove Bending of five fingers Roll and pitch angles Flock of Birds 6 DOF tethered tracker 10

Multi-user virtual world manager Deep. Matrix by Geometrek Pure Java VRML world server Java client runs as browser plugin Uses standard VRML browsers which support EAI 11

Deep. Matrix screen shot 12

Audio Would like to add audio conferencing capability to Deep. Matrix Java Media Framework? MS Net. Meeting? Maybe better just to simulate audio for user studies 13

Quick and dirty survey Asked avatar users their thoughts on gestures and avatars Sent to WEAVE list ----women+ interested in avatar communities One response 14

Response highlights [a] In which situations do you find [avatar gestures] most useful? Occasionally in general conversation for emphasis, but, as I am [a role playing game] mage as well as an empath in [real life], to enhance the effect of that. 15

Response highlights [b] Do you find the…gestures enjoyable to use? 1=not at all; 5=extremely 4 What do you like or dislike about using avatar gestures? Don’t like time involved in accessing each one…and in some cases having to send them to other person… 16

Response highlights [c] Do you think gestures are necessary for communicating in avatar worlds? 1=not at all; 5=extremely 5 Why or why not? Since no other body language so to speak is available, is an important way to tap into that aspect of communication 17

Response highlights [d] What, if anything, is missing from gestures in avatar worlds? Well, just being able to actually make your av do it…but I do realize how hard that is Needs to be more fluid if you know what I mean… 18

Measuring effectiveness Idea: borrow from studies such as Berger and Popelka Present twenty sentences to subjects who were to write down what they heard In one condition, sentences accompanied by appropriate gesture In secondition, no gesture 19

Installation Set up a station at, say, the Exploratorium and have kids communicate using avatars Follows in tradition of anecdotal and observational study of gesture (e. g. Efron 1972, Birdwhistell 1970) 20

The end Of this talk 21