HumanComputer Interaction A Computer Science Perspective Benjamin Lok

Human-Computer Interaction A Computer Science Perspective Benjamin Lok September 20 th, 2004

Outline n HCI Computer Science take n Research n n My Research Virtual Reality n Interactive Virtual Characters n

HCI Community n Academics/Industry Research n n Experimenters n n n Taxonomies Theories Predictive models Empirical data Product design Other areas (Sociologists, anthropologists, managers) n n Motor Perceptual Cognitive Social, economic, ethics

Computer Science n Preliminary Evaluate interfaces n Design new approaches n Command Line->Direct Manipulation n n Current New interfaces n Mobility, Immersion, n Helping companies develop better products n n Hardware, software, interaction technology

Computer Science HCI n How do you design interfaces to systems for: Users with disabilities n Children n Elderly n Culture and international diversity n Cognitive diversity n Physical n Universal usability n

Case Study: Library of Congress Database Design n http: //catalog. loc. gov/ n Two interfaces n Catalog New Books n n Search Catalog of Books n n n General public – too complex, command language and complex cataloging rules Solution n n 3 -6 hour training course - staffers Touch screen Reduced functionality Better information presentation Eventually Web based interface Same database and services, different interfaces

Example n Five fastest places to click on for a righthanded user?

Example n What affects time?

Human Computer Interaction n We see this all the time. n What’s good about the design of this error box? n n The user knows there is an error What’s poor about the design of this error box? n n n Discouraging Not enough information No way to resolve the problem (instructions or contact info)

My Choice n n i. Pod by Apple Computers Pros: n n n portable power ease of use # of controls Cons: n n n scratches easily no speech for car use proprietary

HCI Tools n n n Sound 3 D Animation Video Devices n n n Size (small->very large) Portable (PDA, phone) Plasticity Context sensitive/aware Personalizable Ubiquitous

Usability Requirements n Goals: n n Usability Universality Usefulness Achieved by: n n Planning Sensitivity to user needs Devotion to requirements analysis Testing

Bad Interfaces n n n Encumbering Confusing Slow Trust (ex. windows crashing) What makes it hard? n n Varies by culture Multiple platforms Variety of users Think of a game you’ve played with a bad interface n UNIX

Standardization, Integration, Consistency, Portability n Standardization – common user-interface features across multiple applications n n Integration – across application packages n n n Apple Web Windows file formats Consistency – common action sequences, terms, units, layouts, color, typography within an application Portability – convert data and interfaces across multiple hardware and software environments n Word/HTML/PDF/ASCII

Accommodating Hardware and Software Diversity n n Support a wide range of hardware and software platforms Software and hardware evolution n OS, application, browsers, capabilities backward compatibility is a good goal Three major technical challenges are: n n n Producing satisfying and effective Internet interaction (broadband vs. dial-up & wireless) Enabling web services from large to small (size and resolution) Support easy maintenance of or automatic conversion to multiple languages

HCI Goals n Influence academic and industrial researchers n n n Provide tools, techniques and knowledge for commercial developers n n Understand a problem and related theory Hypothesis and testing Study design (we’ll do this!) Interpret results competitive advantage (think ipod) Raising the computer consciousness of the general public n n Reduce computer anxiety (error messages) Common fears: n n I’ll break it I’ll make a mistake The computer is smarter than me HCI contributes to this!

Near & Future Interfaces • Time to learn • Speed of performance • Rate of errors Let’s review • Retention over time n Minority Report n Steel Battalion • Subjective satisfaction n Eye Toy n Dance Revolution n

Overview n Computer generated characters and environments n n Amazing visuals and audio Interacting is limited! n Reduce applicability? n Goals: n n Aki from Final Fantasy: The Spirits Within Create new methods to interact Evaluate the effectiveness of these interaction methods Walking Experiment PIT - UNC

Collaborators Xi Yong Wang, Aaron Kotranza, Benjamin Lok University of Florida Danette Allen NASA Langley Research Center

Virtual Environments n n n Been around for almost 30 years # of systems in research labs > day to day use Why? n n Interaction with the virtual environment is too poor Everything is virtual isn’t necessarily good n n Example, change a light bulb Approach: n n n Real objects as interfaces to the virtual world Merge the real and virtual spaces Evaluate what VR is good for!

n n n Getting real objects into VR to aid engineering design Collaboration w/ Mars Airplane (Langley Research Center) Get tools, parts, and other (possibly distributed) collaborators in a shared space

Immersive Virtual Characters for Educating Medical Communication Skills J. Hernendez, A. Stevens, D. S. Lind Department of Surgery (College of Medicine) M. Duerson Department of Community Health and Family Medicine (College of Medicine) K. Johnsen, R. Dickerson, A. Raij, B. Lok Department of Computer and Information Science and Engineering (College of Engineering) The University of Florida, Gainesville, FL J. Jackson, M. Shin Department of Computer Science The University of North Carolina at Charlotte, NC

What is a Virtual Character? n n Virtual character - a character who represents the state of a system In TRON (1982), humans and humans that represents software interacted within a world that represented the hardware.

What is a Virtual Character We look to to have humans and human the represent software interact in the real world. http: //movies. yahoo. com/shop? d=hv&id=1807432839&cf=trailer n

Motivation n n “Doctor, I have a pain in my side!” What you do next depends on: n n n Age Gender Ethnicity Visual Cues Audio Cues

Motivation n Medical Diagnosis Components n n n Current Teaching Methodologies n n n Books Standardized Patients (actors) Results in n n Patient Interview Physical Examination Minimal training frequency Minimal scenario variety Lack of immediate feedback Medical students are not adequately prepared for many diagnosis

Solution n Interactive Virtual Characters n n n Life-Sized Computer Generated Natural Interaction Responds to User Responses based on accepted medical procedure n COTS Equipment: n n Projectors Laptops Web cameras Tablet PC

Solution n Interactive Virtual Characters n n Computer Generated Natural Interaction Responds to User Responses based on n Results in: n n n More Scenarios More Training Standardized Experiences

Why do we want digital characters n n Propose: Digital Characters as a new (meta-) medium to interact with information Why would we want a digital character? n Effective Interaction n n n Better than keyboard and mouse for certain tasks Dynamic (output easily augmentable) 3 D Natural interaction Low Bandwidth Effective Collaboration n n Controlled conveyed visual information Non-verbal communication (60%)

Let’s look at interaction n n Each participant in a communication has three stages: perception, cognition, and response Define interaction as both the input and output Digital Character Respondin g Thinking Interaction Perceivin g Participant Perceivin g Respondin g Thinking

Combines Speech n Computer Vision n Eye Gaze n Gesture Recognition n Repeat your gestures n n High Quality 3 D models Animation n Rendering n Visualization n

Integrates n Computer Science n n Medicine n n Training Education Standardization Education n n Computer Graphics Image Processing Natural Language Processing Multimedia Learning Technology based Learning Training What other areas could this be used for? n n n Psychology Social Science Education

Current and Future Work n Current Status: Initial scenario created with gesture, speech, and visualization components integrated n Evaluating with a group of Medical Students Year 2 n n Future Work: Formal evaluation studies n Increase and improve scenarios n Enhance interactivity n n Show video