HCI in Ubiquitous Computing AIM AI Media Lab

HCI in Ubiquitous Computing 양현승 AIM (AI & Media) Lab KAIST 전자전산학과 hsyang@cs. kaist. ac. kr http: //mind. kaist. ac. kr

Contents § HCI in U-C § § § HCI in U-C Embedding Interaction U-C HCI Researches KAIST, AIM Lab

Ubiquitous Computing § Ubiquitous computing is the method of enhancing computer use by making many computers available throughout the physical environment, but making them effectively invisible to the user (Mark Weiser, Xerox PARC) KAIST, AIM Lab

Ubiquitous Computing § We are surrounded by computing § Computing and processing is embedded into everyday devices § There are many computers/processors person § Information access and communication is possible virtually everywhere § Dedicated computing devices – information appliances – are all around us § Devices can be connected and networked § What gets us here? KAIST, AIM Lab

Ubiquitous Computing § Mark Weiser: Computers enter everyday life § Help people with everyday tasks in the office and at home (at any time, any place) A good tool is an invisible tool. By invisible, I mean that the tool does not intrude on your consciousness; you focus on the task, not the tool. [Weiser 94] KAIST, AIM Lab

HCI themes with U-Life § Three past interaction themes: § § § Natural Interfaces Context-Aware Interaction Automated Capture & Access to Life Experiences § New interaction theme proposed: § Everyday Computing KAIST, AIM Lab

Natural Interfaces § Forms of Natural Interfaces § § Speech, gestures handwriting (pen-based/free-form) § Issues Encountered § § Need a way to represent information with new interface Error-prone, even humans can’t perfectly read handwriting KAIST, AIM Lab

Context-Aware Interaction § What is appropriate context to use? § § Current – user and location Future – time, history, other users § How to represent this context? § Incorporate hierarchal info and relations § Truly Ubiquitous? § Limitation of many technologies. KAIST, AIM Lab

Context-Aware Interaction location identity objects KAIST, AIM Lab

Everyday Computing: Things to be Considered § § § No clear beginning & end to all activities Interruption is expected Multiple activities operate concurrently Time is important discriminator Associative models of information KAIST, AIM Lab

Embedding Interaction

U-Life Web servers Web browsers Electronic servers ? Mobile browsers KAIST, AIM Lab

Change of UI Paradigm Single Screen-based UI Interact with a number of U-devices (distributed + interconnected) Ubiquitous Computin g • Highly personal and mobile appliances • Systems that are integrated in everyday environment KAIST, AIM Lab

User interface in U-C § Requirements § Distribution of UI § All U-devices are distributed. § Implicit HCI § To reduce the need for explicit HCI § To let explicit interfaces virtually disappear into the environment § Awareness of the situation, the environment and the aims of the user § Being noticed only when needed KAIST, AIM Lab

User interface in U-C § Current Interaction § Explicit HCI § By command-line § By direct manipulation using a GUI, gesture, or speech input § Interaction in U-C § Implicit HCI § It allows the computer to interpret the user’s behavior and the surrounding situation and use this information as input KAIST, AIM Lab

What is different from traditional ‘HCI’ and ‘HCI in Ubi. Comp’ ? § Output modalities § § not just an audio visual channel all senses! § Input modalities § more than pressing buttons and moving an object in two dimensions § Distribution – physical and conceptual § Magic beyond the screen § … it is a vivid physical relationship KAIST, AIM Lab

Development Process? Research Approach? § Not anymore designing and programming a GUI § Interdisciplinary teams – ethnography, design, CS § It is about creating an experience by § § § Understanding the interaction and process Designing and constructing a set of devices and an environment Implement the human-information interface based on the created devices/environment Test it yourself Test it with users … go back an refine the hardware and start again KAIST, AIM Lab

Prototypes Functional prototypes are essential to learn, understand experience how to interact with the ubiquitous computer § From the idea to knowledge § § Prototyping has been central to hallmark research in the area (e. g. Parc. Tab, Active. Badge) Learning occurs when along the prototyping process as well as in use § Evaluation § § Functional prototypes are the means for evaluation “Confronting” real people – already with version 0. 001 Deployment in a living lab environment Facilitating everyday environments with real users KAIST, AIM Lab

Ubi-Comp Environment is itself the Interface § Everyday objects augmented with sensing § § table chairs glasses … § Creating a digital shadow reflecting the interaction KAIST, AIM Lab

Embedding Interaction § Basic technologies for embedding interaction § Sensing technologies § § § § Environmental conditions Users’ location Co-location with others Physiological and emotional state of the user User goals User schedules … Agent technologies § Combining a multitude of sometimes contradictory inputs to make sense at a higher level § Adopting a system’s output to be appropriate to whatever situation might arise KAIST, AIM Lab

Implicit Interaction (1/2) § Implicit Human-Computer Interaction (i. HCI) § i. HCI is the interaction of a human with the environment and with artifacts which is aimed to accomplish a goal. Within this process the system acquires implicit inputs from the user and may present implicit output to the user. § Implicit Input § Implicit inputs are actions and behaviour of humans, which are done to achieve a goal and are not primarily regarded as interaction with a computer, but captured, recognized and interpret by a computer system as input. § Implicit Output § Output of a computer that is not directly related to an explicit input and which is seamlessly integrated with the environment and the task of the user. KAIST, AIM Lab

U-C HCI Researches

OXYGEN Project Speech and vision technologies enable us to communicate with Oxygen as if we’re interacting with another person, saving much time and effort MIT Media Lab KAIST, AIM Lab

Aware. Home Designing the Interactive Experience § Digital Family Portrait § reconnects geographically distant extended family members by allowing them to remain aware of each other in a non-obtrusive, lightweight manner § What Was I Cooking? § a context-aware system that captures the transient information of recent activities and passively displays them as visual cues. § Gesture Pendant recognizes and then translates gestures into commands for your home appliances Aware. Home with human-like perception could improve quality of life for many, especially seniors. Georgia Tech. KAIST, AIM Lab

Easy Living(1) § § § Easy. Living is developing a prototype architecture and technologies for building intelligent environments § § System Architecture Person Detector person tracking Person Detector Seat Sensors Person Tracker PC Logon Fingerpri nt Logon Microsoft world model Agent Lookup World Model Key features § room control Room Control UI Rules Engine authentication Desktop Manage r § Room Lights A/V Media Systems § § Terminal Server § KB/Mou se Redirect Computer vision for persontracking and visual user interaction. Multiple sensor modalities combined. Use of a geometric model of the world to provide context. Automatic or semi-automatic sensor calibration and model building. Fine-grained events and adaptation of the user interface. Device-independent communication and data protocols. Ability to extend the system in many ways. KAIST, AIM Lab

Easy Living(2) § Personal Detection § § § Stereo Processing with commercial software Background subtraction and person detection Reports sent to central personal tracker about 7 Hz . . color depth people patches “Person creation zone” Past locations § Personal Tracking § New sensor measurement Process each new report from a sensor Predicted location Microsoft KAIST, AIM Lab

Home. Lab § Philips Home. Lab § § appearance looking and feeling like a regular home for testing its new home technology prototypes in the most realistic possible way § § WWICE PHENOM EASY ACCESS POGO: an interactive game for children § § virtual story world interfaced by active tools Intelligent Personal-Care Environment § based on measurements from the activity monitor and heart rate sensor Philips Research KAIST, AIM Lab

KAIST AIM Lab Research

Role of Wearable Computer in Ubiquitous Computing Environment § § § It easily acquires personal data (personalization). It guarantees safety of personal data (privacy). It enhances user’s interaction with many devices. It reduces network traffic about personal data transmitting. It assists us to work (agent). KAIST, AIM Lab

Background Various electronic media will be scattered around us in the near future (ubiquitous computing environments). We will frequently interact with those media. (We will feel much annoyed with this interaction. ) A system that assists us in interacting with those media in our daily life is required. • This system should understand a user’s intention or preference. • This system should communicate with various electronic media. KAIST, AIM Lab

Research Objective ♦ To Establish Some Concepts - IEM - IWAS ♦ To Propose an IWAS prototype and IEM prototypes ♦ To demonstrate interaction of IWAS and IEM KAIST, AIM Lab

IEM § Interactive Electronic Media § electronic media in ubiquitous computing environment § § that are not only controlled by a user’s command but that also respond to context or the user’s emotional state . . . Wireless Control IEM KAIST, AIM Lab

IEM § IEM Examples § IEM encapsulated electronic appliances § such as a TV, a video player, a radio, a computer, and etc. § Responsive digital media § interactive media artworks § All objects with embedded computer chips or sensors § an automatic curtain that rises or falls according to a user’s intention or preference § a lamp that intelligently controls the intensity of light according to a user’s emotional state § IEM Features § Wireless control => ultimately, automation (agent system) § Unique ID § Interaction capability KAIST, AIM Lab

IWAS § Intelligent Wearable Assistance System KAIST, AIM Lab

IWAS H/W Design KAIST, AIM Lab

IWAS H/W Design § Self-contained System to Wear § integrating all components of wearable computer with a suit § User-friendly Interface § § input: speech recognition, key-pad, mouse, etc. output: see-through HMD, small speakers. § Various Sensors § § FSR and postural sensing unit Infra-red tag reading unit § Wireless networking § Wireless LAN, IEEE 802. 11 b KAIST, AIM Lab

Functions of IWAS § Intelligent User-Assistance § § § local identification using IR sensor device direct control of IEM using IR remote controller communication via wireless LAN or Bluetooth information service such as schedule alert, email check interacting with media KAIST, AIM Lab

Functions of IWAS PC Audio Player Phone Home G/W Home Network (Home RF, IEEE 902. 11, ···) TV Ubi. Comp Environment Lamp Control Sensing • IEM identification using IR sensor • wireless control using IR remote controller Intelligent Agent • providing personalized information service KAIST, AIM Lab

IWAS H/W Prototype IR tag reader & IR remote controller See-through HMD with speech head set FSR sensor IWAS suit 3 -axis postural sensor KAIST, AIM Lab

Interaction with IWAS and IEM CASE 1: Operating a laptop computer CASE 2: Turning on TV KAIST, AIM Lab

Interaction with IWAS and IEM CASE 3: Controlling virtual system KAIST, AIM Lab