Interaction Contents n n Understanding Conceptualizing Interaction models

Interaction

Contents n n Understanding & Conceptualizing Interaction models Interaction styles Interaction paradigms 2

Understanding & Conceptualizing Interaction n It is tempting to design an interface by going directly to the nuts & bolts n n Whether to use touch screen GUI details Whether speech should be used While these decisions are necessary, they should be done after understanding the problem space 3

Understanding the Problem Space n Before designing a solution, you need to understand the problem n n n What is the problem with the current interface? What do the users find difficult? What is the functionality which the users want? Do all users have the same requirements? Are all users willing to put time into understanding the problem? 4

Example: Understanding Browser Deficiencies n n A major web browser has been losing market share to competitors The development team meets to discuss the problem After asking users, it is found that many have never used bookmarks They zero in on the bookmarks as a potential problem area 5

Example: Understanding Browser Deficiencies n There are various ideas to explain the problem n n n Organizing bookmarks requires dragging URLs to folders which is tedious and error-prone Another member agrees with this assessment based on feedback A software engineer, who scrolls through history rather than using bookmarks, assumes others do the same. He recommends that the history facility be removed due to it being overly complex and unused. 6

Example: Understanding Browser Deficiencies n The result of this discussion is: n n They have various theories as to what the problem is They still do not understand the real problem They must go back, ask more users for feedback, and think about the problem for a longer time You cannot create a solution until you understand the problem ! 7

Conceptualizing the Design Space n “A conceptual model is a high-level description of how a system is organized and operates” Johnson & Henderson n The conceptual model is how the user will see and understand the system 8

Conceptualizing the Design Space n “The most important thing to design is the user’s conceptual model. Everything else should be subordinated to making the model clear, obvious, and substantial. This is almost exactly the opposite of how most software is designed” Liddle 9

The Conceptual Model n n n Outlines what people can do with a product Presents the concepts necessary to interact with the product The metaphors and analogies used to convey an understanding of the product to the user 10

The Conceptual Model n The model n n n Gives the user an understanding of how to use the product Forms a shared vision between all of the designers Should be designed after the problem space is understood but before the rest of the solution is designed 11

Contents n n Understanding & Conceptualizing Interaction models Interaction styles Interaction paradigms 12

Interaction Models n n Interaction models can help us understand the interaction between the user and the system Terminology n Domain n n Important concepts about the problem space In graphic design these would be shapes, drawing tools and a drawing surface 13

Interaction Models n Tasks n n Goals n n The desired result of a task Intention n n Operations to manipulate the concepts in the domain A specific action designed to achieve a goal Task analysis n The identification of the problem space in terms of domain, tasks, goals and intentions 14

Interaction Models n Core language n n A language used to describe computational aspects of the state of the system Task language n Describes psychological aspects of the domain relevant to the state of the user 15

The Execution-evaluation Cycle n n n This is also called Norman’s model of interaction It is probably the most influential model It is broken into two major phases n n n Execution Evaluation These are further broken into a total of 7 stages 16

The Execution-evaluation Cycle 1. 2. 3. 4. 5. 6. 7. Establishing the goal Forming the intention Specifying the action sequence Executing the action Perceiving the system state Interpreting the system state Evaluating the system state with respect to the goals and intentions 17

The Execution-evaluation Cycle n n n Each of these stages is performed by the user The user determines a goal, figures out how to achieve it, performs the action, and then looks at the system to see if it worked as expected Norman uses this model to explain why users have problems with interfaces 18

The Gulf of Execution n Remember that the system operations are defined in terms of the core language and the user describes actions in terms of the task language Since these languages are not the same, it can result in misunderstandings This means that there is a gulf between the users actions to achieve a goal and the systems interpretation of those actions 19

The Gulf of Evaluation n The gulf of evaluation is the distance between the actual state of the system and the state expected by the user The more difficulty the user has interpreting the system state, the less effective the interaction Norman’s model provides a simple framework for the understanding of interaction 20

The Interaction Framework n n Norman’s model is for understanding the interaction It does nothing to consider the system’s side of the communication This is addressed by an extension to Norman’s model by Abowd and Beale This extension is the interaction framework 21

The Interaction Framework n This breaks the system into 4 major components n n n The The system user input output It uses the core language and the task language as well as languages for both input and output 22

The Interaction Framework There are 4 steps in the interaction cycle n 1. 2. 3. 4. The user formulates a goal and a task to achieve it. This is expressed in the input language. Input languages is translated into core language System transforms itself into a new state The state of the system is then rendered in the output language and sent to the user O output U S core task I input 23

The Interaction Framework There are 4 translations involved in the interaction n 1. 2. 3. 4. The users task must be articulated in the input language The input language is then translated into stimuli for the system Once the system state is updated, evaluation begins with the presentation of results in the output language Finally, observation translates the output language into understanding by the user presentation O output U S core performance observation task I input articulation 24

Articulation n Poor articulation n n A user wants to turn on lights at one end of a room The user is confronted with a bank of unlabeled switches The user has no idea of which switch controls the lights at the end of the room The problem is that the user cannot articulate the goal (turn on lights at one end) in the language provided (unlabeled switches) 25

Articulation n Good articulation n n Virtual reality systems use data gloves This allows every motion of the user’s hand to be translated directly into operations to be performed by the system This is an example of a direct manipulation interface It provides an excellent translation from the task language to the core language 26

Output n n Consider writing a paper with a word processor You must be able to see the effects of your actions Often, you can see only the immediate paragraph without the surrounding text where there might be changes as well This means that all of the state changes cannot be be displayed in the output language 27

Using the Interaction Framework n n n The interaction framework can be used to judge the usability of the interface By concentrating on translations, it allows us to determine if concepts are being communicated correctly This might be used as the basis for choosing one tool over another 28

Contents n n Understanding & Conceptualizing Interaction models Interaction styles Interaction paradigms 29

Interaction Styles n n The choice of interaction style has a profound impact on the nature of the interaction We will look at n n n n Command line interface Menus Natural language Question/answer dialog Forms WIMP Point and click Three-dimensional interfaces 30

Command Line Interfaces n n n The earliest interfaces were command line Despite the availability of more modern interfaces, they continue to be used Command line interfaces n Provide abbreviations and function keys to speed use 31

Command Line Interfaces n n n Provide direct access to commands without having to navigate menus Allow options to be used to modify command behaviour Are very powerful Have a steep learning curve before they can be used effectively This can be lessened by using meaningful command names 32

Menus n n These display the available choices The user can select one by mouse, numeric entry, or arrow keys Larger menus are organized hierarchically so not all of the menu is displayed at once This makes navigation more difficult since the user cannot see where they want to go 33

Menus n n The advantage of menus is that they lessen the load on the user’s memory since commands no longer have to be remembered This still means that the commands must be meaningful and easy to understand 34

Natural Language n n On the surface it is attractive to allow instructions in a natural language like English Problems are soon encountered due to the ambiguity of natural language n n The boy hit the dog with the stick Did the boy use a stick to hit a dog or did the boy hit a dog carrying a stick? 35

Natural Language n n There is also ambiguity in the meaning of individual words as well as the sentences Although research continues into natural language understanding, it remains problematic for the near term 36

Question/answer Dialogs n n The user is asked a series of questions and provided with a series of answers from which to choose These are easy for novices to use They restrict queries to those supported by the system General purpose query languages provide more flexibility but are more difficult to learn and use 37

Forms n n This is used primarily in data entry and occasionally in data retrieval The user is presented with a form that looks like a familiar paper form Advanced forms allow the user to leave fields blank and to fill in the fields in different orders A spreadsheet is a special type of form in which the results of changes can be seen immediately 38

WIMP Interface n n This is the familiar windowing system consisting of Windows, Icons, Menus and Pointers This has become a standard interface supported by many computer systems including n n n Microsoft Windows X Windows Apple Macintosh 39

WIMP Interface n Windows n n Areas of the screen where text and or graphics can be rendered Icons n n Small pictures representing concepts or actions Used for buttons, labels, and representing windows in collapsed form 40

WIMP Interface n Pointers n n Pointing devices such as mice, touchpads, trackballs, tablets, etc. These are essential to WIMP interfaces The pointer is represented on screen by a cursor which can take on a variety of shapes Often the shape of the cursor is used to indicate state information about the system 41

WIMP Interface n Menus n n These allow the user to issue commands by selecting from menus There are pull-down menus from a menu-bar as well as pop-up menus They all reduce the load on the user’s memory There also menus which are circular as well as ones which pick up gestures by the user with the pointing device 42

WIMP Interface n Buttons n n Buttons are small windows which can be clicked to perform an action Buttons can be labeled with text, graphics or both Buttons can change appearance to indicate their state Radio and checkboxes are buttons with a slightly different behaviour and appearance 43

WIMP Interface n Toolbars n n This is an area of the screen where buttons and icons are collected It acts similarly to a menu by allowing the user to select from many commands It often makes selections more obvious by showing them all at once Often, the user can customize their display by showing a selection of multiple toolbars which are available 44

WIMP Interface n Palettes n n A palette is a collection of controls which can alter the state of the system They usually select values for various options They provide immediate feedback on the state of these options Graphics systems often use palettes to se 4 lect the drawing color, line thickness, etc. 45

WIMP Interface n Dialog boxes n A dialog box is a window which conducts an interaction with the user to n n n Provide information Notify the user of an error Solicit input from the user Often, these dialogs are modal so that the user must interact with them before continuing System modal locks the entire windowing system and are usually a bad idea as the user cannot interact with other programs 46

Contents n n Understanding & Conceptualizing Interaction models Interaction styles Interaction paradigms 47

Paradigms for Interaction n n This section will look at the key technologies which have contributed to interactivity This will be partially historical in nature so that you can see how it evolved over time 48

Time Sharing n n n Initially, computers were batch systems used by a single user In the 1960 s advances in computing power made time sharing possible This allowed a single computer to be used by several individuals They were able to interact with it via teletypewriters Many people at the time were excited by the possibilities 49

Video Display Units n n n The teletypewriter gave way to the video display unit Ivan Sutherland invented Sketchpad, a program which allowed you to draw on the screen and have it stored in memory This formed the basis for all modern windowing systems and established VDUs and the main display technology 50

Programming Toolkits n n n Douglas Engelbart had the idea that the computer should be used to augment human capabilities, not just for data processing He created systems which were forerunners of modern word processors He advocated the creation of software toolkits which could be used to easily build ever more complex software 51

Personal Computing n n The 1970 s saw computing power offered to the masses The LOGO system was designed to allow children to draw simple graphics using a turtle which draw with its tail Alan Kay at Xerox Parc had a vision of the Dynabook, a handheld personal computer which is more advanced than what we have today The tablet PC is the closest thing to a Dynabook 52

Windowing Systems n n The success of personal computing with the introduction of the PC created a need for richer interaction Kay developed the WIMP interface at Xerox which produced the Xerox Star in 1981 One of the programmers from the project was hired by Apple…and the Mac was born Soon WIMP came to predominate 53

The Metaphor n n Metaphors are a good way to introduce users to new technology They allow people to understand something new based on something they already understand LOGO used the metaphor of a turtle dragging its tail on the ground Many systems are based on the metaphor of the office desktop 54

The Metaphor n Although the metaphor is useful to get users used to a new system it can be harmful if carried too far n n n the word processor looks and seems to act just like a typewriter Few people know that they should format paragraphs rather than putting an extra space between them Few people realize that space is a character, since it is just empty space on a typewriter 55

The Metaphor n n n The most extreme example of the metaphor is the virtual reality system This places the user in the metaphor, creating an alternate world These interpret the users normal actions as instructions to manipulate virtual objects in the virtual world 56

Direct Manipulation n Direct manipulation interfaces are characterized by rapid feedback on every action performed by the user This makes it appear that the user is manipulating an object in real time This is a very satisfying experience that mirrors what happens in the real world 57

WYSIWYG n n What You See Is What You Get These are interfaces that show you what the end result will look like in real time They are ubiquitous in word processing The trouble is that a lot of processing power is spent on it and sometimes it is less convenient to include diagrams this way than to use other techniques 58

Language vs. Action n n While direct manipulation is good for many tasks, it can be less efficient when applied to repetitive tasks Languages are better for this sort of task The user can describe the action to be performed and the system will interpret and perform the action as many times as desired This is why command languages are still used for certain types of tasks 59

Hypertext n n Vannevar Bush proposed the idea of interlinked text in 1945 Since then, it has been used for n n n Linking of text in help systems Linking of documents in the Web Among the problems of hypertext is the tendency for users to get lost and not be able to navigate back to a specific point or to forget their location 60

Multi-modality n Normally humans communicate with computers using two modes n n Typing, a haptic sense Reading, a visual sense Multi-modality uses the other senses including haptic and auditory This can allow more information to be conveyed at once 61

Computer Supported Cooperative Work n n The development of the network led to the desire for people to work on shared documents remotely This created n n n Email Chat systems Shared whiteboards Systems for the delivery of lectures remotely This area is also called groupware 62

The World Wide Web n n The introduction of the web in 1989 is the most important change in computing It moved the internet out of the research lab and made it available to all It formed a union of computing and communications It made everyone aware of computers, not just the few who used them on a daily basis 63

Agent-based Interfaces n n n An agent is something which performs some action on your behalf An email filter is an agent which identifies and flags junk email Help agents watch the user’s actions and make suggestions or provide help Some agents search the internet to find specific items Others can be dispatched to sites on the net to perform actions on your behalf 64

Ubiquitous Computing n n Ubiquitous or pervasive computing moves it out of beige boxes so that computing permeates our environment We have refrigerators which can surf the net for menus We have appliances which can be programmed from a PC anywhere on the internet We are developing RFID tags where items can report their location to a central computer 65

Sensor-based Interaction n n We are used to entering data into a computer What if the computer had sensors which gathered information from the environment? This happens with lights which detect someone entering a room Computers can have sensors which let them find out about the real world and respond appropriately 66

Context-aware Interaction n n Once the computer knows about the state of the environment, it can make decisions as to what is appropriate for the environment This can use artificial intelligence to take actions which are appropriate to what the user is currently doing 67