stanford hci group cs 376 User Interface Toolkits

stanford hci group / cs 376 User Interface Toolkits Jeffrey Heer · 26 May 2009 http: //cs 376. stanford. edu

Final Project Presentations Tuesday June 9, 3: 30 -6: 30 pm, 104 Gates 8 minute presentations 6 min for research, 2 min for questions Start with an overview: 1 sentence statement of your research result 1 slide / 4 sentences of what you did and why Rest of time on details. Assume audience is familiar with HCI issues: focus on your work 2

In the beginning… cryptonomicon. com/beginning. html

The Xerox Alto (1973)

Event-Driven UIs Old model (e. g. , UNIX shell, DOS) Interaction controlled by system, user queried for input when needed by system Event-Driven Interfaces (e. g. , GUIs) Interaction controlled by user System waits for user actions and then reacts More complicated programming and architecture

Periodic Table of Motif Widgets

Toolkit Example: Java Swing GUI toolkit with a widget set and an API 7

User Interface Builders 8

Event Dispatch Loop Event Queue • Queue of input events Mouse moved (t 0, x, y) Event Loop (runs in dedicated thread) • Remove next event from queue • Determine event type • Find proper component(s) • Invoke callbacks on components • Repeat, or wait until event arrives Component • Invoked callback method • Update application state • Request repaint, if needed

Event Dispatch Event Queue • Mouse moved (t 0, x, y) • Mouse pressed (t 1, x, y, 1) • Mouse dragged (t 2, x, y, 1) • Key typed (t 3, ‘F 1’) • … Window (queues and dispatches incoming events in a dedicated thread) Panel Label Text. Area /* callback for Text. Area */ public void mouse. Moved(e) { // process mouse moved event } Panel Button

Success of Tools Today’s tools are highly successful Window Managers, User Interface Toolkits, and Interface Builders are ubiquitous Most software built using them Are based on many years of HCI research: Brad A. Myers. “A Brief History of Human Computer Interaction Technology. ” ACM interactions. Vol. 5, no. 2, March, 1998. pp. 44 -54. 11

Why Tools? The quality of the interfaces will be higher. Why? Rapid prototyping. Easier to incorporate changes motivated by evaluation. Re-use affords investment in high quality tools. Consistency of interface design. Enable collaboration among specialists. 12

Why Tools? The user interface will be easier to create and maintain. Why? Less code to write due to component re-use. Better modularization, separation of concerns Tools may abstract complex systems or algorithms. Easier to port an application to different hardware or software environments if device dependencies are isolated in the user interface 13

What should tools do? Help design the interface given a specification of the tasks. Help implement the interface given a design. Help evaluate the interface after it is designed. Create easy-to-use interfaces. Allow the designer to rapidly investigate different designs. Allow non-programmers to create user interfaces. Provide portability across different machines and devices. Be easy to use themselves. 14

Tools might do: Provide sets of standard UI components Guide the implementation Help with screen layout and graphic design. Validate user inputs Handle user errors Handle aborting and undoing of operations Provide help and prompts Deal with field scrolling and editing Insulate the application from all device dependencies and the underlying software and hardware systems. Support features in the interface that allow the end user to customize the interface. 15

Application Types What application domains are “deserving” of specialized toolkit support? 16

Application Types Each has own unique UI style, and implementation challenges Word processors Drawing programs (CAD/CAM) Painting programs Mail readers Spreadsheets Programming environments / code editors WWW Interactive games Visualizations Automated-teller machines (ATM) Virtual Reality Multi-media Video Animation Controlling machinery 17

Diamond. Spin Toolkit for tabletop user interfaces [Shen, Vernier, Forlines, Ringel] CHI ’ 04 18

19

Tabletop UI Needs Multi-user support Identity-aware widgets Multiple menus Public and personal spaces Resolving conflicting actions Arbitrary orientation of UI elements Techniques to control orientation and layout Rotation sensitive components 20

i. Stuff Toolkit Physical UI components for ubiquitous computing environments (multiple users, devices, and applications) [Ballagas, Ringel, Stone, Borchers], CHI ‘ 03 21

i. Stuff Design i. Stuff components Device + proxy (“smarts” are in the proxy) Patch. Panel Translate between i. Stuff events and application-specific events Run-time retargetable events Address dimension mismatches 22

23

Discussion of Themes Address the useful & important aspects of UIs Narrower tools have been more successful than ones that try to do “everything” Do one thing well Threshold / Ceiling Research systems often aim for high ceiling Successful systems often seem to instead aim for a low threshold Impossible to have both? 24

Threshold & Ceiling Programming in C Visual Basic Director (v 6) Hyper. Card MFC C Programming Difficulty of Use nd a k Clic e t Crea x. Cmds Lingo Hyper. Talk Basic Goal Sophistication of what can be created 25

Discussion of Themes, cont. Path of Least Resistance Tools should guide implementers into better user interfaces Goal for the future: do this more? Predictability Programmers do not seem willing to release control Especially when system may do sub-optimal things Moving Targets Long stability of Macintosh Desktop paradigm has enabled maturing of tools 26

Evaluating User Interface Tools “An API is a user interface where programmers are the users” ? Evaluate toolkit as you would a UI? Factors Expressiveness (Ceiling) Development Rate (of skilled user) Learning Rate (to become skilled) Performance Portability 27

The Future of Interface Tools Supporting… Prototyping Collaboration Evaluation of interfaces built with tools of tools themselves – how to prototype, test, iterate? Emerging interface styles, such as mobile recognition-based UIs (speech, pens, vision) multiple devices 28

Current Research Challenges Complex design space e. g. , do we have to update the toolkit every time someone creates a new sensor or actuator? Ambiguous input Speech, gestures, computer vision, etc. aren’t recognized as accurately as mouse clicks. Should the toolkit handle the recognition? 29

Summary Toolkits provide reusable interface components to simplify UI development Toolkit trap: it’s tempting to only make UIs that the toolkit makes easy, instead of making what’s best for a specific app Toolkit types: WIMP (Garnet, Swing, Motif, etc) Specialty (Phidgets, i. Stuff, Papier-Mache, Diamond. Spin, Group. Kit, Peripheral Displays Toolkit, etc) 30

Next Time: Adaptive Interaction You should use Sharon’s online paper viewer for the next set of readings. 31