05 830 Advanced User Interface Software Brad Myers

Course: l Course web page: http: //www. cs. cmu. edu/~bam/uicourse/830 spring 20 l Temporarily: http: //www. uicourse. org/ l Schedule / Syllabus: http: //www. cs. cmu. edu/~bam/uicourse/830 spring 20/schedule. html l Mondays and Wednesdays, 1: 30 PM - 2: 50 PM Room GHC 4301 l Last offered 2017 l l See previous schedule, homeworks, etc. http: //www. cs. cmu. edu/~bam/uicourse/830 spring 17 © 2020 - Brad Myers 2

Instructor l Brad Myers l l l l Human Computer Interaction Institute Office: Newell-Simon Hall (NSH) 3517 Phone: x 8 -5150 E-mail: bam@cs. cmu. edu http: //www. cs. cmu. edu/~bam Office hours: By appointment or drop by No TA © 2020 - Brad Myers 3

Acknowledgements l l Thanks to Ph. D student Mary Beth Kery for consultations on

Readings and Homeworks l Schedule of readings: http: //www. cs. cmu. edu/~bam/uicourse/830 spring 20/schedule. html l l Links to the slides Course schedule is tentative Note required readings CMU-only and ACM DL, use CMU network or VPN l Homeworks http: //www. cs. cmu. edu/~bam/uicourse/830 spring 20/homeworks. html l No midterm or final l Create a framework for UI software l l Like SVG / Angular. JS or Flutter or Flash / Flex or Amulet / Garnet See homework policies l l Java for Swing Or in Java. Script for Canvas Due before class on the scheduled date, homeworks can be turned in late for a penalty, homeworks are individual, no cheating, turn in on Canvas. Schedule + Homeworks = Syllabus © 2020 - Brad Myers 5

Final Project l Create your own tool l l l Toolkit Interactive Tool Can be a reimplementation or novel & publishable Will be in groups Start April 1 Some ideas: http: //www. cs. cmu. edu/~bam/uicourse/830 spring 20/finalproject. html © 2020 - Brad Myers 6

What is this class about? l “User Interface Software” l l All the software that implements the user interface “User Interface” = The part of an application that a person (user) can see or interact with (look + feel) l l l “Implements” – course will cover tools that let you code the UI l Not covering the design process or how to use the APIs We will cover design & prototyping tools, & evaluation of tools User Interface Software Tools l l Often distinguished from the “functionality” (back-end) implementation Some ambiguity if a tool is specifically for “UI” part Ways to help programmers create user interface software Tools used by UI Designers and UI Builders © 2020 - Brad Myers 7

Names l Tools = UI Frameworks, Toolkits, Development Kits = SDK, Libraries l l UI APIs = Application Programming Interfaces (APIs) Old names = User Interface Management Systems (UIMS), User Interface Development Environments (UIDE) Micro-Service Architecture for web UIs Also interactive tools: Resource Editors, Interface Builders, Prototypers, UI Builders © 2020 - Brad Myers 8

Examples of UI Software Tools l We created a list in 2017 = https: //docs. google. com/document/d/1 qe. Bk. Qao. IBmw. K 9 Z 7 Lm. WP 4 L_4 GKgddz. Nn 7 Ymv. SQu. Puk. Go l l l Visual Basic. Net Adobe Flash, Adobe Catalyst Prototypes like Axure, Balsamiq, Adobe XD Research systems: l l l Java. Script/Web: SVG, Angular. JS, React. JS Apple Cocoa, Carbon Microsoft Foundation Classes, . Net, wx-Python Java AWT, Swing, Android UI classes Interactive tools l l Over 100! APIs for UI development: l l l Will make an updated list this year Garnet Amulet Constraint. JS Inter. State …What else? © 2020 - Brad Myers 9

What Will This Class Cover? l l l Principles, models and architectures for tools Current tools and their design History of User Interface Software Tools l l Future of UI Software Tools l l What has been tried What worked and didn’t Where the currently-popular techniques came from Current research How to evaluate tools l Good or bad © 2020 - Brad Myers 10

Why is This Important? l l Virtually all UIs are created using such Tools Previous research has influenced today’s tools l l New tools are created all the time l l Avoid “reinventing the wheel” What are the “best practices” for these tools Modern UIs and Tools for web, phones, wearables, Smart TVs, etc. all use similar designs l l E. g. , Flutter for Dart language and mobile Some are easier to use than others! Principles and architectures for good designs l l Enormous impact! Speech and conversational interfaces are different Research topic in ACM UIST, CHI l Also ICSE, SPLASH, PLATEAU, CHASE, many others © 2020 - Brad Myers 11

Analogy with OS or Compilers l In CS curriculums, often teach Operating Systems, Compilers, Networking l l Some people will need to know how to build them l l Even though most people will just use these Behind-the-scenes The algorithms, architectures, design patterns, and principles are useful in other situations as well © 2020 - Brad Myers 12

Homework 1 l l l http: //www. cs. cmu. edu/~bam/uicourse/830 spring 20/homework 1. html Assigned today; due in 2 weeks = Monday, January 27, 2020 Implement a specified design using the tools of your choice l l l But must be Java or Java. Script Flexibility in some aspects of the design Early example of expected level of programming l ~ 700 LOC © 2020 - Brad Myers 13

Lecture 1: Overview of UI Software and Tools Brad Myers © 2020 - Brad

Stakeholders l l Many groups of people Need unambiguous names l l l Too many “users” Tool Designers (= you!) Tool Users (are programmers) Users of products created with the tools = consumers or end-users Source: Jeffrey Stylos and Brad Myers, "Mapping the Space of API Design Decisions, " 2007 IEEE Symposium on Visual Languages and Human-Centric Computing, VL/HCC'07. Sept 23 -27, 2007, Coeur d'Alene, Idaho. pp. 5057. pdf © 2020 - Brad Myers 15

Who Are Developers? l Programming tools are not just used by highly-trained professional programmers l End-User Programmers = People whose primary job is not programming In 2012 in USA at work: — [Scaffidi, Shaw and Myers 2005] l l l l 3 million professional programmers 6 million scientists & engineers 13 million will describe themselves as programmers 55 million will use spreadsheets or databases at work 90 million computer users at work in US All these may be the tool users! © 2020 - Brad Myers 16

UI Runtime Pipeline Hardware Sensor Device Driver OS Toolkit input Application Code output Screen

UI Runtime Pipeline w/ Browser Hardware Sensor Device Driver OS Browser Toolkit Html / canvas spec Toolkits input Application Code output Toolkits Screen Driver Device Driver OS Drawing Package © 2020 - Brad Myers Toolkit Html / canvas spec 18

UI Tools stack Interactive Tools (Builders, Prototypers) Framework (Architecture, Objects) Toolkit (library, programming interface) OS / Windows Interface (Input and Output) Device Drivers & Hardware © 2020 - Brad Myers 19

From the bottom: Windows & OS l Window System + Operating System l l Unix & older OS’s separated OS, Windows l l l Sun. OS: X Windows or Ne. WS or Sun. Tools Low level input events – keycodes, mouse position, values from accelerometers Low level graphics primitives l l Microsoft Windows, Mac. OS, Android, i. OS, etc. Draw Circle, Draw Line, set pixel color Clipped to window boundaries © 2020 - Brad Myers 20

Toolkits l l (Specific meaning, one part of the tool set) A library of procedures l l Provides higher-level “widgets” l l l Only a programming interface Also called “controls” Scroll bars, buttons, text input fields Examples: l l l Html, canvas, svg Java Swing, SWT, AWT Win 32, Macintosh “toolbox” © 2020 - Brad Myers 21

Frameworks l l Higher-level programming architecture Common design patterns l l l Significantly affects design of applications Often object-oriented l l l React native, Flutter, Microsoft’s Xamarin, Titanium, … Electron (https: //electronjs. org/): cross-platform toolkit for desktop apps Sometimes hard to distinguish from “toolkits” l l “Foundation Classes” Often cross-platform (i. OS + Android) l l Listener pattern, data bindings, etc. (So we usually won’t!) Other Examples: l l Historical: Apple Mac. App, my Amulet Current: Unity © 2020 - Brad Myers 22

Interactive Tools l l l Not a programming interface Supports designers who might not be programmers Select widgets and place them l l l Prototypes or real code l l Layout, possibly with constraints Specify properties of widgets For real code, often built into IDEs Examples: l l l Adobe Dreamweaver for web pages Prototypers: Balsamiq, Axure, etc. Resource editors & builders: Eclipse, Xcode IB, Android studio © 2020 - Brad Myers 23