UCI Details Basic Services UCI Basic services Service

UCI Details

Basic Services UCI • Basic services Service Description Default Agents reusable data collection agents providing generic functionality (e. g. , “Use” and “Value Provided”) Agent Service generic data abstraction, selection, and reduction services and GUI for authoring agents Event Service generic event and state monitoring services Default Model toolkit-specific “wrapper” providing generic access to Java’s standard GUI components, events, and state • Basic services API Method Description void begin() to initiate monitoring and data collection void end() to initiate data reporting void set. Name(Object source, String name) to uniquely name components of interest that could not be uniquely named by the Default Model 2

Event Service UCI • Events referred to using “event specs” Event Spec Description MOUSE_CLICKED|Window/Print/OK|* All mouse clicks in the “Print” window “OK” button component MOUSE_CLICKED|*|javax. swing. JButton All mouse clicks in components of class “javax. swing. JButton” MOUSE_CLICKED|*|* All mouse clicks *|Window/Print/OK|* All events in the “Print” window “OK” button component *|*|javax. swing. JButton All events in components of class “javax. swing. JButton” *|*|* All events • Event service API Method Description void publish(String event. Name) make events available for subscription void subscribe(Subscriber subscriber, String event. Spec) register interest in events matching event. Spec void unsubscribe(Subscriber subscriber, String event. Spec) unregister interest in events matching event. Spec void post(Object source. Id, String event. Name) create and dispatch an event to interested parties String get. State(String key) retrieve global or component state void set. State(String key, String value) store global state 3

Event Dispatch UCI • Hashtable with entries for each unique event spec for which there are one or more subscribers – keys => event specs – values => lists of subscribers • For each posted event, subscribers retrieved for following event specs and notified in subscription order – EVENT_NAME|SOURCE_NAME|* – EVENT_NAME|*|SOURCE_CLASS – *|SOURCE_NAME|* – *|*|SOURCE_CLASS – *|*|* 4

Agent Service UCI Attribute Description Trigger List of event specs and composition operator (e. g. , OR, AND, THEN) Trigger Guard Constraints on trigger source (e. g. , Source. Is. New, Source. In. Set, Source. Not. In. Set, Source. Class. Not. In. Set) Guard Constraints on global and/or component state (e. g. , global flag = “true”; global counter > “ 5”; component value = “”) Action Event to post including event name (e. g. , “MENU”) and event source (No Source, Trigger Source, Agent Source) List of global state updates (e. g. global flag = “true” or global counter = global counter + “ 1”) Event Data List of event specs to be recorded in addition to Trigger events and an event data reduction algorithm (Events, Transitions, Sequence) State Data List of global and/or component state variables and a state data reduction algorithm (Values, Vectors, Value Per Event) User Data User notification specification including type (None, Non-Intrusive, Intrusive), header, and message 5

Agent Types UCI • Single-triggered Begin Trigger Guard Action Data (Event, State, User) • Double-triggered Begin Trigger Guard Action Data (Event) End Trigger Guard Action Data (State, User) 6

Agent Algorithm UCI IF (Satisfied Begin. Trigger) IF (Is. True Begin. Trigger. Guard & Is. True Begin. Guard) IF (Enabled Event. Data) Begin. Recording Event. Data // record begin time IF (Is. Single. Triggered Agent) IF (Enabled Event. Data) End. Recording Event. Data // record event that satisfied this trigger; elapsed time = 0 IF (Enabled State. Data) Record State. Data // record state IF (Enabled User. Data) Record User. Data // notify user; user may provide feedback IF (Enabled Begin. Action) Perform Begin. Action // generate abstract event and/or update global state ELSE IF (Is. Dual. Triggered Agent) IF (Enabled Event. Data) Record Event. Data // record event that satisfied this trigger IF (Enabled Begin. Action) Perform Begin. Action // generate abstract event and/or update global state Disable Begin. Trigger // disable begin trigger (until end trigger satisfied) Enable Data. Trigger // enable data trigger (until end trigger satisfied) Enable End. Trigger // enable end trigger (until end trigger satisfied) Pass event to End. Trigger // event that satisfied this trigger may also help satisfy end trigger IF (Satisfied Data. Trigger) IF (Enabled Event. Data) Record Event. Data // record event that satisfied this trigger IF (Satisfied End. Trigger) IF (Is. True End. Trigger. Guard & Is. True End. Guard) IF (Enabled Event. Data) End. Recording Event. Data // record event that satisfied this trigger; elapsed time = end - begin IF (Enabled State. Data) Record State. Data // record state IF (Enabled User. Data) Record User. Data // notify user; user may provide feedback IF (Enabled End. Action) Perform End. Action // generate abstract event and/or update global state Enable Begin. Trigger // reset begin trigger Disable Data. Trigger // reset data trigger Disable End. Trigger // reset end trigger 7

UCI 8

UCI Problems & Solutions

Abstraction Problem UCI • Observation – questions about usage occur in terms of concepts at higher levels of abstraction than represented in software component data – questions of usage may occur at multiple levels of abstraction • Data quality – difficult to relate data to concerns about design and use – difficult to identify patterns that might otherwise be obvious • Implication – need for “data abstraction” mechanisms to allow low-level data to be related to higher-level concepts such as UI and application features as well as users’ tasks and goals 10

Abstraction Solution UCI • Abstract interaction events – “Use Text”, “Use Non-Text”, “Use New” – “Value Initial”, “Value Provided” • Relating data to UI features – “Menu”, “Toolbar”, “Window” agents • Relating data to application features – “Command” agents • Relating data to users’ tasks and goals – “Section” agents 11

Selection Problem UCI • Observation – amount of data necessary to answer usage questions typically a small subset of the much larger set of data that might be captured • Scalability – failure to select increases amount of data that must be reported • Data quality – difficult to identify patterns that might otherwise be obvious – need to group data and define “contexts” for analysis • Implication – need for "data selection" mechanisms to allow data of interest to be captured and grouped prior to reporting and analysis 12

Selection Solution UCI • Selecting events – “Menu”, “Toolbar”, “Command” agents – “All Menus”, “All Toolbars”, “All Commands” agents • Selecting event contexts – “Window” agents • Selecting state – “Window” agents 13

Reduction Problem UCI • Observation – much of the analysis that will ultimately be performed to answer usage questions can actually be performed during data collection • Scalability – failure to reduce increases amount of data that must be reported • Data quality – when analysis is left as a final step, it can be difficult to perform (due to forgotten or faulty assumptions) or may fail to be performed at all (due to missing context and/or lack of automation) • Implication – need for "data reduction" mechanisms to automate reduction and reduce amount of data that must be reported analyzed 14

Reduction Solution UCI • Reducing event data – events, transitions, sequences • Reducing state data – values, vectors, value per event 15

Event Reduction (Events) UCI 16

Event Reduction (Transitions) UCI 17

Event Reduction (Sequences) UCI 18

State Reduction (Values) UCI 19

State Reduction (Vectors) UCI 20

State Reduction (Value Per Event) UCI 21

Context Problem UCI • Observation – potentially critical information necessary in understanding usage often not available in event data alone – such information may be spread across multiple events or missing altogether, but is often available “for the asking” from the user interface, application, artifacts, or user • Data quality – lack of context makes interpretation challenging at best, and impossible or misleading at worst • Implication – need for "context-capture" mechanisms to allow important UI, application, artifact, and user state information to be used in data abstraction, selection, and reduction 22

Context Solution UCI • User interface state – “Print Window”, “Enter ZIP to complete City/State” • Arbitrary state – “OK to select mode of travel”, “Not OK to select mode of travel” – “Menu count increment”, “Menu count reset”, “Menu count > 5” • Application state – “File Type” • Artifact state – e. g. , document content statistics • User state – “Mode of travel reselected” 23

Evolution Problem UCI • Observation – data collection needs typically evolve over time (perhaps due to results of earlier data collection) more rapidly than do applications • Scalability – unnecessary coupling of data collection and application code increases cost of evolution and impact on users • Implication – need for "independently evolvable" data collection mechanisms to allow incontext data abstraction, selection, and reduction to evolve over time without impacting application deployment or use 24

Reference Architecture UCI Data Capture Data Pre. Processing Data Packaging Data Transport Data Preparation Data Analysis System Model of UI & App: Analyst Model of UI & App: Components Events State Human-Readable Names Mapping 25

Instrumentation-Based Approach UCI Instrumentation Intertwined w/ Application Data Capture Data Pre. Processing Data Packaging Data Transport Data Preparation Data Analysis System Model of UI & App: Analyst Model of UI & App: Components Events State Human-Readable Names Mapping 26

Event Monitoring-Based Approach UCI Event Monitoring Infrastructure Data Capture Data Pre. Processing Data Packaging Data Transport Data Preparation Data Analysis System Model of UI & App: Analyst Model of UI & App: Components Events State Human-Readable Names Mapping Database 27

The Proposed Approach UCI Event Monitoring Infrastructure Data Capture Data Pre. Processing Data Packaging Data Transport Data Preparation Data Analysis “Pluggable” Data Abstraction, Selection, & Reduction Code System Model of UI & App: Analyst Model of UI & App: Components Events State Human-Readable Names Mapping Database or Automatic Naming 28

UCI 29

UCI Implementation

A Layered Arcitecture UCI • A layered architecture User Agents Default Agents Agent Service Event Service Default Model Application 31

Application UCI • Application – components – relationships – event data – state data Application 32

Default Model UCI • Default Model – component naming – relationship monitoring – event monitoring – simplified access to state Default Model Application 33

Event Service UCI • Event Service – event publishing – event subscription • NAME|SOURCE_TYPE – event posting – event notification – state set and get methods Event Service Default Model Application 34

Agent Service UCI • Agent Service – triggers • begin and event patterns – guards • trigger source • component and global state Agent Service – actions • post an event Event Service • update global state – data specs • event data • state data • user data Default Model Application 35

Default Agents UCI • Default Agents – e. g. , “Use” – e. g. , “Value Provided” Default Agents Agent Service Event Service Default Model Application 36

User Agents UCI • User Agents – e. g. , menus, toolbars, dialogs, commands, menu count, file type – e. g. , section sequence constraints User Agents Default Agents Agent Service Event Service Default Model Application 37

UCI 38

UCI Usage Scenario

Usage Scenario UCI • A cargo query interface 40

Usage Scenario (cont’d) UCI • Developers author agents to collect data and feedback 41

Usage Scenario (cont’d) UCI • Agents monitor use and collect data unobtrusively • Agents may post messages (optional) 42

Usage Scenario (cont’d) UCI • Users may learn more about expectations (optional) • Users may provide feedback (optional) 43

Usage Scenario (cont’d) UCI • Data and feedback stored for later analysis and review 44

Usage Scenario (cont’d) UCI Trigger BEGIN_EVENT|*|* OR USE|Window/Cargo Query/Reset|* OR USE|Window/Cargo Query/Submit|* Action Update. State(“Ok to select mode”, True) Trigger USE|*|* Trigger Guard Source. Not. In. Set( Window/Cargo Window/Cargo Action Update. State(“Ok to select mode”, False) Trigger USE|Window/Cargo USE|Window/Cargo Guard Check. State(“Ok to select mode”, “!=“, True) Action Update. State(“Ok to select mode”, True) Data Record. User. Data(“Mode of Travel Reselected”, “Reselecting the ‘mode of travel’ after making subsequent selections resets all selections. " Begin • “OK to select mode” Begin • “Not OK to select mode” Begin • “Mode reselected” Query/Air Query/Ocean Query/Motor Query/Rail Query/Any Query/Reset Query/Submit) Query/Air|* OR Query/Ocean|* OR Query/Motor|* OR Query/Rail|* OR Query/Any|* 45

UCI 46

UCI Other Slides

Interrelationships UCI Depends on Context Tension Evolution Depends on Selection Abstraction Depends on Reduction Transitively Depends on 48

Summary of Solutions UCI • Abstraction – select events of interest (triggers) – specify abstract events (actions) • Selection – select events of interest (triggers) – select contexts in which to capture events (triggers) – indicate when to capture state information (triggers) – select state information of interest (data spec) • Reduction – select among pre-defined data reduction algorithms — events, transitions, sequences, values, vectors, value per event — (data spec) 49

Summary of Solutions (cont’d) UCI • Context – check constraints on global and component state (guards) – update global state (actions) – capture UI, global, and user state (data spec) • Evolution – evolution without impacting application deployment or use (architecture) 50