MultiPath Development of User Interfaces Quentin Limbourg LouvainlaNeuve

Multi-Path Development of User Interfaces Quentin Limbourg Louvain-la-Neuve, 4 th November 2004 July, 2003 (Oviedo, Spain) 1

Presentation Plan § § Context Problem Statement Thesis Proposal – Ontological proposal – Methodological proposal § Case Studies and tools § Conclusion & Perspectives Multi-Path Development of User Interfaces 2

Context Task and Stage A Domain Abstract User. Interface Stage B Concrete User. Interface Stage C § Transformational-driven Development of User Interfaces – Endorses the motivation of SE for HCI – User-Centred Design of UIs – Deriving a UI is heuristic and is not a straight line User Interface Code Stage D Multi-Path Development of User Interfaces 3

Context (cont’d) § UIs are running fast…after change – – Task redefinitions or reallocation Organizational adaptation Domain evolution Obsolescence of languages and New Computing languages – New platforms § These changes give rise to the ellicitation development scenario Multi-Path Development of User Interfaces 4

Context cont’d § These development scenarios are relevant to several problem areas : – Computer-assisted derivation of UIs – UI maintenance and refactoring – Multi-context UI development – Reverse engineering of UIs Multi-Path Development of User Interfaces 5

Problem Statement § Ontological shortcomings – Explicitness, rigorousness, extendibility, commitment, communication § Methodological shortcomings – Expliciteness, rigorousness, consistency in application, predictability, flexibility, modifiability, communication. Multi-Path Development of User Interfaces 6

Thesis Statement § It is possible to address the previously outlined shortcommings by: – A definition of an explicit and rigorous ontological framework for defining concepts involved in UI development – A definition of an explicit and rigorous methodological framework that is transformation driven and multi-path Multi-Path Development of User Interfaces 7

Multi-Path Development of UI Development Scenario * * Development path 1 * Development step Forward engineering Any Relevant Reverse Adaptation Composition Engineering § Multi-path development qualifies a methodology that support the realization of multiples types of development paths withing a single framework Multi-Path Development of User Interfaces 8

Proposal § To support this approaches in a single framework we need: – An ontological framework that provide a description of concepts for each development stage – A methodological framework that allows a definition, organization, and execution of development paths Multi-Path Development of User Interfaces 9

Ontological Framework § An ontology is a formal specification of a conceptualisation Abstract Concepts Model description with UML class diagrams Semantic Mapping Abstract Syntax Identified, Labeled, Typed, Constrained – Graphs Abstract to Concrete Mapping Concrete Syntax AGG Visual Notation + usi. XML Multi-Path Development of User Interfaces 10

Ontology: Abstract Concepts § Structured in viewpoints and in models – Task (CTT + some improvements) – Domain (UML Class/Object diagram + some improvements) – Abstract User Interface (vocabulary independent of the modality) – Concrete User Interface (vocabulary independent of the toolkit) – Context of use (<User, Platform, Environment>) – Inter-model relationship mappings (traceability, integration of all views) Multi-Path Development of User Interfaces 11

Ontology : Syntax § Abstract syntax – – Directed, labeled, attributed, typed graphs. Nodes are concepts Edges are relationships between these concepts Result: a UI specification is a BIG WHOLE graph § Concrete syntax : USIXML and AGG visual notation – USer Interface e. Xtensible Mark-Up Language – (graph structure is achieved by defining explicitly relationships) – AGG Visual Notation Multi-Path Development of User Interfaces 12

Methodological Framework { Taskand Domain Reification | u Abstraction z Abstract User. Interface Translation Code Generation v } y Concrete User. Interface w User. Interface Final Interface Code User. Code Interface Multi-Path Development of User Interfaces Code Reverse Engineering x 13

Model-to-Model Transformation § In the literature: – – Opportunistic algorithms APIs XSLT … § Drawbacks – – – – Execution semantics is poorly defined Consistency in applying steps Consistency of resultant model Procedurality Cryptic syntax Low predicatbility Hidden rationale (entails low re-use) Multi-Path Development of User Interfaces 14

Conditional Graph Rewritting § § § § Generalization of string grammars Grounded execution semantics (pushout construction) Side-effect free Attractive syntax Declarativeness Determinism (application strategy) Seamlessness with ontological world (rules manipulate patterns of specification) § Tools § Extension (PAC, NAC) Multi-Path Development of User Interfaces 15

How it works… § Find an occurrence of LHS in G (this occurrence is called a match). If several occurrences exist, choose one nondeterministically. § Check preconditions of both type PAC and NAC. If not verified, then skip. § Remove the part of G which corresponds to (LHS – K), where K is the morphism specified between LHS and RHS. § Add RHS – K into G – (LHS – K) as it is given by the corresponding relation between RHS – K and K. § Check postconditions of both type PAC (and notably that the resulting graph is properly typed) and NAC. If not verified, then undo the transformation rule. Multi-Path Development of User Interfaces 16

Methodological World and Transformation World A development library stores (in usi. XML textual format) paths, steps and sub-steps definition and their associated transformation systems and transformation rules Multi-Path Development of User Interfaces 17

Substeps: From AUI to CUI § Definition of AUI structure – Which objects should be logically grouped ? § Definition of AIC types – Which « functionnality » should assume AICs and what data do they manipulate ? § Definition of spatio-temporal arrangement – How should AIC be arranged in space and/or in time ? § Definition of dialog control – What is the valid flow of action on AIOs ? Multi-Path Development of User Interfaces 18

Substeps: From AUI to CUI § Define CUI structure – Which AIC is a window ? § Define CIC type ? – Which « widget » should represent which AIC ? § Define placement ? – What layout can be specified between CICs, CCs… § Define navigation ? – Which container can be started or closed from which container ? § Define dialog control ? – What is the valid flow of action on AIOs Multi-Path Development of User Interfaces 19

Typed Model-to-Model Transformation Meta-Model Graph Structure is instance of Meta-Model Our Meta-Model Subset 1 Meta-Model Subset 2 e. g. , Task+Domain Model e. g. , Concrete UI Model Uses language is instance of Initial UI Model e. g. , My. Task. And. Domain. Model is instance of Transformation Rule Our transformation catalog Multi-Path Development of User Interfaces Resultant UI Model e. g. , My. Concrete. UIModel 20

Example : Widget Replacement (1) <? xml version="1. 0" encoding="UTF-8" standalone="yes"? > <cui. Model name="My. Model"> <version modif. Date="2004 -03 -24 T 17: 09: 17. 402+01: 00" xmlns="">7</version> <author. Name xmlns="">Youri</author. Name> <window height="500" width="600" name="Formulaire (2/5)" id="window_1"> <box relative. Height="100" name="box 1_0" id="box 1_0"> <box type="vert" name="box. Todo" id="box. Todo"> . . . <box type="horiz" name="box_2_2_2_1" id="box_2_2_2_1"> <text. Component default. Content="Sexe" is. Bold="true" id="label_2"/> <radio. Button group. Name=“grupo 01" default. Content="Femme" default. State="false" id="radiobutton_0"/> <radio. Button group. Name="grupo 01" default. Content="Homme" default. State="true" id="radiobutton_1"/> </box> . . . </box> </window> </cui. Model> Excerpt for an usi. XML CUI specification. Multi-Path Development of User Interfaces 21

Example : Widget Replacement (2) Multi-Path Development of User Interfaces 22

Example : Widget Replacement (3) The usi. XML graph before aplying any rule Multi-Path Development of User Interfaces 23

Example : Widget Replacement (4) Rule 1: Create a new combo. Box with the same id and name as the name of the group of radio. Buttons. NAC LHS Multi-Path Development of User Interfaces RHS 24

Example : Widget Replacement (5) Rule 1: Create a new combo. Box with the same id and name as the name of the group of radio. Buttons. The usi. XML graph after aplying the first rule Multi-Path Development of User Interfaces 25

Example : Widget Replacement (6) Rule 2: Convert every radio. Button within the group “x” into an item for the combo. Box “x”, we have just created. LHS RHS : : = Multi-Path Development of User Interfaces 26

Example : Widget Replacement (7) Rule 2: Convert every radio. Button within the group “x” into an item for the combo. Box “x”, we have just created. The usi. XML graph after aplying the second rule Multi-Path Development of User Interfaces 27

Example : Widget Replacement (8) <? xml version="1. 0" encoding="UTF-8" standalone="yes"? > <cui. Model name="My. Model"> <version modif. Date="2004 -03 -24 T 17: 09: 17. 402+01: 00" xmlns="">7</version> <author. Name xmlns="">Youri</author. Name> <window height="500" width="600" name="Formulaire (2/5)" id="window_1"> <box relative. Height="100" name="box 1_0" id="box 1_0"> <box type="vert" name="box. Todo" id="box. Todo"> . . . <box type="horiz" name="box_2_2_2_1" id="box_2_2_2_1"> <text. Component default. Content="Sexe" is. Bold="true" id="label_2"/> <combo. Box id="combo. Box 001" name="label_3" is. Drop. Down="true"> <item id="radiobutton_0" name="radiobutton_0" default. Content="Femme"/> <item id="radiobutton_1" name="radiobutton_1" default. Content="Homme"/> </combo. Box> . . . </box> </window> </cui. Model> Excerpt from the final transformated usi. XML specification Multi-Path Development of User Interfaces 28

Example : Widget Replacement (10) Multi-Path Development of User Interfaces 29

Case Studies § 2 case studies were considered § Modus Operandi Multi-Path Development of User Interfaces 30

Transformi. XML API/GUI § API = set of transformation functions § GUI (Video) Multi-Path Development of User Interfaces 31

Tool Support § Prototypes – Transformi. XML API/GUI : transformation tool (previously shown) – Ideal. XML: Task, Domain and AUI editors – Grafi. XML : CUI Hi-Fi + Code Generator (Java Swing, XHTML) – Visi. XML : CUI Lo-Fi, MS Visio Plug-in – Flashi. XML : flash renderer – Reversi. XML : reverse engineering from HTML to CUI Multi-Path Development of User Interfaces 32

Tools : Ideal. XML Demo Multi-Path Development of User Interfaces 33

Tools : Grafi. XML Multi-Path Development of User Interfaces 34

Tools : Visi. XML Multi-Path Development of User Interfaces 35

Tools : Flashi. XML Demo Multi-Path Development of User Interfaces 36

Conclusion: Validation § External: Feasiblity of the approach has been exposed by the realization of two case studies and the provision of transformation catalogs § Internal: – Ontological requirements: • Expliciteness, Expressivity, Human readability, Formality, Machine readability, Separation of concerns, Verifiability, Homogeneity, Reuse, Extendibility, Standards – Methodological requirements: • Expliciteness, Flexibility, Formality, Executability, Separation of concerns, Extendibility Traceability, Correctness, Tool interoperability, Reuse. Multi-Path Development of User Interfaces 37

Conclusion: Main Contributions § Ontological contribution – Definition of a formal language for describing viewpoints on a UI system § Methodological contribution – Explicit, open and formal definition of development steps and paths – Unification of several problem area in a unique framework thanks to the paradigm of multi-path development of user interface – Communication and incremental research and development effort Multi-Path Development of User Interfaces 38

Conclusion: Secondary contributions § For SE community: – Application of transformational development concepts to UIs. § Graph Transformation Community: – Application of theoretical concepts to a new problem domain Multi-Path Development of User Interfaces 39

Obstacles and Shortcomings § Main obtacles and shortcomings : – Ontological • Lack of expressivity of models – Methodological • • • Complexity of certain sub-steps Difficulty of finding an approprite level of generalization Difficulty in ordering rules and steps Lack of disjunction in the rule expression Difficulty to get meta-information Difficulty in managing large sets of transformation catalogs Multi-Path Development of User Interfaces 40

Future Works § § § § § Extension to other types of UIs Extension to other types of models Define high-level building blocks (“constructive” patterns) Extension of methodological steps definition and transformation catalogs Extend formal foundations Validate catalogs by human factors experts Hide complexity Evaluate transformation for run-time scenarios Continue the development of ongoing tools … Multi-Path Development of User Interfaces 41

Thank You ! Multi-Path Development of User Interfaces 42