2 nd Mobile Device Software Development and Web

2 nd Mobile Device Software Development and Web Development (MDSD) 2012" Model-based engineering of multi-platform, synchronous & collaborative UIs Extending Usi. XML for polymorphic user interface specification George Vellisa Dimitrios Kotsalisa Demosthenes Akoumianakisa Jean Vanderdoncktb a. Department of Applied Informatics & Multimedia, Technological Education Institution of Crete, Greece b. Université catholique de Louvain, Louvain Interaction Laboratory (www. lilab. eu) Louvain School of Management, Belgium Piraeus, Athens, Greece Oct. 5 -7, 2012

Introduction • User interface challenges for software engineers – Programming intensive – Broad range of interaction components and widgets – Platform specificity (GUI objects, Web widgets, etc) – Lack of interoperability due to platform specificity • (Some) Model-based engineering claims – UI engineering = Model specification – Focus on design rather than implementation – Linking (rather than direct calls) to platform-specific libraries

Introduction • User interface challenges for software engineers – UI development time takes a large portion of time System feasibility Validation Software plans and requirements Validation UI Design occupied 45% of the total amount of time devoted to each activity Product design Verification Mean = 44% Detailed design Verification Code 50% Unit test Integration Product verification Implementation System test Operations and maintenance Revalidation 37%

Retrospective assessment • Claims proven useful – Model specifications can provide an engineering basis for UI engineering – Run-time linking alleviates the complexity of programming • Claims proven possible – Multi-platform execution – Relatively straightforward Uis (e. g. , form-based UI)

Retrospective assessment (Cont. ) • Several initiatives – One of them is Usi. XML which is used in this work • Reference Architectures – CAMELEON Reference framework www. usixml. eu www. usixml. org • Recommended by W 3 C Working Group on Model-based User Interface Design • Usi. XML is compliant with this framework • Plethora of methods with some focus on collaboration – Flowi. XML, AMENITES, CIAM, TOUCHE

Retrospective assessment (cont. ) • Challenges still pending attention – non-standard and customized widgets? – synchronous and collaborative UIs? – new affordances such as social awareness in distributed collaborative settings? – radically different interaction vocabularies? – design and run-time support tools for all the above?

Research design • Theory development – Build abstraction to capture interaction components as assemblages of capabilities (or affordances) – Substantiate such capabilities (or affordances) to polymporphic instantiation schemes – Specify polymorphic properties in a model-based fashion • Engineering framework – Extend Usi. XML as needed to support the new specification – Provide dedicated design tools and run-time components – Demonstrate the concept’s validity through pilot scenarios

Why abstraction? • Consider various generations of ‘buttons’ – GUI buttons (in the 90 s) • The (primitive) assemblage of button = {two-state device to exercise control over functionality}

Why abstraction? • A continuum of abstractions (Cameleon Ref. Framework)

‘Adding’ social scent • Scented button (cf. Willett et al. , 2007) • The (augmented) assemblage of ‘scented’ button = { Two-state device to exercise control } + { Collectivity (i. e. , who else is in the same state of mind)}

‘Adding’ expression of intention • ‘Like’ buttons (in the Web 2. 0) • The (new) assemblage of the ‘like’ button = {Two-state devices to exercise control} + {Expression of opinion (i. e. I like this)}

Putting them all together • ‘Like’ buttons with social scent • The (combined) assemblage = {Two-state device to exercise control} + {Expression of opinion (i. e. I like this)} + {Collectivity (i. e. , who else is in the same state of mind)}

Generalizing the concept • Why limit our creativity to conventional widgets? – elastic buttons and their interactive behaviour Note the embedded affordances

Adding to complexity • What happens when an elastic button is to be instantiated across devices and platforms

Motivating question • How can we … – model digital assemblages in interaction components? – instantiate them as the need arises and depending on context / platform capabilities – integrate them into specifications as first class elements / properties – compile specs that are general enough

(One) Answer: Polymorphic instantiation • A scheme that relies on implementation agnostic (i. e. , abstract) specifications of UIs – i. e. an abstract button can be instantiated as • a conventional GUI button when implemented with the ‘primitive’ assemblage • a scented button when implemented with the ‘augmented’ assemblage • a ‘like’ button when implemented with the ‘combined’ assemblage • At run-time and once user and usage context parameters are discovered, the implementation agnostic spec is translated to context-specific interaction vocabularies using dedicated tools

Polymorphic Widget Specifications Basic concept • Introduce new widgets and their properties as firstclass design elements WSL Libraries Resources Widget Archive

The role of MBUI engineering • Specify complex capabilities & affordances in a model -based fashion – – Devise required properties Define model elements Create specification Mapping models

The role of MBUI engineering the case of a socially aware button

A new set of models • Extend Usi. XML as needed to support the new specification – New set of models • Widget Resource Model • Behavior model – Additional models for collaborative work • • Squad model Consistency model Abstraction model Session model

A new design tool

A new run-time environment UI-1 Design Environment deploy. UI 6. attach. States 5. build. UI Platform Server 4. get. Archives 1. init. PS UI Mo Collaboration Plugin Abstraction Classes Polling track. Changes Client-1 Widget Spec Tool Runtime Environment P. S WSL Resources Dependencies et apply. Changes distribute. Changes UI Design Tool Widget Archives Repository 3. g Social Proxy UI-N de Collaboration Plugin l UI-Model Repository 2. join Notification Queue Session Manager Web-Services Layer Axis 2 -Framework Server Side Framework (SSF) Social Proxy Abstraction Classes Polling Client-N

UIDL’ 2011 (simple) desktop scenario

MDSD’ 12 multiplatform scenario (in the paper) Player-1: Desktop/PC Player-2: Android/mobile Viewer-1: Desktop/PC

Post MDSD’ 12 scenario • Collaborative assembly of vacation packages • User roles – – Business partner (desktop) Administrator (desktop or mobile Platforms (desktop/swing, mobile/Android) Context (synchronous and collaborative) • Complex and custom widgets (i. e. , elastic buttons) • Synchronous collaboration

The desktop case administrator’s view of vacation package / swing

The desktop case business partners’ view of vacation package / swing

The mobile case administrator’s view of vacation package - Android Due to space constraints certain functions become automatically adapted to vocal interaction

The designer’s role? Designing for certain affordances 1 2 3 4 5 6

The run-time instantiation the specified affordances enabled 1 2 3 4 5 6 2

Future work & acknowledgements • Current and on-going work – Advanced toolkits (i. e. , visualization) – Run-time adaptivity and UI plasticity in distributed and ubiquitous settings – ‘Social’ games • Acknowledgements – The work is supported by ARCHIMEDES III – Thanks to Université catholique de Louvain (UCL) for supporting the first two authors’ Ph. D dissertations • Due to be submitted in 2013