Usi XMLEICS 2010 Berlin Germany June 20 2010

Usi. XML-EICS 2010 - Berlin, Germany - June 20, 2010 User Interfaces Composition with Usi. XML Sophie LEPREUX (1), Jean VANDERDONCKT (2), Christophe KOLSKI (1) LAMIH “Automated Reasoning and Human-Computer Interaction” Research Group University of Valenciennes and Hainaut-Cambrésis, France (2) BCHI University of Louvain-La-Neuve, Belgium

Introduction - To take the user caracteristics, the platform used, the environment into account, to allow an universal access to information, knowledge, services… in numerous different contexts Brain-Computer Interface ØNew UI design problems, focus on UI Composition

Summary - Tree-algebra based UI Composition - Usi. XML framework - Composition/decomposition Operators - Compose to support the 7 Dimensions - Conclusion

Tree-algebra based UI Composition tag <cui. Model id="Client_registration-cui_31" name="Client_registration-cui"> <window id="window_component_0" name="window_component_0" default. Content= « Nouveau client" width="500" height="350"> <box id="box_0" name="box_0" type="vertical"> <output. Text id="output_text_component_2" name="output_text_component_2" default. Content= « Fiche Client" is. Visible="true" is. Enabled="true" is. Bold="true" text. Color="#000000"/> <box id="box_1" name="box_1" type="horizontal"> <output. Text id="output_text_component_2" name="output_text_component_2 « default. Content= « Fiche Client" is. Visible="true" is. Enabled="true" is. Bold="true" text. Color="#000000"/> … </window> </cui. Model> content Window (id=window, name=window, width= « 500 » height= « 350 » ) Box (type= « vertical » ) Box Output (type = horizontal) (Default. Content = « Fiche Client » ) Output Input (…) Box (type = horizontal) Output (…) Input (…) Box (type = horizontal) Button (Default. Content=Close) Button (Default. Content = Save)

Summary - Tree-algebra based UI Composition - Usi. XML framework - Composition/decomposition Operators - Compose to support the 7 Dimensions - Conclusion

Usi. XML framework Source = < User, Environnement, Platform > Target = < User, Environnement, Platform > Task & Concepts Abstract UI Concrete UI Final UI

Summary - Tree-algebra based UI Composition - Usi. XML framework - Composition/decomposition Operators - Compose to support the 7 Dimensions - Conclusion

Composition/decomposition Operators Unary operators T 1 Selection( T 1, ) T 1 T 2 Binary operators Similarity Equivalence Complementary( T 1, ) Left difference Projection( T 1, ) Right difference Fusion Normal union Unique union Intersection ∩ Cut( T 1, )

Composition/decomposition Operators Example: Normal Union operator applied on two user interfaces U horizontal =

Composition/decomposition Operators Illustrations of operators on CUI models T 1 = tourist application UI (a) Selection (T 1, set of elements) T 2 = event management application UI (b) Projection (T 1, {output. Text, Button}) (c) Intersection (T 1, T 2)

Summary - Tree-algebra based UI Composition - Usi. XML framework - Composition/decomposition Operators - Compose to support the 7 Dimensions - Conclusion

Compose to support the 7 Dimensions - Use of (de)composition operators to adapt the UI according to the 7 dimensions: (1) Multi-device Usage and/or (2) Multi-User interface, (3) Multi-linguality, (4) Multi- organization, (5) Multi-context usage, (6) Multi-Modality usage, (7) Multi-platform Usage

Compose to support the 7 Dimensions (1) Multi-device usage - Illustration: control of a music player Adaptation 1: Source = < User, Environnement, Platform = 1 device > decomposition of the CUI Source in several CUIs Adaptation 2: Fusion of several decompositions of the CUI source Task & Concepts Abstract UI Concrete UI Final UI Target = < User, Environnement, Platform = n devices > Task & Concepts CUI’ 1 = Selection (set elements of Music controller task, Concrete UI) CUI’ 2 = Selection (set elements of Sounder player task, Concrete UI) CUI’ 3 = Selection (set elements of volume controller , Concrete UI) CUI’ 1 = Selection (set elements of Music controller task, Concrete UI) CUI’ 2 = Selection (set elements of Sounder player task, Concrete UI) Union Selection (set elements of volume controller , Concrete UI) CUI’ 3 = Selection (set elements of volume controller , Concrete UI) Abstract UI CUI’ 1 FUI’ 1 … CUI’n … FUI’n

Compose to support the 7 Dimensions (2) Multi-user interface - Two interesting cases: Case #1: a shared UI is co-localized, the users can simultaneously interact on one UI Ex. : tactile multi-touch interface or tangible objects used Case #2: a user arrives in a collective work situation (or leaves it): the user needs a work space, with personal data and/or functions - Other interesting problems to consider : Distant shared UI Consideration of the user rights or roles (ex: moderator UI in a brainwriting session at a distance: composed differently as the UI of the other participants)

Compose to support the 7 Dimensions (3) Multi-linguality - if two similar UI are modeled at the CUI level (or higher) with specifications given for one (or more) language and another UI modeled with others languages, it can be possible to compose these UI to obtain one UI modeled for all these languages - Importance to consider translation considerations leading to necessities about composition/decomposition due to socio-cultural aspects Ex: source language read from left to right, and downwards (english) target language read from right to left, and downwards (arabic), or upwards (for instance: Chinese) Consequence: different zones of the UI have to be composed/decomposed differently

Compose to support the 7 Dimensions (4) Multi-organization - Importance to consider how the tasks have to be performed in different organizations or groups Source = < User, T 2 & C T 1 & C Environnement, Platform > Target = < User, Environnement, Platform > T’ = T 1 U T 2 T’& C - Importance to consider the organization models described in the literature (ex. in social sciences or multi-agent system domain)

Compose to support the 7 Dimensions (5) Multi-context usage - Several composition/decompositions strategies are possible: Ex. : development methodology based on patterns, business component and machine learning (plastic UI approach, [Ph. D Hariri 08]) Ex: integration of the task model in the business component in order to compose complete application and to facilitate the co-evolution of the system [Bourguin G. , Lewandowski A. & Tarby J. C. , TAMODIA’ 2007] - A context modification can generate evolutions at each abstraction level Ex: a lighting change acts on the luminosity of the device (FUI) whereas the background noise change acts (or not) on the choice of the vocal modality. Ex: a modification of the user work brings modification of tasks and/or on the tasks planning Ø Interesting composition/decomposition problems to consider

Compose to support the 7 Dimensions (6) Multi-Modality usage - The AUI layer allows specifying the UI independently of the modality Ø According to needs brought by context change, a UI composition at a high level allows merging UI which will be concretized by different modalities in the lower levels - Ex : Food order realized with several modalities AUI 1 AUI’ =AUI 1 U AUI 2 AUI’ T’ = T 1 U T 2 Normal Union (AUI (order a pizza), AUI (Order Chinese food))

Compose to support the 7 Dimensions (7) Multi-platform Usage - Analogy: example of multi-platform language is: Java Ø Execution on several platforms without changing model or implemention - Our objective: to do the same from the UI point of view Ø to generate code from CUI model to FUI model Ø For example, graphical editor Graphi. XML allows generating java or XHTML code ü So the graphical UI which is modeled with Graphi. XML is multi-platform ü The composition operators: not needed into this process

Summary - Tree-algebra based UI Composition - Usi. XML framework - Composition/decomposition Operators - Compose to support the 7 Dimensions - Conclusion

Conclusion - Presentation of operators developed initially to compose at design time and at the CUI level - Currently: operators used to each abstraction level of UI - Operators: could be used at the runtime - These operators: a support to UI adaptation - Illustrations in the paper for the 7 dimensions: Multi-device Usage, Multi-User interface, Multi-linguality, Multi- organization, Multi-context usage, Multi-Modality usage, Multi-platform Usage Ø But: adaptations to criterion (dimension) can be done to the detriment of the others Ø As a result, the adaptation must be global i. e. the adaptation has to take into account all the criteria simultaneously and not considered one by one - Research questions are open concerning: Ø (1) the one by one or simultaneous consideration of these dimensions, Ø (2) the order in taking them into consideration …

Usi. XML-EICS 2010 - Berlin, Germany - June 20, 2010 User Interfaces Composition with Usi. XML Sophie LEPREUX (1), Jean VANDERDONCKT (2), Christophe KOLSKI (1) LAMIH “Automated Reasoning and Human-Computer Interaction” Research Group University of Valenciennes and Hainaut-Cambrésis, France (2) BCHI University of Louvain-La-Neuve, Belgium THANK YOU FOR YOUR ATTENTION Ajout logo 1 JVD Ajout logo 2 JVD