Bridging the gap between formal and informal learning

Bridging the gap between formal and informal learning: the HCI perspective Nelson Baloian

Research goals • Discover meaningful novel learning scenarios for which the “information loop” applies • Unified, coherent HCI across different platforms

Some “very old” Hystory • 1993 IPSI-GMD receives 2 of the first 8 electronic boards released outside XEROX. Palo Alto Labs. (Live. Board) • Norbert Streitz PUBLISH: DOLPHIN – Ambiente: the disappearing computer • Ulrich Hoppe COSOFT: Cosoft – The Coputer-integrated Classroom • 1995 Move to Duisburg, COLLIDE

Rich teacher-student interactions

The NIMMIS Classroom • Computer technology supporting reading/writing learning in the classroom • Using the “reading to writing” learniong methodology

Merging of different Media

Back in Chile: Teaching Java Using “off-the-shelf” Software Power point presentation

Text editor

Back in Chile: Teaching Java Using “off-the-shelf” Software System Console

Back in Chile: Teaching Java Using “off-the-shelf” Software

Results of a survey • “The teacher simply spends too much time trying to show us things”, “We get easily distracted in class” • I started to take some books with me and began to read them there. Finally I quit attending classes”, • “Why does it take so much time to move from one program to another? ”

The Ci. C Project Duisburg 2001

The Java Pallete in Freestyler

Using it in a real scenario

Lessons learned • HCI matters in learning scenarios • Minimize the interaction time • A lot of unexpected collaborative learning situations (as in Darmstadt) • Good for a particular situation -> the effort made by students to learn how to use it might not be really rewarding • “Set-up” time too long

Deep Board • • • Trying to Flexible creation, structuring and presentation of learning materials, Implementing “depth” to the interaction with interactive whiteboards. Student access through multiple tablets and keyboards. Web interface • Developed in Waseda by U. de Chile student Felipe Baytelman • Used for supporting video-based lectures in the GITS between Nishi-Waseda and Honjo

Lessons Learned • Keep it simple (HCI) – Simple structure – Simple HCI principles, easy to remember • Pages are not the only simple, easy to remember information structure • 3 D information graphs: not too complicated to manage • Still being used !! Video o Demo ?

Mobile Learning Scenario • 2005: • Idea: to use gesture-based interaction for mobile devices – Natural way to interact with a PDA – “Expand” the capabilities of the screen (deep board) • Other principles: – Keep rich face-to-face interaction while using computer technology – Dynamic group formation/reconfiguration

MCSketcher • Take a picture and start generating ideas on it collaborativeley • Interaction based exclusively on gestures, minimizing the number of widgets and the need of a virtual keyboard – maximizing the space available for content. • Content organized as a 3 D concept map • No switching between sketching and gesturing

Darkened Margin denotes user is in an innersub-node Highlighted Session menu shows the work needs to be saved Design spots show there are other sketching pages attached to this one, which can be explored by following the link shown as a yellow circle. In this case, there are 2 of them “Document three icon” The “group icon” shows that 2/3 of the users are in this node

A framework for developing mobile applications • Many scenarios sharing certain common characteristics have similar requirements • Gesture recognition module – Able to recognize some gestures – Can be extended to add more gestures • A flexible, lightweight communication platform for peer-to-peer applications

The Middleware • Classes implementing an API for developing distributed P 2 P applications easily. • Available in Java and C# • Convertion of objects from internal representations into an XML representation, transmit them and convert the XML representation into the corresponding internal one. • Provides services for discovering partners in the ad-hoc network and establishing connections among the different applications in order to synchronize shared data.

Discovering partners and establishing connection Functionality implemented in a single object which must be instantiated by the application in order to use the synchronizing functionalities TCP/IP connections to other applications for transmitting/ receiving objects TCP/IP connections manager Multicast UDP traffic for discovering and discarding partners Multicast discovery manager Active partners list Communication Node

Sharing objects Data sharing mechanism based on a “shared objects” principle. A shared object is an abstract class which should be extended in order to create an object class whose state will be transmitted to all active participants when the object changes its state, this is when one or more variables change their value. Application Communication node XMLSerializer C# World Application Communication node Castor Java World

Example 1: Supporting rich interaction in the classroom with mobile devices (WMUTE 2008)

Mobile technology in the classroom • Is it really necessary ? – Dinamic group reconfiguring during learning activity – Face-to-face interaction additional to shared workspaces – Personal gesture-based interaction allows for a flexible and swift exchange of information between teacher and learners

System Architecture View groups’ shared Working area Teacher Group 1 Send problems (re)configure groups Assesment Work together Group 3 Group 2

Group configuration Teacher’s module, in Group setup mode. Dragging user Nelson to group 2. Users are displayed automatically when discovered The groups are defined by the teacher. The icon will show allways What is actually on the groups’ shared workspaces

Problem creation writing the problem definition and the answers and delimiting the elements by closing them in rectangles For defining problems with Alternatives problem parts are Dragged into the respective areas

Synchronized work Screenshots of two students’ PDAs jointly working in the same group work solving different part of a problem

Sending the answers • Students must chose an answer by clicking on the “Share answer with group” button. (right) • After choosing an answer, the button turns into and agreement indicator (center). Here, the button shows the student’s answer coincides with one student but differs from two other. • Once all members agree on the answer, it turns into the “Submit answer” button (right),

Assesment The teacher’ view in the “show results” mode. Group 1 has answered 2 problems right (one open and the other with alternatives) Group 2 has answered 2 problems with alternatives, one wrong one right and the other with alternatives Group 3 answered 3 problems 2 with alternatives (right) one open (wrong). Cheking of solutions in open problems is done “by hand”

Example 2: A Platform for Motivating Collaborative Learning Using Participatory Simulation Applications (CSCWD’ 07 )

Participatory simulations • Role-playing activity oriented towards learning complex and dynamic systems – Mapping real world problems to simulated context and behaviors – Knowledge and patterns emerge from local interactions among users • Highly effective in large groups – Simple to set up and interact with – Simple decision process: Analyze information, exchange information, make decisions and see the outcomes – It allows to relate actions and their consequences • Highly motivating even in large groups – Participation and collaboration increase the understanding of the simulated reality and problem-solving abilities – Mobility has positive effects in engagement – Can be integrated in a whole classroom – learning by doing

Designing (creating) roles &Items Example: A Trust building rules learning scenario

Designing Items Example: Diseases, symptoms and treatments a b

Exchanging Items: Proximity+ Ir. DA

Exchanging Items (Example: shares)

Teacher support to oversee the activity

Collaborative Learning of Cryptographic Protocols (with Prof. W. Luther) • In situations where cryptographic protocols are necessary there are various actors: Alice, Bob, intruder, authentication authority… • Students assume a role and trigger the corresponding action at the right time • Petri-nets used to check the correctness • First version: Duisburg 2006, for normal PCs

PCs interface screenshot

Problems-solutions • Participants need to synchronize face-to-face • Much better with mobile devices !!! • Flexibility to get together to discuss the sequence of the operations, reconfigure groups, etc. . • Last version: Students have to (re)configure the interface (place buttons in a meaningful position) Journal of Future Generation Computer Systems (available online)

Example 3: Mobile Collaborative Knowledge Management System (CSCWD’ 08)

Design Principles of Mobile KM System • support different processes of KM at the moment that is needed while the users are on the move • use advanced information visualization techniques for displaying and navigating structures in complex information spaces • Support face-to-face communication by interfaces that use gestures and sketching • Organize knowledge using two-dimensional concept maps • Contextualize the situation by proximity: IRDA functionalities embedded in mobile devices may be used to detect close proximity between 2 users wishing to exchange information.

Description of the mobile KM System Creating nodes

Description of the mobile KM System Writing a label for a link

Description of the mobile KM System exchange of data by proximity with IRDA

Description of the mobile KM System The tree view

Description of the mobile KM System Contrasting coupled nodes. Green nodes belong to one user and blue ones to another. The coupled nodes are shown in the middle, one over the other. Doing a circle gesture over the coupled node will change the foreground-background order of the nodes view.

Formal evaluation of gestures » Which gesture is good on which platform ? » Diverse Platforms: PDA, smartphone, i-pad, Galaxy-tablett, HP-Tablet PC, E-board, and we keep adding new ones ! » What is Good: easy to learn, easy to remember, robust (it does what the user wants to)

Mode 1: certain sketches are recognized as commands Multiple selection Delete Undo/Redo Scroll Copy/Paste Mode 2: using a pregesture (could be double-click) Mode 3: Depend on starting absolute position

Towards unifying HCI principle across learning platforms » Fragmentation of learning experiences with different tools in different contexts create a need for integration of classroom activities (structured) & learning “in the wild” (unstructured) » Different interaction rules in each application » Briggs’ Technology Transition Model assumptions: “potential users will adapt technology if its benefits outweigh its disadvantages (notably, the cost of learning how to use it), adjusted by the frequency of use”.

Workflow: mobile → Individual→ Collaborative

Research Questions Is it possible to develop an integrated interaction model across platforms ? Widgets ? Gestures ? Both ? Which type of learning activities can be modelled in this way ? Can we describe them with a pattern language ?

Example 4: Learning with patterns (best paper CSCWD‘ 10)

Learning with patterns and/or Learning patterns ? Teacher explains a pattern: (for example, columns of neo-classical architecture) Students go „out“ to collect examples of instatiations of this pattern (congress palace, national library, this may also include geographical information) They process the gathered material in their homes They show their findings to the class, share.

Pattern Learning Mobile Application Creating Patterns: teacher creates a pattern during the lecture defining its components. Duplicating patterns: Students record their findings and create new patterns. create “clones” of the patterns Instantiating patterns: the students create instantiations of pattern with photographs and/or handmade sketches Linking patterns: defines a “visual pattern language” enriching the understanding of the pattern. Sharing patterns and their instantiations: a) Teacher can distribute an initial pattern b) Students work synchronously on the field creating pattern/instances c) Students exchange patterns/instances and clones Comparing patterns: Comparing patterns in order to highlight their differences and similarities is also an important part of the process of learning with patterns.

Creating a Pattern

Linking Patterns and Tree view

Defining parts of a Pattern

Comparing Patterns

The architecture

Example 5: Including Geocollaboration Georeferenced data plays an important role in completing the task, Some scenarios: a) Geology. students perform collaborative activities like field measurements and observations. Students must georeference their notes, take pictures and make recordings at concrete points; b) Architecture/design students may recognize types, styles of construction, and design patterns in certain areas the teacher has determined they must visit. Students may also receive instructions to collaboratively survey construction styles or design patterns in a certain zone using georeferenced notes to understand the changes in the development of constructions; c) Languages (Ogata). Using a map, foreign students having some basic knowledge of the local language receive instructions to go to different places in a city and perform different activities that reinforce that knowledge; d) Social sciences. Students of anthropology, psychology or sociology may have to make field observations for which collaborative created data and information notes of diverse nature (text, images, video and sound clips), associated with its localization will enrich their observations.

Task creation. The task consists of following a path

Pattern creation

Assigning tasks to the students (left) and the Student’s view of the task (right)

Lave, J. and E. Wenger, Situated Learning: Legitimate Periperal Participation, ed. C. U. Press. 1990 C 1 C 2 C 4 C 5 C 6 C 7 C 8 • Provide authentic contexts reflecting the way the knowledge is used in real life • Patterns instances are searched for in the very place they appear naturally • Provide authentic activities • Finding pattern instances in natural environments is a typical work experts often do. • Provide access to expert performances and the modeling of processes • After completing the field work, back in the classroom the teacher provides examples from the expert’s regarding the task. • Support collaborative construction of knowledge • Students work collaboratively on the field in order to collect the relevant data, share the data and • Promote reflection to enable abstractions to be formed • Students present their findings in front of the class reflecting about the patterns they found • Promote articulation to enable tacit knowledge to be made explicit • The system allows students to collect data, relate and communicate them formalizing their unsorted ideas about what they find • Provide coaching and scaffolding by the teacher at critical times • The teacher can help the students during the work on the field as well as back in the classroom

E-planing tool (best paper CRIWG 10)

Where do we go from now ? Even more diverse mobile and non-mobile devices appear every day on the market They differ in shapes, sizes and operative systems How to develop one plataform for all ? Is 100% P 2 P nowadays really necessary ? Internet is everywhere, and fast We are trying HTML 5 http: //saduewa. dcc. uchile. cl: 8090 http: //saduewa. dcc. uchile. cl: 8088 (start server)