Opening the Black Box of Interaction in Visualization
Opening the Black Box of Interaction in Visualization Hans-Jörg Schulz 1, Tatiana v. Landesberger 2, Dominikus Baur 3 VIS Tutorial 2014 D 1. 2. 3. Fraunhofer IGD, Rostock, Germany TU Darmstadt, Germany Dominikus Baur Interfacery . MINIK. US
PART I: INTERACTION ACTIVITIES Speaker: Tatiana von Landesberger VIS Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 2
Part 1: Interaction activities Activities: What the user does to trigger a change in the computer (Action) Metaphor: What the user thinks the computer is doing and vice versa (Understanding) Architecture: What the computer actually does (Reaction) Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 3
Part 1: Interaction activities Activities: What the user does to trigger a change in the computer (Action) Metaphor: What the user thinks the computer is doing and vice versa (Understanding) Architecture: What the computer actually does (Reaction) Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 4
Overview of Part 1 • 6 W’s of User’s Interaction and interaction loop • Systematization of interaction • Human (Ws) and System (Vis) • Vis/VA-focused systematizations: 1. Visualization, 2. Visual Data Mining 3. Reasoning • Third view: Interaction Support Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 5
Motivation • System developers: • What to include in my system • Researchers: • What is there and what is missing • Users: • What to expect from the system • Developers, researchers, …: • Canonicum for evaluation and system testing Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 6
Make it easier for you… Motivation Systematization of perspectives … Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 7
USER’S ACTION AND THE INTERACTION LOOP Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur http: //www. smartgirl. org/brain-food/career-hubs/human-computer-interaction. png What is interaction from user’s point of view? 8
6 Ws of Interaction WHY do we interact? What is the goal? WHAT is the purpose? What is the intended effect of interaction? Effects and means HOW do we interact? Which means do we use/have at disposal? WHO interacts? Who are the users interacting? What is their background? WHEN do we interact? Context of interaction When is interaction needed? WHERE is interaction used? Where users interact? Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur [adapted & merged Roth 13, Jansen et al 13] 9
6 Ws of Interaction WHY do we interact? What is the goal? WHAT is the purpose? What is the intended effect of interaction? Effects and means HOW do we interact? Which means do we use/have at disposal? WHO interacts? Who are the users interacting? What is their background? WHEN do we interact? Context of interaction When is interaction needed? WHERE is interaction used? Where users interact? Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur [adapted & merged Roth 13, Jansen et al 13] 10
6 Ws of Interaction WHY do we interact? What is the goal? WHAT is the purpose? What is the intended effect of interaction? Effects and means HOW do we interact? Which means do we use/have at disposal? WHO interacts? Who are the users interacting? What is their background? WHEN do we interact? Context of interaction When is interaction needed? WHERE is interaction used? Where users interact? Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur [adapted & merged Roth 13, Jansen et al 13] 11
6 Ws of Interaction WHY do we interact? What is the goal? WHAT is the purpose? Effects and means What is the intended effect of interaction? HOW do we interact? Which means do we use/have at disposal? WHO interacts? Who are the users interacting? What is their background? WHEN do we interact? Context of interaction When is interaction needed? WHERE is interaction used? Where users interact? [adapted & merged from Roth 13, Jansen et al 13] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 12
Hierarchic View on Interaction WHY do we interact? WHAT is the purpose? HOW do we interact? Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 13
Norman’s Model of Interaction 8. Take further action (compare outcome with goal) EXECUTION 1. Establish a goal (Why? ) Execution/ 2. Form intention/identify task Evaluation loop (What? ) 3. Specify action sequence (How? ) 4. Execute action EVALUATION 7. Evaluate the outcome (Why? ) 6. Interpret the system’s state (What? ) 5. Perceive the state of the system (How? ) [Norman 88] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 14
Preliminary Summary 1. 6 Ws of interaction: 1. Effects+means: Why? , What? How? 2. Context: Who? Where? When? 2. Hierarchic nature of interaction 3. Execution/Evaluation loop Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 15
Which interactions exist? INTERACTION SYSTEMATIZATION Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 16
TWO VIEWS ON INTERACTION Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 17
What is interaction Systematization 1. 6 Ws of interaction: 1. Effects+means: Why? , What? How? 2. Context: Who? Where? When? 2. Hierarchic nature of interaction 3. Execution/Evaluation loop Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 18
What is interaction Systematization 1. 6 Ws of interaction: 1. Effects+means: Why? , What? How? 2. Context: Who? Where? When? 2. Hierarchic nature of interaction 3. Execution/Perception loop Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur User-centric view 19
What is interaction Systematization 1. 6 Ws of interaction: 1. Effects+means: Why? , What? How? 2. Context: Who? Where? When? 2. Hierarchic nature of interaction 3. Execution/Perception loop User-centric view Visualization side? Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 20
Human ↔ Visualization WHY WHAT HOW Subjective perception Visualization changes What should be modified in the view (goal/intention) What in the visualization is modified How it should be modified (which action) How this is done (software/hardware) [adjusted Roth 13] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 21
Human ↔ Visualization WHY WHAT [Card et al. 99] HOW Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 22
Human ↔ Visual Analytics WHY View manipulation Visualization Mapping Transformation Data WHAT Model buildin g Model visualizatio n Models Knowledge Parameter refinement Information mining HOW Feedback loop Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur [Keim et al. 2008] 23
Information Visualization Model [Card et al. 99] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 24
Information Visualization Model Data Visual [Card et al. 99] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 25
Visualization Visual Analytics Simple Information Visualization Model Visualization View Manipulation Mapping Transformation Data Knowledge [Keim et al. 2008] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 26
Visualization Visual Analytics Simple Data Mining Model Transformation Data Knowledge Data Mining/ Models Parameter refinement Information mining Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur [Keim et al. 2008] 27
Visual Analytics Model View manipulation Visualization Mapping Transformation Data Model building Model visualization Models Knowledge Parameter refinement Information mining Feedback loop Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur [Keim et al. 2008] 28
3 Ways of Visual Analytics Way 1: Info. Vis View manipulation Visualization Mapping Transformation Data Model building Model visualization Models Knowledge Parameter refinement Information mining Feedback loop Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 29
3 Ways of Visual Analytics Way 2: Visual Data Mining View manipulation Visualization Mapping Transformation Data Model building Model visualization Models Knowledge Parameter refinement Information mining Feedback loop [Keim et al. 2008] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 30
3 Ways of Visual Analytics Way 3: Provenance/Sensemaking/Reasoning View manipulation Visualization Mapping Transformation Data Model building Model visualization Models Knowledge Parameter refinement Information mining Feedback loop [Keim et al. 2008] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 31
Interaction Need in Visual Analytics Support all 3 ways via visual means View manipulation Visualization Mapping Transformation Data Model building Model visualization Models Knowledge Parameter refinement Information mining Feedback loop [Keim et al. 2008] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 32
Systematization of Interaction 2. Visual Data Mining WHY [Bertini & Lalanne 09] [Shneiderman 96] […] 1. Information Visualization [Keim 02] WHAT [Pike et al 09] [Parsons & Sedig 14] [Dix & Ellis 98] [Spence 07] [Yi et al. 07] [Roth 13] [Wybrow et al 14] [Zhou & Fesner 98] [Jansen, Dragicevic 13] […] 3. Reasoning [Heer & Shneiderman 12] [Gotz & Zhou 08] HOW Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur [Kerren & Schreiber 12] […] 33
Systematization of Interaction 2. Visual Data Mining WHY [Bertini & Lalanne 09] [Shneiderman 96] […] 1. Information Visualization [Keim 02] WHAT [Pike et al 09] [Parsons & Sedig 14] [Dix & Ellis 98] [Spence 07] [Yi et al. 07] [Roth 13] [Wybrow et al 14] [Zhou & Fesner 98] [Jansen, Dragicevic 13] […] 3. Reasoning [Heer & Shneiderman 12] [Gotz & Zhou 08] HOW Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur [Kerren & Schreiber 12] […] 34
Levels of Systematization: Example WHY [Yi et al 07] WHAT ? Select, Explore, Reconfigure, Encode, Abstract/Elaborate, Filter, Connect HOW Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 35
Levels of Systematization: Problem of ambiguous terms WHY [Yi et al 07] WHAT ? Select, Explore, Reconfigure, Encode, Abstract/Elaborate, Filter, Connect What/where in the pipeline? HOW Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 36
Pipeline-Focused Interaction Systematization 2. Visual Data Mining [Shneiderman 96] 1. Information Visualization [Keim 02] [Pike et al 09] [Roth 13] [Parsons & Sedig 14] [Dix & Ellis 98] [Spence 07] [Yi et al. 07] [Bertini & Lalanne 09] […] 3. Reasoning [Wybrow et al 14] [Zhou & Fesner 98] [Heer & Shneiderman 12] [Jansen, Dragicevic 13] [Gotz & Zhou 08] […] [Kerren & Schreiber 12] […] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 37
Pipeline-Focused Interaction Systematization 2. Visual Data Mining [Shneiderman 96] 1. Information Visualization [Keim 02] [Pike et al 09] [Roth 13] [Parsons & Sedig 14] [Dix & Ellis 98] [Spence 07] [Yi et al. 07] [Bertini & Lalanne 09] ? […] 3. Reasoning [Wybrow et al 14] [Zhou & Fesner 98] [Heer & Shneiderman 12] [Jansen, Dragicevic 13] [Gotz & Zhou 08] […] [Kerren & Schreiber 12] […] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 38
Unified VA Interaction Systematization 1. Visualization 2. Data mining 3. Reasoning Data changes • Selection • Editing • Analytic process tracking • Editing (annotation) Visualization changes Data mining changes • Scheme: type and mapping • Parameters • Scheme: data processing type • Parameters Reasoning changes • Scheme: change analysis type • Parameters [von Landesberger et al. 2014] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 39
View manipulation Visualization Mapping Transformation Data Model building Model visualization Models Knowledge Parameter refinement Information mining Feedback loop 3 Ways of Visual Analytics: Info. Vis (Way 1) INFOVIS – FOCUSED SYSTEMATIZATIONS Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 40
1. Visualization Systematizations [Shneiderman 96] 1. Information Visualization [Keim 02] [Pike et al 09] [Roth 13] [Parsons & Sedig 14] [Dix & Ellis 98] [Spence 07] [Yi et al. 07] [Wybrow et al 14] [Jansen, Dragicevic 13] [Zhou & Fesner 98] Infovis pipeline based systematization […] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 41
1. Visualization Systematization Text #Words #Sentence 50 9 100 20 80 7 #Words Size #Senten ces Color [Card et al 99] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur
Info. Vis Interaction: View Transformation Text #Words #Sentence 50 9 100 20 80 7 #Words Size #Senten ces Color [Card et al 99] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur
Info. Vis Interaction: View Transformation • Navigation • Pan, zoom, scroll, . . . • Highlighting Hover Select+highlight Navigation in visible space Brushing and linking Magic lenses View reconfiguration (Re-)arrange multiple views on the screen Open/close new views Source: maps. google. com Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 44
Info. Vis Interaction: View Transformation NAVIGATE: Paris Sydney • Navigation • Pan, zoom, scroll, . . . • Highlighting Hover Select+highlight Navigation in visible space Brushing and linking Magic lenses View reconfiguration (Re-)arrange multiple views on the screen Open/close new views Source: maps. google. com Problem: Cumbersome, time consuming, “lost in space” Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 45
Info. Vis Interaction: View Transformation Navigation Video 01: Topology-Aware Navigation in Large Networks [Moskovich et al. 09] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 46
Info. Vis Interaction: View Transformation • Navigation Pan, zoom, scroll, . . . • • Highlighting • • Hover Select+highlight Brushing and linking Magic lenses • View reconfiguration • • (Re-)arrange multiple views on the screen • Highlighting important information Open/close new s Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 47
Info. Vis Interaction: View Transformation • Navigation Pan, zoom, scroll, . . . • • Highlighting • • Hover Select+highlight Brushing and linking Magic lenses • View reconfiguration • • (Re-)arrange multiple views on the screen Open/close new views • Configuring multiple views Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 48
Info. Vis Interaction: Visual Mapping Text #Words #Sentence 50 9 100 20 80 7 #Words Size #Senten ces Color [Card et al 99] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 49
Info. Vis Interaction: Visual Mapping • Visualization type • Type of visualization • Scatterplot/matrix • Node-link/matrix • Type of mapping • E. g. color/size/form Video 10 [van den Elzen & van Wijk 13] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 50
Info. Vis Interaction: Visual Mapping • Visualization type • Type of visualization • Type of mapping • Mapping parameter • Data mapping • E. g. color scheme Video 10 [van den Elzen & van Wijk 13] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 51
Info. Vis Interaction: Visual Mapping • Visualization type • Type of visualization • Type of mapping • Mapping parameter • Data mapping • E. g. color scheme Video 10 [van den Elzen & van Wijk 13] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 52
Info. Vis Interaction: Visual Mapping • Visualization type • Type of visualization • Type of mapping • Mapping parameter • Data to be mapped • E. g. color scheme • Further specifics • E. g. type of layout, sorting Video 10 [van den Elzen & van Wijk 13] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 53
Info. Vis Interaction: Data Manipulation Text #Words #Sentence 50 9 100 20 80 7 #Words Size #Senten ces Color [Card et al 99] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 54
Info. Vis Interaction: Data Manipulation • Data navigation • drill down, expand, filter, … • Data transformation • Change data values by calculation • Normalization, aggregation, … • Data editing • Change data values by editing • Create data Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 55
Info. Vis Interaction: Data Manipulation • Data navigation • drill down, expand, filter, … • • Data transformation Top down • • Change data on values by calculation Filter, details demand • • Normalization, aggregation, … Bottom up • • Data editing Expand on demand • • Change data values by editing Middle out • • Create Start indata the middle [von Landesberger et al 11] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 56
Info. Vis Interaction: Data Manipulation • Data navigation • Search, Show Context, Expand on Demand drill down, expand, filter, … • • Data transformation Top down • • Change data on values by calculation Filter, details demand • • Normalization, aggregation, … Bottom up • • Data editing Expand on demand • • Change data values by editing Middle out • • Create Start indata the middle [van Ham & Perer 09] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 57
Info. Vis Interaction: Data Manipulation Video 02: data search and expand Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 58
Info. Vis Interaction: Data Manipulation • Data navigation • drill down, expand, filter, … • • • Normalization (lin, log, exp, . . ) Aggregation (manual, according to data, …) … • Data editing • • Change data values by editing Create data Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur Source: Gapminder. org • Data transformation 59
Info. Vis Interaction: Data Manipulation • Data navigation drill down, expand, filter, … • • Data transformation • Normalization (e. g. lin, log) • Aggregation • • Manual • According to data attributes • According to data structure (e. g. communities) Etc. Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 60
Info. Vis Interaction: Data Manipulation Data navigation • • drill down, expand, filter, … Data transformation • • Normalization (lin, log, exp, . . ) • Aggregation (manual, according to data, …) Data editing • • Change values • Create data • Individual values • Whole datasets Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 61
Info. Vis Interaction: Data editing Video 03: edit Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 62
PC/DC On the Highway to Data [Bremm et al 2012] Info. Vis Interaction: Data creation Video 04: create [Bremm et al 2012] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 63
Summary: Info. Vis Interaction Text #Words #Sentence 50 9 100 20 80 7 #Words Size #Senten ces Color
Visualization View manipulation Mapping Transformation Data Model building Model visualization Models Information mining Knowledge Parameter refinement Way 2: Visual data mining VISUAL DATA MINING INTERACTION Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 65
VA Interaction Systematization 2. Visual Data Mining [Bertini & Lalanne 09] 1. Information Visualization […] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 66
Visual Data Mining • Computationally enhanced Visualization (V++) • Visually enhanced Mining (M++) • Integrated Visualization and Mining (VM) [Bertini & Lalanne 09] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 67
Visual Data Mining Interaction • Manipulating and tuning: Vis: changing representation parameters DM: changing model parameters • Changing the scheme: Vis: changing the visual mapping or visual representation DM: changing the data model [Bertini & Lalanne 09] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 68
Visual Data Mining Interaction • Manipulating and tuning: Vis: changing representation parameters For example: changing color scheme DM: changing model parameters For example which motif is searched Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 69
Visual Data Mining Interaction • Changing the scheme : Vis: changing the visual mapping or visual representation For example: changing node-link diagram to adjacency matrix DM: changing the data model For example motif search vs clustering Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 70
Visual Data Mining Interaction: Motif search and Visualization Video 05: data mining [von Landesberger et al 09] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 71
Visualization View manipulation Mapping Transformation Data Model building Knowledge Model visualization Models Information mining Parameter refinement [Keim et al. 2008] Way 3: Feedback loop SENSEMAKING, PROVENANCE, REASONING Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 72
VA Interaction Systematization 2. Visual Data Mining 1. Information Visualization 3. Reasoning [Heer & Shneiderman 12] [Gotz & Zhou 08] [Kerren & Schreiber 12] […] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 73
Reasoning/Provenance Systematization Visualization Provenance [Heer & Shneiderman 12] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 74
Reasoning/Provenance Systematization • Insight • Visualization Annotate, Bookmark • History • Record • Guide • Provenance • Cooperate • [Heer & Shneiderman 12] redo, undo, revisit Share Combined [Gotz & Zhou, Kerren et al, 75 Heer & Shneiderman 12]
Reasoning/Provenance Visualization Visual history & annotation [von Landesberger et al 14] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 76
Reasoning/Provenance Visualization Visual history & annotation [von Landesberger et al 14] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 77
Reasoning/Provenance Visualization Visual history & annotation [von Landesberger et al 14] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 78
Reasoning/Provenance Visualization Visual history & annotation [von Landesberger et al 14] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 79
Reasoning/Provenance Visualization Visual history & annotation Same interaction type aggregation [von Landesberger et al 14] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 80
Reasoning/Provenance Visualization Visual history & annotation Same interaction type aggregation using interaction systematization [von Landesberger et al 14] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 81
Reasoning/Provenance Visualization Visual history & annotation Aggregated interaction [von Landesberger et al 14] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 82
Reasoning/Provenance Visualization Visual history & annotation [von Landesberger et al 14] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 83
Reasoning/Provenance Visualization Visual history & annotation “Go back” - review/revise [von Landesberger et al 14] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 84
Reasoning/Provenance Visualization Visual history & annotation “Go back” - review/revise [von Landesberger et al 14] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 85
Reasoning/Provenance Visualization Visual history & annotation Start a different exploration path [von Landesberger et al 14] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 86
Reasoning/Provenance Visualization Visual history & annotation Start a different exploration path [von Landesberger et al 14] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 87
Systematization of Interaction 2. Visual Data Mining WHY [Bertini & Lalanne 09] [Shneiderman 96] […] 1. Information Visualization [Keim 02] WHAT [Pike et al 09] [Parsons & Sedig 14] [Dix & Ellis 98] [Spence 07] [Yi et al. 07] [Wybrow et al 14] [Zhou & Fesner 98] [Jansen, Dragicevic 13] 3. Reasoning [Roth 13] […] [Heer & Shneiderman 12] [Gotz & Zhou 08] HOW Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur [Kerren & Schreiber 12] […] 88
Systematization of Interaction - according to Info. Vis Pipeline 2. Visual Data Mining WHY [Bertini & Lalanne 09] [Shneiderman 96] […] 1. Information Visualization [Keim 02] WHAT [Pike et al 09] [Parsons & Sedig 14] [Dix & Ellis 98] [Spence 07] [Yi et al. 07] [Wybrow et al 14] [Zhou & Fesner 98] [Jansen, Dragicevic 13] 3. Reasoning [Roth 13] […] [Heer & Shneiderman 12] [Gotz & Zhou 08] HOW Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur [Kerren & Schreiber 12] […] 89
Info. Vis-Focused Systematization: Problem of ambiguous terms WHY WHAT HOW ? Why? What for? How exactly? “Change mapping” Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 90
SYSTEMATIZATION: INTERACTION SUPPORT Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 91
Interaction hierarchy WHY WHAT HOW [Sedic & Parsons 10] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 92
Interaction hierarchy Time & cognitive burden WHY WHAT HOW [Sedic & Parsons 10] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 93
Three Dimensions: Human rt o p up s n o ti c a er Int Visualization Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 94
Interaction support • Slow/cumbersome • High cognitive burden • Full control “manual” “DOI-based” “supported” “data-driven” • Fast • No cognitive burden • No control “smart” “automatic” Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 95
Interaction support • Slow/cumbersome • High cognitive burden • Full control “manual” “DOI-based” “supported” “data-driven” • Fast • No cognitive burden • No control “smart” “automatic” Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 96
Interaction support Video 06: edgelens Supported • Snap to grid • Edgelens “DOI-based” “supported” “data-driven” “smart” “manual” “automatic” [Wong & Carpendale 03] Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 97
Interaction support Data driven • Topology-aware navigation • Data-aware selection • Slow/cumbersome • High cognitive burden • Full control “manual” Video 07: data driven “DOI-based” “supported” • Fast • No cognitive burden • No control “smart” “data-driven” “automatic” Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 98
Interaction support Video 08: guidance • Slow/cumbersome • High cognitive burden • Full control “manual” Guidance • Small multiples • DOI-based exploration “DOI-based” Guidance “supported” • Fast • No cognitive burden • No control “smart” “data-driven” “automatic” Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 99
Interaction support Smart • Tableau “show me” • Slow/cumbersome • High cognitive burden • Full control “manual” “DOI-based” “supported” • Fast • No cognitive burden • No control “smart” [Heer et al 08] “data-driven” “automatic” Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 100
Interaction support Video 09: automatic • Slow/cumbersome • High cognitive burden • Full control “manual” Automatic • Node. Trix “DOI-based” “supported” • Fast • No cognitive burden • No control “smart” “data-driven” “automatic” Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 101
Part 1: Interactions SUMMARY Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 102
Summary 2. Visual Data Mining WHY [Bertini & Lalanne 09] [Shneiderman 96] […] 1. Information Visualization [Keim 02] WHAT [Pike et al 09] [Parsons & Sedig 14] [Dix & Ellis 98] [Spence 07] [Yi et al. 07] [Wybrow et al 14] [Zhou & Fesner 98] [Jansen, Dragicevic 13] 3. Reasoning [Roth 13] […] [Heer & Shneiderman 12] [Gotz & Zhou 08] HOW Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur [Kerren & Schreiber 12] […] 103
THANK YOU Q&A Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 104
Next: Interaction architecture Hans-Jörg Schulz Activities: What the user does to trigger a change in the computer (Action) Metaphor: What the user thinks the computer is doing and vice versa (Understanding) Architecture: What the computer actually does (Reaction) Vis Tutorial: Opening the Black Box of Interaction in Visualization – H. -J. Schulz, T. v. Landesberger, D. Baur 105
- Slides: 105