Centro de Filosofia das Cincias da Universidade de

Centro de Filosofia das Ciências da Universidade de Lisboa January 16, 2008 Diagrammatic reasoning Theoretical questions and empirical research Valeria Giardino Università di Roma ‘La Sapienza’ Institut Jean Nicod valeria. giardino@gmail. com http: //valeria. giardino. googlepages. com/

Outilne of the talk n Reasoning and diagrams: the topic and how to approach it n Cognitive issues: The proposed framework and empirical investigations n Further research: the project

Prologue Reasoning and diagrams

The background There has always been this aspiration to redescribe all cognitive processes in terms of one type of representation only. . . Studies of the development of scientific fields: - dialectical play of qualitative and quantitative modes of description and argumentation - neither cognitive primacy or epistemic superiority of the linguistic mode

Extended minds (Clark 1997) n Scaffolded minds exploitation of external structures which might alter and inform an individual’s intrinsic mode of processing and undestanding If we remove the external component the system's behavioral competence will drop, just as it would if we removed part of its brain. This sort of coupled process counts equally well as a cognitive process, whether or not it is wholly in the head.

Some ‘external components’ n advanced external computing resources n the use of pen and paper to perform long multiplication (Mc. Clelland et al 1986, Clark 1989) n the use of physical re-arrangements of letter tiles to prompt word recall in Scrabble (Kirsh 1995) n the use of instruments such as the nautical slide rule (Hutchins 1995) n the general paraphernalia of language, books, diagrams, and culture

Conceptual blending Hutchins (2004) n Material anchors physical objects are themeselves input to the conceptual blending process better to ask under what conditions something becomes a material anchor than whether it is a material anchor

A conceptual blend operates in two input mental spaces to yield a third space, the blend. Partial structure from the input spaces is projected into the blended space, which has emergent structure of its own.

Association of conceptual structure with material structure as a key cognitive strategy

Diagrams as external components or material structures for reasoning: methodological issues n Why a general framework is necessary: heterogeneity of the literature n Methodological problem: Case studies or general framework? n Two approaches: historical and cognitive investigations

Cognitive investigations

The proposed framework n cognitive advantage in using a diagram in a number of reasoning and communication tasks n this cognitive advantage not only determined by the fact that diagrams are just ‘more visual’ than linguistic sentences n diagrams represent a cognitive advantage not in principle, but depending on the task in question

How to use a diagram CONSTRAINTS ‘regularities’ in the way the diagram is perceived which interact (perceptual constraints) n a context of reference and its theoretical assumptions (conceptual constraints) n AIMS to display data: static diagrams n to generate inference: dynamic diagrams n

CONSTRAINTS

For free? ? ? Conventions and domain specificity The important lesson is that diagram literacy in science must be taught n n Configurations are only preconditions for interpretation

Domain specificity n “. . . developing diagrammatic competence goes hand-inhand with acquiring conceptual knowledge in the domain of application. ” Novick (2006) n Advanced subjects: perceptual chunks and ‘more abstract’ diagrams n From perceptual resemblance to relevant features?

Questions of isomorphism n Iconic diagrams n Graphs and charts n Schematic diagrams Hagarty, Carpenter and Just (1991)

One example: Butcher and Kintsch (2004) (1 a) A simple diagram emphasizing functional relations (1 b) A detailed diagram preserving anatomical accuracy

• Diagrams effective when they prompt learners to engage in the cognitive processes necessary for deep understanding • Differences in diagram representation can affect comprehension processes • In this situation, simple diagrams most effectively guided learners; however, one must be careful not to misinterpret this finding

AIMS

Information extraction: Static diagrams n Problem solving: organizing information n Short-term memory (spatially organized external location in which to note down information) n Long-term memory (information retrieval)

Reducing the search space Larkin and Simon (1987) We have three pulleys, two weights, and some ropes, arranged as follows: 1. The first weight is suspended from the left end of a rope over Pulley A. The right end of this rope is attached to, and partially supports, the second weight. 2. Pulley A is suspended from the left end of a rope that runs over Pulley B, and under Pulley C. Pulley B is suspended from the ceiling. The right end of the rope that runs under Pulley C is attached to the ceiling. 1. Pulley C is attached to the second weight, supporting it jointly with the right end of the first rope.

Reducing cognitive efforts: n Informational equivalence n Computational difference n Locational indexing

Manipulation: Dynamic diagrams n Problem solving: finding the answer n Informal inferences: logic of discovery? n Main problem: discovery

Procedures n Central issue: the way diagrams are manipulated to perform some inference n Informal inferences which often take the form of n The rules are implicit but are usually externalised as procedures transformations

Some proposals

Epistemic actions Kirsh and Maglio (1994) external actions that an agent performs to change its own computational state The TETRIS case: Epistemic uses of rotation

Compared with mental computations, they reduce • space complexity (memory) • time complexity (number of steps) • unreliability (probability of errors)

Spatial transformations Trickett and Trafton (2006) Graphs comprehension: Perceptual and conceptual processes (interpretation) + SPATIAL transformations (from explicit information to implicit information)

A spatial object is transformed from one mental state or location into another mental state or location

üadding or deleting features ümental rotation ümentally moving an object üanimating a static image ümaking comparisons between different views ü. . . any other mental operation which transforms a spatial object from one state or location into another

Hypothetical drawings Shimojima and Fukaya (2003) In the presence of a visually accessible graphical representation, an agent assumes a hypothetical transformation of the graphic that adds a premise to it, inferes about the results of a tranformations, and translates an obtained conclusion to a conclusion about the referent of the graphic One precise way a cognitive agent interacts with graphical representations to unburden inferential loads

Reasoning. . . about a picture as the referent inference projection from the picture to the physical hinge about the picture’s referent? picture only information source and memory aid

Eye fixations in a trial on a type-a problem (left), on a type-b problem (middle), and on a type-a problem (right) Eye fixations in a trial on a type-e problem (top), on a type-c problem (middle), and on a type-c problem (bottom).

Epilogue The project

n Shift from thinking about the one true mental representation system to thinking about how users adopt the representations they do n If two groups of users are shown to contrast in how they respond to using different kinds of external representation of information, then these global differences can be used to pinpoint differences in underlying processes

Objectives (1) the evaluation of the cognitive advantages in the use of diagrams; (2) a classication of diagrams.

Objective (1) n Question 1: are diagrams are only auxiliary for reasoning or they are more fundamental? n Empirical tests and theoretical framework; finding the good question

Objective (2) n Question 2: is it possible to give a classication of diagrams? n To tidy up the vocabulary commonly used in the literature and clarify the topic; to answer to the question whethere is something that can be acknowledged as typically diagrammatic, or it is necessary to hypothesise a continuum

n Hypothesis (a. ) Diagrams confer cognitive advantage over linguistic representations n Hypothesis (b. ) Diagrams constitute a cognitive advantage over mental computations n Hypothesis (c. ) It is necessary to move from the question of what a diagram is to the question of what a diagram is used for

First step towards a classification for diagrams Two categories: I. static diagrams (data displays) II. dynamic diagrams (inference generators) Some coordinates: First group (cognitive features) 1. Non-symbolism (presence of labels and captions) 2. Extra-mental computation required 3. Isomorphism to target Second group (pragmatic features) 4. Context 5. Task 6. Target

Second step experimental tests General schema: - subjects will be shown two different formats of diagrams designed to contain the same set of data, and to generate inferences - subjects will be given a questionnaire to fill containing both questions about data extraction and possible conclusions to draw, and questions about feedback on the degree of difficulty of the tasks The experiment design will be developed along with the definition of the classification

Thank you! References Butcher - Kintsch (2004), Learning with Diagrams: Effects on Inferences and the Integration of Information Clark (1989). Microcognition - (1997), Being there Gooding (2004), Visualization, Inference and Explanation in the Sciences Hagarty et al. (1991), Diagrams in the comprehension of scientific texts Hutchins (1995), Cognition in the Wild - (2004), Material anchors for conceptual blends Kirsh - Maglio (1994), On Distinguishing Epistemic from Pragmatic Actions Kirsh (1995), The intelligent use of space Larkin – Simon (1987), Why a diagram is (sometimes) Valeria Giardino Università di Roma ‘La Sapienza’ Institut Jean Nicod Marie Curie Fellow to be (after the long and annoying negotiation. . . ) worth ten thousand words Mc. Clelland et al. (1986), The appeal of parallel distributed processing Novick (2006), The Importance of Both Diagrammatic Conventions and Domain-Specific Knowledge for Diagram Literacy in Science Shimojima - Fukaya (2003), Do we Really Reason about a Picture as a Referent? Stenning – Lemon (2001), Aligning Logical and Psychological Perspectives on Diagrammatic Reasoning Trickett - Trafton (2006), Toward a Comprehensive Model of Graph Comprehension: Making the Case for Spatial Cognition