MATHTRACK MOBILE GAZE TRACKING FOR THE STUDY OF

MATHTRACK: MOBILE GAZE TRACKING FOR THE STUDY OF ATTENTION AND EMOTION IN COLLABORATIVE MATHEMATICAL PROBLEM SOLVING Markku S. Hannula Faculty of Education, University of Helsinki, Finland with Anu Laine, Eeva Haataja, Enrique García Moreno-Esteva, Miika Toivanen, Visajaani Salonen & Jessica Salminen. Saari, Phuoc Tran (University of Helsinki), Nora Mc. Intyre (Scheffied) David Clarke, Man Ching Esther Chan & Gaye Williams (University of Melbourne) www. helsinki. fi

STRUCTURE OF THE PRESENTATION Affect as a research domain Gaze tracking as a method Math. Track project Kasvatustieteellinen tiedekunta 2 Käyttäyty mistieteel linen tiedekunt a / Henkilön nimi / Esityksen 17. 9. 2020 nimi

MATHEMATICS AND AFFECT • • • Emotions in the regulation of problem solving behaviour Beliefs about mathematics explaining learning behaviour Identity and choices (e. g. career decisions) Anxiety and the mechanisms how it hinders learning Enjoyment and flow as optimal learning mode Boredom and its consequences The mechanisms of affective regulation are largely nonconscious and inaccessible to introspection or observation Affective processes in school take place in a social context where different social processes interact How to dig deeper? Kasvatustieteellinen tiedekunta

HISTORY PRE 1990 S (MCLEOD, 1992) Behaviorism neglected affect as ”imaginary constructs” Social psychology measured attitudes Large number of quantitative studies Differences between countries Decline in affect over school years Gender differences Lack of a theoretical framework Confusing , ambiguous terminology Local theories (e. g. Self-efficacy, mathematics anxiety, attribution theories, aesthetics) Kasvatustieteellinen tiedekunta Markku Hannula 17. 9. 2020

NEW TRENDS IN 1990’S (MCLEOD, 1992) Cognitive science avoided affect to avoid complexity ”affective and cognitive domains are intimately linked. ” ‒ E. g. Metacognition and affect (interest, confidence) Cognitive psychology emphasizes theoretical issues, qualitative methods, beliefs and emotion Kasvatustieteellinen tiedekunta Markku Hannula 17. 9. 2020

ELEMENTS OF AFFECT Kasvatustieteellinen tiedekunta

MAIN CRITICISMS Ambiguous terminology / Definitions Conceptual framework Social turn Kasvatustieteellinen tiedekunta Markku Hannula , Joensuu 17. 9. 2020

SOCIAL TURN IN ME (LERMAN 2000) Theories on affect focussed on the individual Cobb & Yakel: beliefs – norms (extension) Identity (ICME 2004, Sfard & Prusak, etc. ) Kasvatustieteellinen tiedekunta Markku Hannula 17. 9. 2020

A NEW (META)THEORETICAL FOUNDATION (Hannula 2011, 2012) Kasvatustieteellinen tiedekunta Markku Hannula 17. 9. 2020

STATE AND TRAIT ASPECTS OF THE ELEMENTS State Cognition Motivation Emotion Thoughts in mind Active goals Emotional state Trait Concepts, Needs, (memory) facts, values, scripts, etc. desires Kasvatustieteellinen tiedekunta Emotional disopositions (attitude)

HOW ABOUT THE SOCIAL TURN AND THE EMBODIED COGNITION? Kasvatustieteellinen tiedekunta Markku Hannula 17. 9. 2020

IS AFFECT INDIVIDUAL OR SOCIAL? • Generated by individual experiences • Constructed socially • Characteristics of the individual • Emerging in shared social scenarios • Psychological theories • Social theories • In-depth interviews • Observing situated behaviour Kasvatustieteellinen tiedekunta

AFFECT IS INDIVIDUAL AND SOCIAL ! • Generated by individual experiences • Constructed socially • Characteristics of the individual • Emerging in shared social scenarios • Psychological theories • Social theories • In-depth interviews • Observing situated behaviour Kasvatustieteellinen tiedekunta

THREE READOUTS OF EMOTION (BUCK, 1999) Readout target Readout function I Autonomic/ Adaptation/ endocrine/ homeostasis immune system responding II Expressive Communication/ behaviour social coordination III Subjective Self-regulation experience Kasvatustieteellinen tiedekunta Accessibility Learning Not accessible Physiological adaptation Accessible to others (and self) Accessible to self Markku Hannula Social development Cognitive development 17. 9. 2020

Metatheory for affect (Hannula, 2012) State Trait Cognition Motivation Emotion Faculty of Educational Sciences / Markku S. Hannula www. helsinki. fi/yliopisto 17. 9. 2020 15

State Trait Cognition Motivation Emotion Faculty of Educational Sciences / Markku S. Hannula www. helsinki. fi/yliopisto 17. 9. 2020 16

State Trait Cognition Motivation Emotion Faculty of Educational Sciences / Markku S. Hannula www. helsinki. fi/yliopisto 17. 9. 2020 17

State Trait Cognition Motivation Emotion Faculty of Educational Sciences / Markku S. Hannula www. helsinki. fi/yliopisto 17. 9. 2020 18

State Trait gnition Motivation Emotion Faculty of Educational Sciences / Markku S. Hannula www. helsinki. fi/yliopisto 17. 9. 2020 19

Ps S Em ych oci olo al bo gic die al d State Trait gnition Motivation Emotion Faculty of Educational Sciences / Markku S. Hannula www. helsinki. fi/yliopisto 17. 9. 2020 20

Ps So yc cia h l Em olo bo die gic d al State Trait gnition Motivation Emotion Kasvatustieteellinen tiedekunta Faculty of Educational Sciences / Markku S. Hannula 17. 9. 2020 21

Ps So yc cia h l Em olo bo die gic d al State Trait gnition Motivation Emotion Kasvatustieteellinen tiedekunta Faculty of Educational Sciences / Markku S. Hannula 17. 9. 2020 22

THE OTHER ANGLE TO AFFECT Physiological Affect Neural as a activation, state physiological adaptation. Psychological Feelings, emotions, thoughts, meanings, goals Social interaction, communication, classroom ‘weather’ Affect Brain structure, as a neural trait connections, temperament Attitudes, values, beliefs, motivational orientations Norms, social structures, classroom climate Kasvatustieteellinen tiedekunta

SOCIAL DYNAMICS OF AFFECT Kasvatustieteellinen tiedekunta Markku Hannula 17. 9. 2020

BUILDING THINKING CLASSROOM Liljedahl, P. (2016). Building thinking classrooms: Conditions for problem solving. In P. Felmer, J. Kilpatrick, & E. Pehkonen (eds. ), Posing and Solving Mathematical Problems: Advances and New Perspectives. (pp. 361 -386). New York, NY: Springer. Liljedahl, P. (2014). The affordances of using visibly random groups in a mathematics classroom. In Y. Li, E. Silver, & S. Li (eds. ), Transforming Mathematics Instruction: Multiple Approaches and Practices. (pp. 127 -144). New York, NY: Springer. Kasvatustieteellinen tiedekunta

COLLABORATIVE PROBLEM SOLVING • … is central in modern education • … can be studied using mobile gaze trackers • … is interesting beyond the immediate educational context Kasvatustieteellinen tiedekunta MAVI 24 Markku S. Hannula 17/09/2020 26

MATHTRACK Four-year project 2016 -2020 Funded by Academy of Finland Kasvatustieteellinen tiedekunta 27 17. 9. 2020

WHY VISUAL ATTENTION Affective communication consists of more than just words; facial expressions, gestures, glances, body movement and prosody are also important aspects of it. This multimodality of the communication raises a question regarding the student’s attention to the different channels used. Kasvatustieteellinen tiedekunta

EYE TRACKING • Eye tracking (or gaze tracking) has been used to study visual attention mostly • in laboratory settings • lone individuals • with a focus on cognition BUT: Learning is contextual, affective, and social! Kasvatustieteellinen tiedekunta

MOBILE GAZE TRACKER (HARDWARE) • The teacher and four students wear a mobile gaze tracking device • Frame equipped with mini cameras which produce a video scene and keep track of the gaze direction. • The gaze tracking glasses are connected with three cords to a laptop carried in a backpack. Toivanen, M. , Lukander, K. , & Puolamäki, K. (2017). Probabilistic Approach to Robust Wearable Gaze Tracking. Journal of Eye Movement Research, 10(4), [2]. DOI: 10. 16910/jemr. 10. 4. 2 Kasvatustieteellinen tiedekunta • The glass frame is 3 D-printed, all components are standard off-theshelf, and the instructions and software published • Do it yourself! MAVI 24 Markku S. Hannula 17/09/2020 30

MOBILE GT SOFTWARE • The direction of gaze is softwarecalculated, producing a mark on the video scene. • The gaze location is computed as the recording takes place. Kasvatustieteellinen tiedekunta MAVI 24 Markku S. Hannula 17/09/2020 31

METHODS – MAIN STUDY: RESEARCH DESIGN • Teacher and four students wear gaze-tracking glasses • Students work collaboratively on a mathematical problem • Same task across groups • Same structure (alone – with a friend – group of four – whole class) • Scripted teacher scaffolding (no hints, just encouragement and clarifying questions) • Stimulated recall interviews after the lesson Kasvatustieteellinen tiedekunta

DATA: SYNCHRONIZED VIDEOS + STIMULATED RECALL INTERVIEWS + QUESTIONNAIRE Kasvatustieteellinen tiedekunta

Gaze as a measure of collaboration (and engagement) Garcia Moreno-Esteva et al. (In preparation) www. helsinki. fi/yliopisto

Potential approaches to study the dynamic group level affect • Gaze synchrony • Indication of group cohesion • Eye contact (Eeva’s MAVI paper) or other attention to peers • Communication, cohesion • Reactions to facial expressions and other expressions of affect • Reactivity to other’s, emotional contagion www. helsinki. fi/yliopisto

Hannula, M. S. (2004 a). Affect in mathematical thinking and learning. Turku, Finland: Annales universitatis Turkuensis B 273. Hannula, M. S. (2004 b. Regulating motivation in mathematics. A paper presented at the Topic Study Group 24 of ICME-10 conference. Retrieved September 15 th 2005 at http: //www. icme organisers. dk/tsg 24/Documents/Hannula. Kasvatustieteellinen tiedekunta

Chapman, J. W. , Tunmer, W. E. & Prochnow, J. E. (2000). Early reading -related skills and performance, reading self-concept and the development of academic self-concept: A longitudinal study. Journal of educational psychology 67, 145 -152. Damasio, A. R. (1994). Descartes Error: Emotion, Reason and the Human Brain, New York: G. P. Putnam’s Sons Hannula, M. S. , Maijala, H. , & Pehkonen, E. (2004). Development of understanding and self-confidence in mathematics; grades 5 -8. In M. J. Høines & A. B. Fuglestad (Eds. ) Proceedings of the 28 th Conference of the International Group for the Psychology of Mathematics Education. Vol 3, pp. 17 – 24. Bergen University College. Le. Doux, J. (1998). The Emotional Brain. Phoenix, Orion Books Ltd. Linnanmäki, K. (2002). Matematikprestationer och självuppfattning, en uppföljninggsstudie I relation till skolspråk och kön. [Mathematical proficiency and self-concept; a longitudinal study with an attention to language and gender]. Turku: Åbo Akademi University Press. Hannula, M. S. , Maijala, H. , Pehkonen, E. & Nurmi, A. 2005. Gender comparisons of pupils’ self-confidence in mathematics learning. Nordic Studies in Mathematics Education 10 (3 -4), 29 – 42 Kasvatustieteellinen tiedekunta

Mc. Leod, D. B. (1992). Research on affect in mathematics education: a reconceptualization. In D. A. Grouws (Ed. ) Handbook of Research on Mathematics Learning and Teaching, pp. 575 -596. New York: Mac. Millan. Furinghetti, F. , & Pehkonen, E. (2002). Rethinking characterizations of beliefs. In. G. C. Leder, E. Pehkonen & G. Törner (Eds. ) Beliefs: A Hidden Variable in Mathematics Education, pp. 39 – 58. Dordrecht, The Netherlands: Kluwer. Boaler, J. (1997). Reclaiming school mathematics: The girls fight back. Gender and Education 9, 285 -305. Yakel, E. & Cobb, P. (1996). Sociomathematical norms, argumentation, and autonomy in mathematics. Journal for Research in Mathematics Education 27 (4), 458 -477. Kasvatustieteellinen tiedekunta

Buck, R. (1999). The Biological Affects: A Typology. Psychological Review 106 (2), 301 -336. Fennema, E. & Sherman, J. A. 1976. Fennema-Sherman mathematics attitudes scales. JSAS Catalog of Selected Documents in Psychology, 6, 31 (Ms. No. 1225). Hart, L. (1989). Describing the affective domain: saying what we mean. In Mc. Leod, D. & Adams, V. (eds. ) Affect and Mathematical Problem Solving, 3745. Mc. Leod, D. B. (1992). Research on affect in mathematics education: a reconceptualization. In D. A. Grouws (Ed. ) Handbook of Research on Mathematics Learning and Teaching, pp. 575 -596. New York: Mac. Millan. Kasvatustieteellinen tiedekunta

Lehman, B. , D'Mello, S. K. , and Person, N. , (2008). All Alone with your Emotions: An Analysis of Student Emotions during Effortful Problem Solving Activities. Supplementary Proceedings of the Workshop on Emotional and Cognitive issues in ITS (WECITS). Lerman, S. (2000). The social turn in mathematics education research. In J. Boaler (Ed. ), Multiple perspectives on mathematics teaching and learning (pp. 19– 44). Westport, CN: Ablex. Martin, A. J. (2001) The Student Motivation Scale: a tool for measuring and enhancing motivation, Australian Journal of Guidance and Counselling 11 (2001), pp. 1– 20 Meyer, D. K. & Turner, J. C. (2002). Discovering emotion in classroom motivation research. Educational Psychologist 37 (2), 107 – 114. Kasvatustieteellinen tiedekunta

Rösken, Schoenfeld, A. H. (1985). Mathematical Problem Solving. San Diego: Academic Press. Goldin, G. A. (2000). Affective pathways and representation in mathematical problem solving. Mathematical Thinking and Learning 2 (3), 209219. Liljedahl, P. (2005). Sustained engagement: preservice teachers' experience with a chain of discovery. In Proceedings of the Fourth Congress of the European Society for Research in Mathematics Education 17. – 21. 2. 2005 in Sant Feliu de Guíxols, Spain. Retrieved 3. 11. 2005 at http: //cerme 4. crm. es/Papers%20 definitius/2/wg 2 lis tofpapers. htm Kasvatustieteellinen tiedekunta