ITS 2020 Learning Analytics Dashboard for Motivation and

ITS 2020 Learning Analytics Dashboard for Motivation and Performance Damien Fleur, 12 June 2020 Co-authors Bert Bredeweg, Wouter van den Bos

Short Introduction Damien S. Fleur Research Informatics Institute, University of Amsterdam Metacognition Department of Psychology, University of Amsterdam • Technology-enhanced Learning • Cognitive neuroscience

Learning Analytics (Dashboards) • “Support users in collecting personal information about various aspects of their life, behavior, habits, thoughts, and interests” (Verbert et al. 2014). • Strong evidence in terms of improved learning behavior and learning outcomes remains scarce.

Motivation and Academic Achievement • Motivation found to be related to academic achievement (e. g. Singh, Granville & Dika, 2002) • Highly motivated students tend to be more attentive to their learning and put more effort (Schunk and Zimmerman, 2012)

Learning Analytics Dashboard Motivation Social Comparison Goal Orientation Academic Achievement

Social Comparison theory • Humans inherently compare their abilities others (Festinger, 1954) • Local information tends to be more highly weighted in self-evaluation than distant information (Gerber, Wheeler & Suls, 2018) • People may assimilate their self-evaluation upward to those who are better off, especially when there is not threat on self-esteem (Gerber, Wheeler & Suls, 2018)

Goal orientation • By setting goals and working towards them, students tend to become more motivated and have higher learning outcomes. • Particularly effective when goals are specific, reachable and when feedback is given which shows progress in relation to the goals. (Locke, 1996)

Research Questions • Can motivation be stimulated with a Learning Analytics Dashboard with social comparison? • Can motivation be stimulated with a Learning Analytics Dashboard with goal orientation? • Does it lead to higher academic achievements?

Design Predicted final grade Estimated probability Number of students You vs. peers with similar goal grades Mean grade Final grade github. com/Uv. A-FNWI/coach 3

Social Comparison • 9 anonymous peers with similar goals (blue) You vs. peers with similar goal grades Number of students • Current average grade • Average of the peers 0. 5 -1. 5 > subject • 30 -40% of peers with grade ≤ subject • Variation of the “knapsack algorithm” (Dantzig, 1940) Mean grade

Grade prediction Predicted final grade • Bayesian Ridge Regression (Scikit-learn package) • Estimated final grade (µ) Estimated probability • Grades of previous 2 years • Uncertainty (σ) • Prediction more precise over time Final grade

e w Ne d gra Update Comparison + Prediction (LMS) Accessible within environment Ge da ner sh ate bo s ar d

Materials & Participants • 1 st year Bachelor students, as part of a course 34 38 • Questionnaires (filled by both groups) • • MSLQ: Motivation (extrinsic, intrinsic) MAI: Metacognition 16 10

Procedure LAD updated Start Week 2 Quiz Information & Consent Questionnaires Set goal grade LAD updated Week 3 Hw 1 LAD updated Week 4 Hw 2 Midterm 1 LAD updated Week 5 LAD updated Week 6 Group assignment Update goal grade LAD updated Week 7 Written Midterm 2 Group assignment End

Hypotheses Motivation: Access > No Access Performance: Access > No Access Higher motivation → Higher Performance

Results Interaction effect (β = -1. 07, S. E. = 0. 42, t = 2. 54, p = 0. 01)

Results (β = 0. 36, S. E. = 0. 17, t = 2. 07, p = 0. 04 )

Conclusion ✓ Motivation: Access > No Access ✓ Performance: Access > No Access ? Higher motivation → Higher Performance Future Research • Relation between motivation and achievement • Are the effects driven by social comparison or goal orientation (or both)?

References Festinger, L. : A theory of social comparison processes. Hum. Relat. 7, 117– 140 (1954) Gerber, J. P. , Wheeler, L. , Suls, J. : A social comparison theory meta-analysis 60+ years on. Psychol. Bull. 144, 177 (2018). https: //doi. org/10. 1037/bul 0000127 Locke, E. A. : Motivation through conscious goal setting. Appl. Prev. Psychol. 5, 117– 124 (1996). https: //doi. org/10. 1016/S 0962 -1849(96)80005 -9 Schunk, D. H. , Zimmerman, B. J. : Motivation and self-regulated learning: Theory, research, and applications. Routledge (2012) Singh, K. , Granville, M. , Dika, S. : Mathematics and science achievement: Effects of motivation, interest, and academic engagement. J. Educ. Res. 95, 323– 332 (2002). https: //doi. org/10. 1080/00220670209596607