Evaluating the effectiveness of innovative pedagogies a personal

Evaluating the effectiveness of innovative pedagogies – a personal reflection Simon Lancaster

The plural of anecdote? A. The plural of anecdote is not data. B. The plural of anecdote is data. 0% 0% The plural of anecdote is not The plural of anecdote is data. November 2020

Professor Mymates module evaluation data Mymate module evaluation 120 6 100 5 80 4 60 3 40 2 20 1 0 2008 2009 2010 2011 NSS November 2020 2012 Median Mean 2013 2014 0 2015

Contemporary versus traditional?

What proportion of the time are our students actively thinking during (lecture theatre) teaching? A. <10% B. 10– 30% C. 30– 50% D. 50– 70% E. 70– 90% F. >90% 0% <10% November 2020 0% 0% 10– 30% 30– 50% 50– 70% 70– 90% 0% >90%

Our model of lecture flipping

The importance of the question

What is your objective for a question posed during teaching? Rank 1 2 3 Responses 4 5 6 1 November 2020 2 3 4 5 6

What is your objective for a question posed during teaching?

The Goldilocks Zone

Opposites attract, like repels like November 2020

Which is favourable? A. B. C. D. Li+ + e− Li Li+ Li 2+ + e− Li− + e- Li 2− November 2020 0% 0% A. B. C. D.

How do you feel about your answer? A. Very confident B. Confident C. Not confident D. Not at all confident Which is favourable? A. Li Li+ + e− B. Li+ + e− Li C. Li+ Li 2+ + e− D. Li− + e- Li 2− November 2020 0% 0% Very confident Confident Not confident Not at all confident

Which is favourable? A. B. C. D. Li+ + e− Li Li+ Li 2+ + e− Li− + e- Li 2− November 2020 0% 0% A. B. C. D.

Evidence Student activity 9 12 54 25 Thinking PI Interactions Lecturing

Lancaster’s module evaluation data Lancaster module evaluation 120 6 100 5 80 4 60 3 40 2 20 1 0 2008 2009 2010 2011 NSS November 2020 2012 Median Mean 2013 2014 0 2015

What is learning gain? A. B. C. November 2020 The learning achieved by a student between two points in time. The improvement in knowledge, skills, work-readiness and personal development made by students during their time spent in higher education. The value added by an institution in terms of the increased likelihood of gaining a good honours degree given a certain range of entry qualifications. 0% 0% 0% A. B. C.

How do you feel about your answer? A. Very confident B. Confident C. Not confident D. Not at all confident What is learning gain? A. B. C. November 2020 0% 0% Very confident Confident Not confident Not at all confident The learning achieved by a student between two points in time. The improvement in knowledge, skills, work-readiness and personal development made by students during their time spent in higher education. The value added by an institution in terms of the increased likelihood of gaining a good honours degree given a certain range of entry qualifications.

What is learning gain? A. B. C. November 2020 The learning achieved by a student between two points in time. The improvement in knowledge, skills, work-readiness and personal development made by students during their time spent in higher education. The value added by an institution in terms of the increased likelihood of gaining a good honours degree given a certain range of entry qualifications. 0% 0% 0% A. B. C.

The UEA project At UEA, we are examining 3 ways of measuring Learning Gain: - Student marks/grades/GPA - Self-efficacy - Concept inventories

Concept inventories Conceptual Validated Reliable

Collaborative concept inventory use November 2020

. Normalised gain

Mean normalised gain for the bonding concepts inventory 0. 19

Self-efficacy 1. Determine an average mark for the quiz. 2. Assign student to above or below average group. 0, 1 3. Calculate student average confidence. 4. Assign student to above or below average confidence group. High confidence High attainment Low attainment November 2020 X Low Confidence

Self-efficacy over the course of the module SESSION WK 2 WK 4 WK 6 WK 8 November 2020 SIZE 89 86 83 83 P-value 0. 058 0. 015 0. 001 0. 126 Association BORDER YES NO

ACKNOWLEDGEMENTS Dennis Cook Dr Fabio Arico Prof Neil Ward Helena Gillespie Dr Annamari Ylonen Prof Eric Mazur Dr Ross Galloway Dr Anna Wood Dr Anne Nortcliffe HEFCE Dr Adam Longcroft Turning Technologies

4* evidence