The Effectiveness of Online and Blended Learning A

The Effectiveness of Online and Blended Learning: A Meta-Analysis of the Empirical Literature Barbara Means, Yukie Toyama, Robert Murphy and Marianne Baki. (2013). Teachers College Record, 115(3), pp. 1 -47. Summarised by Shannon Melrose at PERf. ECT Journal Club

Context • Online learning is popular and growing • As prevalence increases, it is increasingly important to verify the effectiveness of online learning methods Norton, A. , Cherastidtham, I. , and Mackey, W. (2018). Mapping Australian higher education 2018. Grattan Institute.

Earlier meta-analyses • Bernard et al. 2004: - Compared distance education with classroom instruction (232 studies) - Found an overall effect size close to zero for student achievement (g+ = 0. 01) - Asynchronous online learning had a small positive effect, whereas synchronous instruction had a small negative effect Bernard, R. M. , Abrami, P. C. , Lou, Y. , Borokhovski, E. , Wade, A. , Wozney, L. , Wallet, P. A. , Fiset, M. and Huang, B. , 2004. How Does Distance Education Compare With Classroom Instruction? A Meta-Analysis of the Empirical Literature. Review of Educational Research, 74(3), pp. 379– 439.

Earlier meta-analyses • Sitzmann et al. 2006: - Compared web-based workplace training with face-to-face instruction (96 studies) - Found online learning has a positive effect for acquiring declarative knowledge, but a negative effect for procedural knowledge - Results differed based on study design Sitzmann, T. , Kraiger, K. , Stewart, D. and Wisher, R. , 2006. The comparative effectiveness of web-based and classroom instruction: A meta-analysis. Personnel Psychology, 59(3), pp. 623– 664.

Earlier meta-analyses • Zhao et al. 2005: - Compared distance education with classroom instruction (51 studies) - Found an overall effect size close to zero for student achievement (d = +0. 10) - Averaged effect sizes from multiple outcomes (e. g. achievement, beliefs and attitudes, satisfaction, student retention) Zhao, Y. , Lei, J. , Yan, B. , Lai, C. and Tan, H. S. , 2005. What makes the difference? A practical analysis of research on the effectiveness of distance education. Teachers College Record, 107(8), pp. 1836 -1884.

Motivation • Update literature to account for advancements in technology • Distinct from previous meta-analyses in several respects:

Blended learning • A substantial portion (but not all) of the instruction is web-based • Some content is still delivered face-to-face • Needs to be more effective than purely face-to-face instruction to justify additional time and costs

This study •

Effectiveness of online learning • Online learning had an overall positive effect on learning outcomes (g+ = 0. 20) • Purely online vs. face-to-face not significantly different (g+ = 0. 05) • Effect size for blended vs. face-to-face significantly positive (g+ = 0. 35)

Moderators of learning effect size • Practices: how online learning has been implemented • Conditions: variables outside the practitioners’ control • Study method • Often difficult coding studies with insufficient information

Online learning practices • Pedagogical approach was the only statistically significant moderator - Expository or interactive approaches were significantly positive - Active learning had an effect size not significantly different from zero • Inclusion of synchronous communication with an instructor was not a significant moderator of learning outcomes

Conditions, study method • None of the tested conditions were significant moderators of learning outcomes • The only study feature found to have a Q-Statistic significantly different from zero was equivalence of curriculum/instruction - Larger positive effect size where online and face-to-face instruction differed

Conclusions • Blended learning has a significant positive effect on learning outcomes as compared with traditional face-to-face instruction • Purely online learning showed no significant difference • Additional studies are needed, with detailed information about the online learning practices used and the conditions under which online learning takes place, in order to further understand the impact of these variables

Conclusions • “[The findings of this meta-analysis] should not be construed as demonstrating that online learning is superior as a medium. Rather, it is the combination of elements in the treatment conditions, especially the inclusion of different kinds of learning activities, that has proved effective across studies. ” • “The meta-analysis findings do not support simply putting an existing course online, but they do support redesigning instruction to incorporate additional learning opportunities online… [and] provide justification for the investment in the development of blended courses. ”