Teaching as a design science Designing and assessing

Teaching as a design science: Designing and assessing the effectiveness of the pedagogy in learning technologies Diana Laurillard London Knowledge Lab Institute of Education e. Learning Forum Asia 2013 Hong Kong Baptist University 29 -31 May 2013

The Challenges to Higher Education The issues • Global demand for HE • The aims of HE • The roles of TEL • Modelling learning technologies • Teachers as innovators

The policy context • Teachers to have the capacity to plan and deliver ICT-enriched learning experiences for students to become self-directed and collaborative learners. . . (Singapore ICT Masterplan, 2008) • Teacher capacities will be developed in instructional design, selection and critical evaluation of digital content, and strategies for effective use of digital content to enhance student learning. (India Government, 2009) • Professional educators connected by technology to empower, and inspire effective teaching (US Plan 2010) • … greater prioritisation of teaching partnerships between technologists, learning support specialists and academics, and an end to the ‘not invented here’ syndrome… Good practice must also be shared. (HEFCE OLTF, 2011) • Professional development is needed for all school staff to support the process of ICT integration in schools (UNESCO, ICT in Primary, 2012)

The global demand for HE By 2025, the global demand for higher education will double from 100 m to ~200 m per year, mostly from emerging economies (NAFSA 2010) The new UNESCO goals for education: • Every child completes a full 9 years of free basic education … • Post-basic education expanded to meet needs for knowledge and skills … (Draft for UNESCO post 2015 goals) - Implying significant teacher training needs for HE 25: 1 student: staff ratio = 4 m new teachers? ?

The aims and purpose of HE The purposes of higher education: • To inspire and enable individuals to develop their capabilities to the highest… • To increase knowledge and understanding for their own sake… • To serve the needs of an adaptable, sustainable knowledge-based economy… • To play a major role in shaping a democratic, civilised, inclusive society (NCIHE, 1997) How could mass HE achieve that nurturing of the individual, while reducing the current 25: 1 student: staff ratio for student support?

An example: The Duke MOOC Bioelectricity: A Quantitative Approach Taught in class for over 20 years Experimental move to a free and open MOOC 12, 000 students enrolled from >100 countries • 8 weeks long • 97 ~6 min videos • 22 GB of data • 1052 files • 18 graded exercises, including a peer-graded writing assignment and final exam (Duke University 2013)

The Duke MOOC Not for undergraduates Enrolled students Potential undergraduates

The Duke MOOC Not for the faint-hearted Comparable with normal online u/g courses

Basic MOOCs vs the Duke MOOC: Comparing the learning experience Basic: 8 weeks, providing 50 hours learning time, no teacher support Duke: 8 weeks, providing 50 hours learning time including support: Videos and pdfs Quizzes High on prep time Wiki Zero contact for 42 hours Peer discussions Peer grading Low on prep time Tutored discussions High contact for 8 hours Summative assessment 420 hours to develop materials and course design 200 hours to support ~500 students for 8 hours = 20: 1 student: staff ratio Report at http: //bit. ly/ZRMbjp

Comparing teacher hours for a basic MOOC and the Duke MOOC (48 hour course) Total teacher time Teacher support time rises to 2000 hours for 5000 students. 3000 2500 2000 Duke MOOC 1500 Basic MOOC 1000 200 Prep time 500 0 50 5000 2000 hours = 1 year of a tutor for a 5 credit course. = 24 FT tutors for 120 credit course. Prep time = 420 hrs Support time 50 5000 Duke MOOC 20 hrs 2000 hrs Basic MOOC 0. 00

Modelling the benefits and costs • We need to understand the pedagogical benefits and teacher time costs of online HE • What are the new digital pedagogies that will address the 25: 1 student support conundrum? • Who will innovate, test, and build the evidence for what works at scale online? TEACHERS!

Pedagogies for supporting large classes Concealed MCQs The virtual Keller Plan The vicarious master class Pyramid discussion groups Conceal answers to question Ask for user-constructed input Introduce content Reveal multiple answers Self-paced Ask user to practice select nearest fit Tutor-marked test 240 individual students produce Tutorial for 5 representative Student becomes tutor for credit response to open question students Until half class is tutoring thejoint rest Pairs compare and produce Questions and guidance represent response all students’ needs Groups of 4 compare and produce joint response and post as one of 10 responses. . . 6 groups of 40 students vote on best response Teacher receives 6 responses to comment on

Digital pedagogies for scaling up to higher student: staff ratios without losing quality Concealed MCQs Like MCQs, no teacher support More challenging learning experience The virtual Keller Plan Little teacher support Every learner receives 1 -1 support The vicarious master class One teacher guides 5 students They represent all students’ needs Pyramid discussion groups Teacher reacts to 6 student outputs All students contribute, discuss, vote

How might technology help teachers? First things first – how do students learn? How does technology help? Tools for teachers to build pedagogic knowledge Teachers as design scientists…

The learner learning Acquiring L L Learner C C concepts Teacher concepts Inquiring Modulate Generate L L Learner P P practice Learning through acquisition, instruction Learning through inquiry

The learner learning L L Learner C C concepts Teacher concepts Modulate Generate Task Feedback L L Learner P P practice Learning environment Actions Learning through practice with meaningful intrinsic feedback

The learner learning Acquiring Ideas, questions L L Learner C C concepts Teacher concepts Peer concepts Ideas, questions Inquiring Modulate Generate Task/Feedback Outputs L L Learner P P practice Learning environment Actions Peer practice Outputs Learning through discussion from peers’ ideas, questions Learning through sharing from peers’ practice

Representing types of learning Acquiring Teacher concepts Modulate Teacher communication Inquiring cycle Producing Peer communication Discussing cycle Peer concepts Peer modelling Sharing cycle Peer practice Modulate Generate Learning environment L L Learner C C concepts Teacher modelling Practising cycle L L Learner P P practice The teacher needs to design for all these types of learning

Books, lectures, Libraries, journals, papers, websites, repositories, videos, AR, podcasts… databases, Web… Tutorials, seminars, small groups, online forums, social media… Deploying learning technologies Book… Podcast Teacher concepts Modulate Generate Learning environment Teacher communication Library… Web cycle Essay… Model L L Learner C C concepts Tutorial… communication Facebook cycle Peer concepts Peer modelling Project… VLE cycle Peer practice Modulate Generate Teacher Lab… modelling Simulation cycle L L Learner P P practice Labs, exercises, problems, Essays, designs, performance, Project groups, The teacher needs to use all types of learning projects, serious programs, games, videos, ppts, teamwork, digital Google technology within Conversational Framework models, simulations… assets, the models, e-portfolios… groups, wikis, VLEs…

Collecting learning analytics What accessed in what sequence Questions asked Book… Podcast Teacher concepts Modulate Generate Learning environment Teacher communication Library… Web cycle Essay… Model L L Learner C C concepts Social interaction patterns Tutorial… communication Facebook cycle Peer concepts Peer modelling Project… VLE cycle Peer practice Modulate Generate Teacher Lab… modelling Simulation cycle Tracked inputs, reaction times, sequence L L Learner P P practice Analysis of essay content, quiz scores, game scores, accuracy of model Tracked group outputs, peer assessment

Forms of TEL/online learning activities Learning activities for online courses • Guided TEL resources (model, AR) • Access to expositions (lecture videos) • Automated grading (MCQs, models) • Readings (pdfs) • Guided collaboration activities (wiki, AR) • Peer group discussion (forum) • Peer grading against criteria (share) • Tutored discussion (forum) • Tutor feedback (assignments) • Tutor-based assessment (exams) What are effective ways of combining and sequencing these activities for learning and formative assessment, that will help students achieve the intended outcomes?

Tools for teachers as learning designers Teachers as designers need the tools for innovation To find or create new ideas To collect learning analytics Adopt Redesign Adapt Analyse Test Publish Creating knowledge about effective online pedagogies for specific learning outcomes Laurillard, D. (2012). Teaching as a Design Science: Building Pedagogical Patterns for Learning and Technology. New York and London: Routledge.

Tools for teachers as learning designers

Tools for teachers as designers - PPC A library of patterns to inspect

Tools for teachers as designers - PPC Each pattern is designed to deliver a specific type of learning outcome Colour-coded text identifies content parameters Black text expresses pedagogy design

Transferring the pedagogy across topics - PPC The Pedagogical Patterns Collector

Transferring the pedagogy across topics - PPC The Pedagogical Patterns Collector

Transferring the pedagogy across topics - PPC

Teacher adopts and adapts a design Add link to an OER, e. g. a digital tool for practice Specify duration of the activity in mins Read, Watch, Listen Inquire Discuss Practice Share Produce Adjust the type of learning activity. Edit the instructions. Export to Word [Moodle] Check the feedback on the overall distribution of learning activity Represent the teacher as present or not Fully specifies the pedagogy for others to adopt, adapt, test, and share

Design for Med students in PPC Explain how to optimise the inputs to a patient simulator to achieve the ideal blood pressure With your partner select different inputs to the patient simulator – can you improve on your previous results?

Export to Moodle for Med students

Re-design for Business students in PPC Explain how to optimise the inputs to a patient simulator to achieve the ideal blood pressure Explain how to optimise the inputs to a business model to achieve the optimal cash flow With your partner select different inputs to the business With your partner select different inputs to the patient model – can you improve on your previous results? simulator – can you improve on your previous results?

Export to to Moodle forfor Business students Export Moodle Med students The process of professional collaboration: Find effective patterns for a learning objective – adopt, adapt, improve – export to Moodle – test with students – improve – test - publish

Comments on the PPC • [The pie-chart] is one of the most useful features … it gives a good overview of the balance between different learning experiences • I rarely consider how the students' time is apportioned … it's good to be made to think about this. • Seeing how the sessions are shaping up in such a visual medium …. would probably make me think more carefully about providing a mix of activities

Modelling learning experience and teacher workload With a computational model, we can estimate the effects of the design decisions on quality of learning experience AND teaching cost: consequences for the pedagogical benefits comparative costs of teachers’ workload Then check actual outcomes using targeted learning analytics

Enhancement: Modelling the pedagogic benefits A computational representation can analyse how much of each activity has been designed in Categorised learning activities Conventional Acquisition Inquiry Discussion Practice Production Blended Acquisition Inquiry Discussion Practice Production Analysis shows more active learning

Efficiency: Modelling teacher time Learning activities for online courses Preparation time (fixed costs) • Guided TEL resources (model, AR) • Access to expositions (lecture videos) • Automated grading (MCQs, models) • Readings (pdfs) • Guided collaboration activities (wiki, AR) • Peer group discussion (forum) • Peer grading against criteria (share) • Tutored discussion (forum) • Tutor feedback (assignments) • Tutor-based assessment (exams) Support time (variable costs)

Modelling the teacher’s workload for increasing student cohort size Planning teacher time for a new online course at IOE 1400 1200 1000 800 600 400 200 0 0 Year 1 Year 2 Stable state Support Preparation Year 1 Year 2 Stable state 40 40 80 160 Students -£ 13000 -£ 600 £ 2000 -£ 13000 £ 19000 £ 61000 Profit/Loss

Modelling the teacher’s workload for increasing student cohort size From the Duke MOOC 3000 2500 Total teacher time in hours 2000 Time for Duke MOOC student Basic MOOC support 1500 1000 500 0 50 5000 Time for preparation of resources Cohort size Scaling up will never improve the student support costs… unless…

Modelling the costs for increasing student cohort size … we develop some clever pedagogical patterns that support students at better than the 25: 1 student: staff ratio The question is – what are they, and how do we develop and share them? Invest in the teachers who can innovate with learning technologies!

Further details… tinyurl. com/ppcollector Rethinking University Teaching: A Conversational Framework for the Effective Use of Learning Technologies (Routledge, 2002) (Chinese edition ECNU Press) Teaching as a Design Science: Building pedagogical patterns for learning and technology(Routledge, 2012) d. laurillard@ioe. ac. uk

Teaching as a design science: SUMMARY The global demand for HE requires investment in pedagogic innovation to deliver high quality at scale Technology-based pedagogic innovation must support students at a better than 25: 1 student: staff ratio Teachers need the tools to design, test, gather the evidence of what works, model benefits and costs Teachers are the engine of innovation – designing, testing, sharing their best pedagogic ideas

The Learning Designer A TLRP-TEL project The project partners Oxford Liz Masterman (Co. PI) Marion Manton (Co. PI) Joanna Wild (RF) IOE/LKL Birkbeck/LKL George Magooulas (Co. PI) Patricia Charlton Dionisis Dimakopoulos Brock Craft (RF) Diana Laurillard (PI) Dejan Ljubojevic (RF) London. Met Tom Boyle (Co. PI) RVC ALT Seb Schmoller Rachel Harris LSE Steve Ryan (Co. PI) Ed Whitley Roser Pujadas (Ph. D Student) Project website at www. ldse. org. uk PPC at web. lkldev. ioe. ac. uk/PPC/live/ODC. html Kim Whittlestone (Co. PI) Stephen May Carrie Roder (Ph. D Student)