th 15 ViceChancellors Annual Teaching and Learning Conference

th 15 Vice-Chancellor's Annual Teaching and Learning Conference Enhancing the Student Experience: Data in Action

Enhancing the Student Experience: Data in Action Professor Mark Stubbs | @thestubbs Head of Learning & Research Technologies Context > Journey > Outcomes > Reflection RT ! SPOILER ALE Story of ity cross-univers collaboration

36, 000+ students 1, 000+ courses 6 faculties 2 campuses Context > Journey > Outcomes > Reflection

University-wide Transformation Programme Data to Improve Data to Personalise 2010 PHASE 2 Systematic Continuous t Improvemen PHASE 1 Coordinated Step-change t Improvemen Context > Journey > Outcomes > Reflection

An Integrated Response to Improving the Student Experience NSS Analysis, Focus Groups, Surveys n #1 Data in Actio rgeting a t s n a e m n Satisfactio ors’ ‘hygiene fact

Enhancing Quality and Assessment for Learning (EQAL) • A coordinated ‘strike’ for step-change improvement In the current climate New Curriculum • Designing new modules, … New Admin Systems & Processes • but Diminishing unit of resource but Everything depends on everything else Personal timetabling, … but New Digital Learning Environment • but We are large and risk averse Moodle, Campus. M, Talis Aspire… New QA & QE Processes • but Facilitating curriculum transformation Board Programme VC chaired by D

Rules for a New Curriculum • Different module credits • Less summative, more formative assessment • Fewer but clearer learning outcomes • Constructive alignment of learning outcomes & assessment • Single source of truth for weightings, deadlines … • Consistent module identifier for all systems 800 new module s to design & approv e each year! ule data Definitive mod using captured once spec form ‘lean’ module Every UG module needed to be re-developed! FOUNDA TION FOR DATADRIVEN I MPROVE MENT: CONSISTE NT IDENT IFIERS

Digital | Balancing Consistency with Ownership • Learning Technology Review (Mar 2010) • Migrate from Web. CT to “Core+ VLE” (Moodle) for all modules & progs • Joined-up, personalised, consistent Recognised: watch & learn how staff use tech Recommended: staff set up & we train to integrate Arranged: we set up institutional accounts & train staff to use Core: we set up & train staff and tell students to expect a consistent ‘front door’

Digital | Wrapping the University around the Learner 50 m hits from 40 k users s 67% off-campu 28% at night moodle. mmu. ac. uk megamashup mws ent Driven by Stud Records System Joined up by consistent codes Free App 29 k registered k 17 k+ active/w T 94% Moodle+T mymmu-mobile

Digital | Integrating Cloud Services using Consistent Identifiers Module code rmed List Policy info ps by Focus Grou Resource list List Policy reflect ed in Buy, Essential & Further tags Tags used to ts for Summarise Lis Learners Relevant resource

Outcomes | Improving NSS Performance NSS: OVERALL NSS: ASSESSMENT

Improving the Assessment Lifecycle Define assessment types & weightings in SRS Data to Improve Record deadlines / return dates in SRS Create & tag VLE assignments with SRS assessment identifier Publish assessment analytics Data to Personalise Publish submission deadlines / feedback return dates in VLE Publish provisional marks in VLE Transfer marks electronically from VLE into SRS Submit VLE assignments Mark & annotate VLE submissions Track VLE submissions ctronic See Jisc’s Ele t of Managemen uide Assessment G

University-wide Transformation Programme Data to Personalise Refresh Curriculum Refresh Admin Processes Wrap University Around Learner 2014 PHASE 1 Coordinated Step-change t Improvemen Listen to Student Voice Be Bold & Joined-up Balance Consistency & Ownership Address Dissatisfaction PHASE 2 Systematic Continuous t Improvemen

Systematic Continuous Improvement • Embed Student Voice • MMUnion • Investment in Reps • Training & Publicity • Internal Student Survey (ISS) • • • Delivered twice a year via Student Portal Single programme & module survey personalised according to student enrolments Questions derived from NSS Closed questions minimised to maximise free-text feedback ‘Grounded Theory’ style qualitative analysis to target improvements in • Technology-Enhanced Learning and Assessment & Feedback • Continuous Monitoring & Improvement (CMI) • Expectation of response to ISS, NSS, Reps built in to CMI process

Systematic Continuous Improvement • ‘Grounded Theory’ style qualitative analysis to target improvements • Relative frequency of mentions of TEL in best things about course/modules by department Departments Qualitative by coding done TEL Advisors

Systematic Continuous Improvement • Data Warehouse (est. 2011) • Microsoft Business Intelligence • SRS, VLE, Submission, Attendance & Survey data • 40 -50 million rows in main fact tables • Dashboards • Heads, Programme & Module Leads • Personal Tutors • Challenges • • Data Quality & Ownership Meeting Demand with Scarce Expertise Verifying Accuracy of Novel Insights Embedding in standard processes

Systematic Continuous Improvement • Dashboards for Heads, Programme & Module Leads • • • Students Progression Marks Satisfaction Comments Action Plans University-wide: every Programme every Module wn o d l l i r D , s d Tren & Search

Systematic Continuous Improvement • Dashboards for Programme & Module Leads Visualise cohort make-up

Systematic Continuous Improvement • Dashboards for Programme & Module Leads Explore anonymised freetext comments cross-referenced with scores

Dashboard for Personal Tutors Student Enrolments Moodle Usage Submissions & Marks Visual overview updated weekly for Personal Tutors & Programme Leaders Data Warehouse Attendance w Weekly lists of those belo engagement thresholds reviewed for follow-up emails & appointments

Directed Independent Learning Tracker App Working on for & with s our Student

University-wide Transformation Programme | Reflection Data to Improve Cross-university collaboration & commitment from highest level Data to Personalise 2010 PHASE 1 Coordinated Step-change t Improvemen Listen to Student Voice Be Bold & Joined-up Balance Consistency & Ownership Address Dissatisfaction PHASE 2 Systematic Continuous t Improvemen Embed Student Voice Empower Programme & Module Leads with Actionable Learner Analytics Empower Students & Personal Tutors with Actionable Learning Analytics

Enhancing the Student Experience: Data in Action Thanks for listening @thestubbs