SPI Administrators Forum Thursday 5 July 2018 What

SPI Administrators’ Forum Thursday 5 July 2018 What three words would you use to describe what the Student Personalised Information Programme is trying to achieve? Please take a post-it and put your words on one of the sheets on the tables.

Welcome • Opportunity to share updates on all student systems and processes across the University • Open to all administrative staff; departmental and central service • Series of short 10 minute updates, and demonstrations

Agenda • Study Abroad Amanda Osborne • Skills. Forge Tracy Horton • Learner Analytics Russell Boyatt • Module Evaluation Amber Thomas • Student Records Workflows Derek Brandist & Emma Ford • Warwick Welcome Chris Luck • SPI Gap Analysis Emma Melia • Review of Assessment Claudia Gray • BREAK • My. Warwick, Module Approval & Scholarships Updates Roland Ingram

Study Abroad Amanda Osborne Study Abroad Manager

SCD Project Integration and automation of Study Abroad processes into SITS Study Abroad Team Amanda OSBORNE

WHY? Study Abroad is identified within the key University strategy and is seen as an integral part of the University of Warwick offer Strategy Partnerships Student preferences Brexit Resources CHALLENGES Duration Study-Work Processes

HOW?

WHEN?

Skills. Forge Tracy Horton PGR Professional Development Officer

• https: //warwick. ac. uk/services/academicoffice/gsp/professionaldevelopment/pgrdevelopment/devs

Feedback Questions in Skills. Forge – looking at quality, relevance and offerings of provisions across the institution Doctoral Skills Programmes Request – for PGRs already doing full ten days of framework Booked events in my. Warwick App – via teaching timetable Professional Development Award – canvassing PGR opinions • https: //warwick. ac. uk/services/academicoffice/gsp/professionaldevelopment/pgrdevelopment/devs

Learning Analytics Russell Boyatt Service Owner (Learning Support Services)

Learning Analytics • "Measurement, collection, analysis and reporting of data about learners" • Understanding student behaviour and optimising the learning environment.

• Identifying students 'at risk'. Learning Analytics • Giving students the tool to understand their own behaviour, attainment and engagement. • Understanding how course and module resources are used.

Other institutions - UK and internationally • UK - Nottingham Trent, Open University, Edinburgh, Exeter • Europe – national projects in Finland • US – Purdue University Course Signals

• Student workshops WIHEA funded project to organise student workshops. • Assessing student attitudes and perspectives on use of data in education: • Many students say they can't assess their progress. • Existing nudges, such as coursework reminders, don't always have a positive impact. • Concern raised about how we use data – even when they can see the benefits.

• At the moment, a discovery and analysis phase. What's next? • Looking ahead: • Improvements to student dashboards – helping student assess progress. • Establishing institutional policy and ethical guidelines. • If we identified 'at risk' students – what would we do? • Looking for ways the data can help academic departments. Interested? Let's talk!

Student Records Workflows Derek Brandist & Emma Ford Administrative Officers, Student Records Management

STUDENT RECORDS REQUESTS LAUNCH: OCTOBER 2018 E: VISION PG Extensions Change of Study Location Course Transfer Authorised Absence Restart RP(E)L Request Suspension of Study Resumption Mode of Attendance PG Upgrades Permanent Withdrawal Voluntary Year Out

• Same system as Temporary withdrawal requests, introduced last year • Efficient – It’s an online version of the current form that instantly reaches the next stage. • Auditable - Can keep an audit trail of documents & messages all in one place. The request can be tracked by staff & the student • Consistent across all Departments • Supported - Guidance documents will be available for each process • GDPR Compliant Student (or Staff) Department Check Department Support Supervisor (PGR) Academic Office Student Dept Check Student

Next Stages: • Departmental Review & Testing • 2 nd July – 11 th July • Departmental Training Sessions • August & September • Launch • October 2018

SPI Gap Analysis Emma Melia SPI Programme Manager

SPI Scope & Workstreams • Initial Programme Workstreams set in May 2017 • Since SPI began: • • Education Strategy Development TEF results Four new Academic Directors on Education Executive Key policy work: Review of Assessment, Module Evaluation… Emergence of alternative pathways – Degree Apprenticeships Opportunities identified to work with academic departments Projects initiated and some areas completed • Time to review & prioritise next phase

2017/18 2018/19 2019/20 2020/21 2021/22 Admissions & Arrivals Warwick Welcome Systems Implementation Universal Admissions Tailored Enrolment Processes (inc. UKVI Requirements, GDPR Compliance) Enquiry management for prospective students Critical Infrastructure Module Approval & Catalogue / Course Approval Exams, Marks & Assessment Mgmt (inc. Ext. Examiners Workflow & PGR) External Reporting (inc. HESA Data Futures) Learner Analytics Student Records System Improvements (inc. Graduate School Processes) Student Finance Infrastructure Timetabling System Replacement Exams Timetabling Student Interface VLE, Reading Lists, Assignment Submission & Study Materials Consistency My Warwick App & Student Comms Channels, (Handbooks, Personalised Comms) Module Selection & Module Evaluation Learner Analytics Student Interface (inc. Attendance Monitoring) Employability, Skills & Transition Transferable Skills Access & Tracking & Careers Portal Student Opportunities: Placements, Year Abroad, Employability, Volunteering, Research, Representation etc. HEAR Improvements & PG Student to Alumni Transition & Alumni Mgmt Student Research Opportunities Access Welfare & Support Cheating & Plagiarism Wellbeing, Complaint & Appeal Case Mgmt Scholarships & Bursaries Mitigating Circumstances Student Query Mgmt (inc. Student Funding, Graduation, etc. ) Tailored Interventions based on Learner Analytics Alternative Pathways Degree Apprenticeships, Work-Based Learning, Distance Learning Infrastructure & IFP

Gap Analysis • Which additional areas of development do you think should be considered for the scope of SPI? • • Student Experience Underpinning infrastructure Compliance Efficiency • Don’t worry if you’re not sure – write it down! • We will review and look for a fit • Think about the elements of the student journey involved • We are exploring channels for gaps/requests in the future Enquiry Application Arrival Study & Student Life Exams & Assessment Leaving Alumni

Break Please help yourself to refreshments

Reading Lists 2018/19 Please send your 2018/19 reading lists to the Library. The sooner you are able to work on your lists and send them to the Library, the more likely we are to be able to ensure the resources are in place in October. • https: //warwick. ac. uk/services/library/staff/reading-lists/ • Contact the Reading List Team readinglists. library@warwick. ac. uk or your Academic Support Librarian https: //warwick. ac. uk/services/library/subjects/academic-support-librarians for further help and advice.

Module Evaluation Amber Thomas Head of Academic Technology Team, ITS

Module Evaluation Review Important aspect of quality enhancement: have robust feedback loops and to be able to use that data in future Subject-level TEFs AQSC Principles 2014 > SLEEC Review 17/18 > governance decisions > core questions and new supported service for 18/19 Consultation: open forums, WIHEA learning circle, specific work on data management and technology approach including a survey and a cross-department technology discussion Explored many issues including: • purpose: module evaluation processes should be about student engagement and teaching enhancement, not individual performance management • mode: paper vs online: online done right can get good response rates, and is more efficient • process: there is huge variation between departmental practices and no single overarching process from survey design to delivery, access, interpretation and access In order to improve practices across the institution we need to introduce some consistency, through core questions and a central supported service We need to increase consistency of process and data across departments, in an efficient way that supports good practice

Module Evaluation Questions Aim: Increase consistency of process and data across departments, in an efficient way that supports good practice Six core questions, but departments can add more All departments are strongly encouraged to use core questions, whatever tool you use Open question: Please name the one thing in the module which have the most impact on your learning 1. The module content is delivered in an engaging way 2. Feedback (on work, in class, or other forms) received on the module enhances my learning 3. The module is well organised 4. The module offers an appropriate level of intellectual challenge 5. Appropriate support is available to me throughout the module [The above will be on a Likert Scale] <Space to add departmental questions> https: //warwick. ac. uk/fac/cross_fac/academy/activities/learningcircles/moduleevaluation/proposedquestions/

Module Evaluation Service Aim: Increase consistency of process and data across departments, in an efficient way that supports good practice Central Module Feedback Survey service offering one more year of paper forms, with initial minimal viable service for online forms starting 18/19 – Service will grow as practice matures – Online forms in Moodle (questionnaire tool) – Exploring best deployment models – Intended for in-session on-screen completion – Students complete survey anonymously – Six core questions with ability to add more – Quick view results available in Moodle – Data service for departmental aggregated and advanced reports (building on existing Paper service) Guidance will be available and iteratively enhanced Tools and service will be reviewed during 18/19

Online Survey Preview Mobile (landscape) Anonymous Intended for on-screen in-session use Option to email non-respondents Laptop/PC

Online Quick View Preview Available only to staff with permissions for the module Free text comments Average rating and percentage of each response per question

Reporting The Student Module Feedback Service will then work with departments to store, aggregate and report on their data, through an enhanced data service This will be aggregated with existing data from the paper scanning service We will be working with you to design and deliver the reports you need and we expect requirements to develop over time

Summary For 2018/19 AY • implement core question set • consider piloting the Service • use opportunities to feed back / review For 2019/20 AY • continue core question set • use the Service unless otherwise agreed

Warwick Welcome Chris Luck Head of Welcome and Student Internationalisation

Review of Assessment Claudia Gray Assistant Registrar, Teaching Quality

Review of Assessment – Background Academic Quality and Standards Committee commissioned “Review of Assessment” in June 2017: Growth in number of examinations (37% increase from 2006/7 to 2016/17) Growth in numbers of students presenting with mitigating circumstances or requiring special exam arrangements HEFCE revised operating model for quality assessment 2015/16, providers required to monitor academic standards and use of data, analysis of trend data Concerns expressed by External Examiners ITLR recommendations re assessment: Overassessment

Review of Assessment – Background Lack of advice and guidance to support diversification of assessment methods Timetabling of exams Individual examination arrangements Assessment Regulations disparate and difficult to navigate and lacking clarity

Review of Assessment Group Led by Deputy Pro-Vice Chancellor (Education) Academic Directors (UG, PG and WIHEA) Academic representation from all four Faculties (at least two members, normally DUGS, DGS) Professional Services Staff from departments, Teaching Quality, Academic Office, Examinations Office, LDC, IT Services, SPA, Space Management and Timetabling Two student representatives: Education and Postgraduate Officer from SU Business in four sub-groups, members nominated by HODs and drawn from the “Review of Assessment Group”

FOUR SUB-GROUPS Progression, Remedying Failure and Timing of Examinations (Dr Phil Young) Examination Board Procedures (Professor Jeremy Smith) Mitigating Circumstances, Reasonable Adjustments (Professor Andy Clark) Assessment Strategies (Professor Gwen van der Velden)

Progression, Remedying Failure and Timing of Exams (Dr Phil Young) PROGRESSION REQUIREMENTS - UNDERGRADUATE STUDENTS Currently, Year 1 to Year 2: 80 credits and 40% overall mark, intermediate year: 60 credits Too low and not in line with sector Proposal endorsed by AQSC on 29 May 2018: at least 90 credits including core modules and overall mark of 40%, approved exit qualifications (90 Cert. HE and 180 Dip. HE) 168 credits for 3 year courses (258 for IM courses) required to be classified, in future 180/270 credits Modelling: slight increase in re-sits, 2 -3%

Progression, Remedying Failure and Timing of Exams (Dr Phil Young) REMEDYING FAILURE: • • RESITS in all modules at EARLIEST OPPORTUNITY (UG: at least 90 credits including core modules for progression to year 2 and 3, 120 credits optional to demonstrate a pass) PGT requirements unchanged FORMAT OF RE-SITS (UG and PGT) for modules with multiple components as set out on MA 1: Either as original assessment and re-sits in all failed components required or Re-sit in one assessment only (exam or assessment) provided learning outcomes of module are met Re-sists set at the same time as first sits to ensure external scrutiny Further discussions ongoing to decide if re-sits for final year students be offered

Progression, Remedying Failure and Timing of Exams (Dr Phil Young) TIMING OF EXAMINATIONS: AQSC endorsement of official January examination to be introduced for all departments in principle Current September re-sit period too late and clashes with Welcome Week Most HEIs in UK offer August re-sit examinations Re-sits to take place when most conducive to improve student experience and resources to be made available to facilitate this Further discussions with about timings, practicalities and logistics before final recommendations are made

Examination Board Procedures (Professor Jeremy Smith) REMIT OF GROUP: Review of departmental examination board procedures Review of IT systems underpinning exam board preparation, merits of developing SITS and horizon scanning Consider standard paperwork for consideration by Boards of Examiners Classification of borderline students Consider establishing standard guidelines on how departments should involve external examiners in assessment process

Examination Board Procedures (Professor Jeremy Smith) Proposals endorsed by AQSC 29 May 2018: STANDARD IT SYSTEM: One standard IT system should be used across University to store marks and produce exam board paperwork, further work ongoing to decide which IT system to be implemented • WBS demo • Further demos expected from Universities using SITS and other exam board administration systems

Examination Board Procedures (Professor Jeremy Smith) Standard data set for exam boards (UG and PGT) from 2019/20: Compulsory: Name of programme Academic Year started their degree Student Identification Number Module Number Total Mark Breakdown of Exam and Assessment Marks Number of CATS Mean Mark for the Year Weighting of Marks Mitigating Circumstances Recommendation of the Board Core & Optional Module Decision of the Board % CATS passed

Examination Board Procedures (Professor Jeremy Smith) Classification of Borderline students (UG): Proposal for standard rules to be introduced from 2019/20: Agreed algorithm if weighted average for classification is within 2% of borderline and promote if: (i) More than 50% of weighted CATS counting towards final classification above class boundary or (ii) Exactly 50% of weighted CATS above class boundary, then either average performance in final year above class boundary or average performance in core modules above class boundary Action: Modelling of UG borderline algorithm PGT: Borderline zone 1% or 2%, more research across sector needed

Mitigating Circumstances – Reasonable Adjustments (Professor Andy Clark) Remit of Group: Review of Mitigating Circumstances Policy in view of feedback from External Examiners, departments and AQSC Consideration to introduce a Fit to Sit and/or deferral of examination policy Review complexities and resource implications when implementing reasonable adjustments for students with disabilities Review Self-Certification policy

Mitigating Circumstances – Reasonable Adjustments (Professor Andy Clark) Proposals endorsed by AQSC on 29 May 2018: Definitions of Mitigating Circumstances and Reasonable Adjustments Introduce extensive guidance to students and staff in relation to mitigating circumstances New Mitigating Circumstances Policy for 2018/19 Introduce an IT portal to collect mitigating circumstances and reasonable adjustments in one place from 2019/20 onwards, GDPR compliant and linking to exam board administration IT system

Mitigating Circumstances – Reasonable Adjustments (Professor Andy Clark) Currently self certification for assessed work less than 10% on no more than two occasions (Section F, S & D conventions) Proposal for future introduction 2019/20: One self-certification request per academic year for an extension to deadline assessed work (e. g. essays, lab reports, posters, dissertations and any other assessed work, but not exams), consulting with departments New option of deferral of examinations for serious mitigating circumstances, further discussions needed with departments Further discussions to create a reasonable adjustments policy, on practical arrangements in departments and recommendations from support services

Assessment Strategies – Professor Gwen van der Velden Remit of Group: Simplification of assessment strategies at the level of the module and course (including student choice of assessment) Broadening/diversification of types of assessment and identifying strategies at the departmental level including seeking advice from LDC and WIHEA on development of assessment strategies (reduce examinations and avoid plagiarism) Development of guidance on assessment design

Assessment Strategies – Professor Gwen van der Velden Discussions so far: Review of literature to produce a good practice guide on principles of good assessment including the constructive alignments of methods, learning outcomes and assessment criteria, diversity of assessment and good practice on assessment and feedback Considered research on student choice of assessment Considered research from the Faculty of Social Sciences on module load and tariff

Assessment Strategies – Professor Gwen van der Velden Agreement: Agreed that student effort on module assessment should be measured in time rather than assessment load and indicative table of how much time spent on assessment might be useful for some students Ongoing work on publication of good principles of assessment and student choice of assessment

My. Warwick, Module Approval, & Scholarships Updates Roland Ingram Process Owner, SPI

MY WARWICK APP App Usage • 24, 000 downloads • 16, 000 regular users • 90% of Y 1 undergraduates

MY WARWICK APP New Features • Study Spaces tile • Shuttlebus tile • O 365 To-Do tile

MY WARWICK APP Development Approach • New feature suggestions • In-app feedback, App Store reviews, focus groups • App Steering Group review • User testing Release

MY WARWICK APP Development Approach • New feature suggestions • In-app feedback, App Store reviews, focus groups • App Steering Group review • User testing Release

MODULE APPROVAL Overview • Change from paper form to online system • Eliminate repeated data entry • Improve module catalogue data • Improve compliance

MODULE APPROVAL Progress So Far • New module data set approved by AQSC and Senate • Project Manager and Business Analyst assigned • Business Case approved by SPI Steering Group

MODULE APPROVAL Next Steps • Form Project Board and Delivery Team • Develop Requirements Catalogue • Select solution • Begin development

MODULE APPROVAL Timeframe • Pilot in 2018 • Release by mid 2019 • …resource/supplier permitting

MODULE APPROVAL Further Info • Project web page: warwick. ac. uk/services/spd/spi/map/

SCHOLARSHIPS Overview • Universal Admissions will replace the course application system • Need to update the scholarship application system • …in the Graduate School

SCHOLARSHIPS Overview • Universal Admissions will replace the course application system • Need to update the scholarship application system • …in the Graduate School

SCHOLARSHIPS Two Stages • Universal Admissions pilot in September 2018 • Universal Admissions launch in September 2019

SCHOLARSHIPS UA Pilot September 2018 • Limited number of PGR courses affected • Will use existing systems and adjust process

SCHOLARSHIPS UA Pilot September 2018 So far: • As-is process mapped • To-be process planned • Working with Grad School, IT Services, SITS team, UA team

SCHOLARSHIPS UA Pilot September 2018 Next: • Document to-be process • Test process • Led by Emma Morrison

SCHOLARSHIPS UA Launch September 2019 • All PGR courses affected • Process replacement not scalable • Looking at system replacement (replacing BPM)

SCHOLARSHIPS UA Launch September 2019 So far: • High-level requirements known • Looking at existing SITS capability • Priority: continue PGR scholarship applications under UA

SCHOLARSHIPS UA Launch September 2019 Next: • Look into connections with SCD scholarships work • Gather detailed requirements • Research solutions