Learning Gain Project Dr Stephen Haddelsey Learning Gain

Learning Gain Project Dr Stephen Haddelsey Learning Gain Project Manager Dr Linda Speight Learning Gain Data Analyst

The Lincoln-Huddersfield Learning Gain Project Ø Three-year, longitudinal, mixed methods study Ø Focus on the development of ‘employment competencies’ Ø Commenced November 2015 Ø Led by the University of Lincoln working in partnership with the University of Huddersfield

Methodology Ø Student Self-Assessment (SSA): in-house development (duration: 10 minutes) Ø Situational Judgement Test (SJT): A&DC psychometric test (duration: 30 minutes) Ø Additional sources: Ø Progression data Ø Attendance Ø Student engagement data: clubs, volunteering, job experience, democracy Ø Student interviews and focus groups

Status Ø 272 Level 1 students (2015 intake) at Lincoln completed one, or both, exercises and gave consent (47% of the target group) Ø Of these, 198 students have completed a second administration (73% of the original cohort) Ø 397 Level 1 students (2016) have also completed one or both exercises (60% of the target group) Ø A second administration is planned for September 2017

What was the…? 8% st Percentage of 1 year students who scored ‘above average’ on the SJT 38% st Percentage of 1 year students’ comments which identified ‘academic activities’ as helping improve skills Communicating & Influencing st Weakest skill set identified by 1 year students in the SJT 34% * st Percentage of 1 year students involved in sports or societies Confidence rd Key thing that 3 year students felt had improved at university? 11% * nd Percentage of 2 year students involved in volunteering * percentages are based on % of learning gain cohort and may not represent the University of Lincoln as a whole

The Issue of Student Engagement Ø Poor engagement has necessitated extended administrations and introduction of incentives Ø Students are taking ~20 minutes to complete the SJT (33% less time than the providers suggest) Ø 39% of students took <20 minutes to complete Ø Integrity of the SJT results are, therefore, questionable Ø These findings reflect local anecdotal concern that students do not properly engage with employment-related activities until their 3 rd year of undergraduate study

Next Steps Ø Continue annual administration of Student Self. Assessment Survey Ø Develop Student Profiles (progression data, test results, student engagement data etc) Ø Collect and incorporate additional data, including: Ø Pre-enrolment survey (Get Set) Ø Qualitative interview data Ø Contribute to body of knowledge on learning gain

Get Survey, 2016 • Designed to support TEF by establishing Level 1 students’ self-perception against a range of skills & competencies • Intended to inform personal tutoring conversations and to identify development needs at a corporate / school / subject level • 1865 Level 1 students completed the survey in August / September 2016 (50% of intake) But…. • Reporting was limited to exported graphs from Survey Monkey, manually collated and emailed to relevant staff • Difficult to obtain an overview of the data, compare between colleges, or for personal tutors to access individual reports unless initiated by the student • Results not used?

Get Survey, 2017 • DVC decision to roll-out survey to all students (still on a voluntary basis): • • • Survey Monkey replaced by Qualtrics: • • • Now incorporates questions around student expectations, adding value and utilisation potential Response options changed from a four point to a five point scale. Careers questions changed to directly match those used by the Career Service Group Status (at 11 September 2017): • • • . More professional in appearance Works across computer and mobile platforms Improved reporting functionality Easy to export data Question set: • • Continues to support TEF Provides additional data for inclusion in Learning Gain Project reporting 1926 L 1 completions 90 L 2 -4 completions Administration scheduled to close 30 October 2017

Get Set 2017 Question Set (1 st Yr)

Get Set Reporting We see two potential benefits of the Get Set data: 1. Improving tutorial support 2. Improving teaching and learning within schools • W hat the uses • W dat do h you • W o sho a? see hat uld • W for w hen ould rece i rec woul you ve r eiv e d e i you like ports t? wan to s ? t t ee? o • Data available ASAP • Data at school level • Tabular data • Trackable data

Contact details: Dr. Stephen Haddelsey Dr. Linda Speight Telephone: 01522 88 6034 Email: shaddelsey@lincoln. ac. uk Telephone: 01522 83 7077 Email: lspeight@lincoln. ac. uk