ACHIEVING POSITIVE OUTCOMES FOR ALL What Can We

ACHIEVING POSITIVE OUTCOMES FOR ALL What Can We Do And Why Does It Matter? Dr Helen May OU e. STEe. M Conference, 30 th April 2020

Definitions OVERVIEW External context What do we mean by positive outcomes? How has the landscape shifted in the last 15 years? What do universities need to evidence in relation to inclusion? What are your claims for excellence? Implications How can you contribute to the evidence base? How do you use the evidence? Conclusions What are the key messages?

DEFINITIONS

HOW DO YOU DEFINE…. . Positive Outcomes for All

DEFINING POSITIVE OUTCOMES FOR ALL All Positiv e Outcomes Positive Outcomes All Achieving potential Success Everyone Being successful Attainment /classification Inclusive High quality Employability or progression Students as individuals Desired attributes Graduate salary /income Students as groups Value added or progress made Confidence, identity, esteem

EXTERNAL CONTEXT HOW HAVE THE PRIORITIES SHIFTED OVER TIME? HOW DOES INCLUSION ALIGN TO TEF AND APP?

SHIFTING PRIORITIES 2005 2010 2020 Equity of opportunity Equity of the institutional culture Equity of student outcome Emphasis on fair access and participation Emphasis on inclusive learning and teaching, retention and student engagement Emphasis on steps taken to maximize and evidence impact

HOW DOES INCLUSION EMERGE IN THE TEF? Criterion 1: Teaching Quality • • Your subject approach is tailored to your students All students are engaged and being stretched Quality is consistent Representative/collaborative work is representative Criterion 3: Learning Outcomes /Gain Criterion 2: Learning Environ • • • Equitable access & take up of learning resources Personalise learning Additional support in place for those who require it • • • Targeting and supporting those at risk of unfavourable outcomes All students have positive outcomes Flexibility to facilitate extra -curricula engagement

ACCESS AND PARTICIPATION PLAN: ACCESS, CONTINUATION, ACHIEVEMENT AND PROGRESSION Assessment of performance for each underrepresented group Identify any gaps in performance Set targets for each group Use theory of change to plan how you will meet targets Identify evaluation strategy Report on progress

PLANNING FOR IMPACT…. Impact Outcomes Outputs Activities • On students • On policy • On practice • Benefits to target groups • Tangible deliverables • Planned products or services Used as a planning tool Change Narrative

EVIDENCING IS A JOINT RESPONSIBILITY • IDENTIFY YOUR IMPLICATIONS FOR PRACTICE CLAIMS • CONTRIBUTE TO THE EVIDENCE BASE • EVIDENCE INFORMED PRACTICE

It was required It was unique WHAT IS YOUR CLAIM FOR EXCELLENCE? It was beneficial It had a wide reach It added value It was effective It led to growth or improvement over time It made a difference

Types of data DIFFERENT EVIDENCE SOURCES: Source: Adapted from May, H. (2019) TEF Guiding Principles York St John University Forms of Narrative and Evidence Validated Performance against external benchmarks e. g. metrics or peer-referenced data Longitudinal Improvements or growth over time e. g. investment, demand, engagement, performance, continuation Comparative Performance relative to others or time period e. g. student groups, disciplines, other institutions, other countries Impact Had a transformative effect on society, subject, students, community, employers – indicating what changed Return on investment Investments have been worthwhile e. g. of resources, estates, IT infrastructure, visits, tutoring, training Reputational Performance has been rated as having a high value e. g. awards, nominations, repeated invitations,

IDENTIFYING YOUR CLAIM: ‘BEYOND THE METRICS’ Sense of belonging/part of an (academic) community Building effective relationships The wider transformational impact of a university education on students Wellbeing of students and staff Student engagement Robertson, Cleaver & Smart (2019)

HOW COULD YOU CONTRIBUTION TO EVIDENCE BASE? Systematic sources of evidence: Scholarship Student reflective assignments Inform students what has changed Increase response rates (e. g. module evaluations) Engage students in selecting what is changed Ask students what they would like to ‘Stop start continue’ Reference requests External examiner reports Validation commendations Partner testimonials Other? . . .

HOW ARE YOU USING YOUR STUDENT DATA TO INFORM PRACTICE? Are you using data to monitor your students? Are you using the data to inform your practice? Can you evidence the difference this has made? Are you continually adjusting and personalising your approach?

MEASURES OF STUDENT ENGAGEMENT Students’ feelings/ satisfaction/emotions Levels of confidence Levels of resilience Interests Attitudes Values held Levels of curiosity Self-efficacy Wellbeing Affective Behaviou r • • • Cognitive Adapted from Evaluation of the Of. S Learning Gain Pilot Projects (Kandiko Howson, 2019) E-resource/e-book access Assignment submissions Moodle forum posts Moodle quiz submissions Moodle page views Moodle logins Library usage Also Levels of effort Persistence Asking questions Seeking help Participation in discussion Participation in extra-curricular activity Cognitive gain Critical reasoning Situational judgement Levels of surface and deep learning Higher order understanding Solving of challenging problems

KEY MESSAGES Discuss APP and TEF in your subject Identify your claim for excellence Routinely discuss and identify impact ‘It is not about doing different things, it is about doing things differently. ’ (May and Bridger, 2010) Use student data to inform decisions Utilise other relevant systematic data to support claims Ensure evidencing impact is a joint responsibility

ANY QUESTIONS /COMMENTS

Kandiko Howson, C. (2019) Final Evaluation of the Office for Students Learning Gain Pilot Projects July 2019 Report to the Of. S by King’s College London https: //www. officeforstudents. org. uk/media/20 ffe 802 -9482 -4 f 55 b 5 a 0 -6 c 18 ee 4 e 01 b 1/learning-gain-project-final-evaluation. pdf May, H. (2019) Teaching Excellence and Student Outcomes Framework (TEF) Guiding Principles: for subject and provider narratives. York St John University REFERENCES May, H and Bridger, K. (2010) Developing and embedding inclusive policy and practice in higher education. York: Higher Education Academy https: //www. advance-he. ac. uk/knowledgehub/developing-and-embedding-inclusive-policy-and-practicehigher-education Robertson, A. , Cleaver, E. , & Smart, F. (2019). Beyond the metrics: identifying, evidencing and enhancing the less tangible assets of higher education. QAA Scotland. https: //www. enhancementthemes. ac. uk/currentenhancement-theme/defining-and-capturing-evidence/theintangibles-beyond-the-metrics Introduction to theory of change https: //www. youtube. com/watch? v=g. Akajtm. Yn. Ng