Inclusive Assessment Strategies Linking assessment and feedback to

Inclusive Assessment Strategies Linking assessment and feedback to how students really learn; Making assessment work better, and giving more students better feedback in less time! Dundalk Institute 25 th October 2013 Phil Race Follow Phil on Twitter: @Race. Phil (from Newcastle-upon-Tyne) BSc Ph. D PGCE FCIPD PFHEA NTF Visiting Professor: University of Plymouth and University Campus Suffolk (Adjunct Professor; James Cook University, and Central Queensland University) Emeritus Professor, Leeds Metropolitan University www. phil-race. co. uk

This slide has two main purposes…. To focus the data projector, when necessary, and…. . . to persuade colleagues never to put this sort of detail onto a slide! www. phil-race. co. uk

About Phil… v Born a Geordie v First a musician v Then a writer v Then a scientist v Then a researcher v And now ‘retired’! (1995, 2009) v Working with students on learning techniques v And lecturers on teaching and assessment strategies v Then a lecturer and warden v And trainers on training design v And how we teach them v Based at Newcastle, UK v Got interested in how students Currently… v Visiting Prof: Plymouth learn v Emeritus Prof: Leeds Met v And the effects assessment and feedback have on them v Travelling around as usual! v Gradually became an educational developer v And on trains v And still a Geordie. And an expert… on train routes and timetables!

You will be able to download my main slides v You don’t need to take notes. v I’ll put the main slides up on my website before the day is out, so sometimes I may go through them very fast! v You may also find it useful to come to the session Sally and I are running in Room M 128, Nursing Building, on ‘Inclusive Teaching: The Big Issues’ from 1230 -1330, or to download the slides from this session, which go into more detail of some of the things I will be covering with you this morning. www. phil-race. co. uk

Thanks to students v I could not do what I do nowadays if I did not continue to spend a fair bit of my time working with students – they’ve taught me most of what I know. www. phil-race. co. uk

Intended learning outcomes By the end of this module, you should be better-able to 1. Make assessment more inclusive. 2. Design inclusive assessment processes and instruments with due regard for achieving validity, reliability, transparency, authenticity, and manageability. www. phil-race. co. uk

What’s an ‘Emeritus Professor? ’ v Someone really old? v Someone who has not been sacked for gross moral turpitude? v Someone who’s still on the books, but not on the payroll? v Or perhaps as in ‘edentate’, the ‘e’ signifies ‘without’. v (C. f. e-learning? ). www. phil-race. co. uk

12 key sources about assessment, feedback and learning in higher education 1. Knight, P. and Yorke, M. (2003) Assessment, learning and employability Maidenhead, UK SRHE/Open University Press. 2. Bowl, M. (2003) Non-traditional entrants to higher education ‘they talk about people like me’ Stoke on Trent, UK, Trentham Books. 3. Flint, N. R. and Johnson, B. (2011) Towards fairer university assessment – recognising the concerns of students London: Routledge. 4. Gibbs, G. (2010) Using assessment to support student learning Leeds: Leeds Met Press. 5. Boud, D. and Associates (2010) Assessment 2020: seven propositions for assessment reform in higher education Sydney: Australian Learning and Teaching Council. 6. Joughin, G. (2010) A short guide to oral assessment Leeds: Leeds Met Press.

12 key sources about assessment, feedback and learning in higher education 7. Petty, G. (2009) Evidence-Based Teaching: A Practical Approach (Second Edition) Cheltenham UK: Nelson Thornes 8. Race, P. (2010) Making learning happen: 2 nd edition London: Sage Publications. 9. Blue Skies (2011) new thinking about the future of higher education: download free from http: //pearsonblueskies. com/ 10. Hunt, D. and Chalmers, L. (eds) (2012) University Teaching in Focus: a learning-centred approach see Chapter 5: Using effective assessment to promote learning, Sally Brown and Phil Race Australia: ACER Press, and London: Routledge. 11. Price, M. , Rust, C. , O’Donovan, B. and Handley, K. (2012) Assessment Literacy, Oxford: the Oxford Centre for Staff and Learning Development (based on the ‘ASKe’ Project). 12. Sambell, K. , Mc. Dowell, L. and Montgomery, C. (2013) Assessment for Learning in Higher Education, London: Routledge.

Much of this workshop is based on Making Learning Happen 2 nd edition Phil Race 2010, Sage, London

Chapter 8: Responding to Diversity and Widening Participation v I’ll put up Chapter 8 of the 2 nd edition for a short while as ‘handout 1’ on my website. Sadly, this Chapter won’t be in the 3 rd edition!! v This goes quite deeply into linking the seven factors underpinning successful learning to: 1. Dyslexia 2. Visual impairment 3. Auditory impairment 4. Mental health needs. www. phil-race. co. uk

Post-it exercise v On a post-it, in your best handwriting, please write your own completion of the starter: “Making assessment and feedback more inclusive would be much better for me if only I …” v Please swap post-its so that you’ve no idea who has yours. v If chosen, please read out with passion and drama the post-it you now have. v Please place them all on the chart as directed. www. phil-race. co. uk

Post-its… v A small, equal opportunities, non-threatening space. v Just about everyone is willing to jot something down on a post-it in answer to a question, whereas they may not offer a spoken answer to a question, or write responses on a blank sheet of paper. v Post-its allow everyone the same opportunity to respond, including the quiet or shy students. v Post-its can be swapped, and students can read out someone else’s ideas, in the relative comfort of anonymity. www. phil-race. co. uk

Finding out where a group is starting from… v Post-its are particularly useful for open-ended questions, such as ‘economics would be much better for me if only I …’ v Responses can be posted on a flipchart or wall, and used as an exhibit. v They can be photocopied and returned to students. v Post-its can be a fast way of finding out what the real intended learning outcomes are for a group. v They can also provide a measure of the learning incomes of the group. www. phil-race. co. uk

Face-to-face communication Making all the channels work 29 October 2020 15

Thirty Second Theatre 29 October 2020 16

Face-to-face one-to-one feedback activity Please work in pairs, moving around the room, talking to different people using the script which follows… The script: A ‘Hello’ (or equivalent). B ‘Hello’ (or equivalent). A ‘You are late’. B ‘I know’. Try to do it completely differently each time. www. phil-race. co. uk

The power of face-to-face communication v When explaining assessment criteria to students, and when linking these to evidence of achievement of the intended learning outcomes, we need to make the most of face-to-face whole group contexts and, , , v. Tone of voice v. Body language v. Facial expression v. Eye contact v. The chance to repeat things v. The chance to respond to puzzled looks v Some things can’t work nearly so well just on paper or on screens. www. phil-race. co. uk

Now is the decade of Post-compulsory Education’s dis-content! Never mind the content – feel the learning www. phil-race. co. uk

I predict a move of post-compulsory education providers away from being the guardians of content, (where everything was about ‘delivery’), towards two major functions: 1. Recognising and accrediting achievement, wherever learning has taken place (i. e. getting the assessment right); 2. Supporting student learning and engagement (not least by making feedback work well for students). www. phil-race. co. uk

So now, it’s time to re-think: 1. How students really learn – and how we can help make learning happen for them; 2. How to make feedback really work for students; 3. How best to measure and accredit students’ achievement (not just more of the same old tired methods). In this module, we’ll touch on all three of these areas. www. phil-race. co. uk

Geoff Petty, 2009 v Teaching is about to embark on a revolution and, like medicine, abandon custom and practice, and fashion and fads to become evidence–based. www. phil-race. co. uk

One of my main worries. . . v We still tend to try to measure. . . what’s in students’ heads, and what they can do with what’s there in terms of two unsatisfactory proxies. . . 1. what comes out of students’ pens in exams; 2. what comes out of their keyboards in essays and reports. But first, let’s think about how to help them get things firmly into their heads. www. phil-race. co. uk

Using what comes out of their pens? Using what they say? Using things they make? Using how they work with each other? Using how they assess each others’ learning? How can we assess what students have learned? Using how they do practical things? Using how they assess their own learning? Using how they solve problems? Using how they answer our questions? Using what they put online?

“[The] pupils got it all by heart; and, when Examination-time came, they wrote Active it down; and the Experimentation Examiner said ‘Beautiful! What depth!’ They became teachers in their turn, Abstract and they said all these Concrete things over Conceptualisation Experience again; and their pupils wrote it down, and the examiner accepted it; and nobody had the ghost of an idea Reflective what it meant” Observation Lewis Carroll, 1893

A question [30 marks] v Please develop an annotated bibliography using exactly 300 words, on wobulatory kinetics: v Top five sources [5] v Exactly referenced (Harvard) [5] v In order of priority (best first) [10] v Explaining why the order is as given. [10] www. phil-race. co. uk

Validity Whodunit? Fairness Six aspects of ‘quality’ of assessment Real-world links Validity, etc Transparency Manageability

Inclusive assessment starts with making learning happen inclusively. . . www. phil-race. co. uk

How Students Really Learn ‘Ripples’ model of learning Phil Race

Seven factors underpinning successful learning I’ve asked thousands of people the following questions about how they learn. The next slides show the strong trends in responses www. phil-race. co. uk

Task: who said this? ‘Everything should be made as simple as possible, but not simpler’. (Jot your answer down anywhere, just guess). (Albert Einstein, 1879 -1955). www. phil-race. co. uk

Timing of feedback is critical v Feedback only really works after we’ve got students to do something. v Feedback on something they’ve actually done is far more powerful than feedback on something they’ve merely thought. www. phil-race. co. uk

Making learning happen v Asking students questions has been an essential part of making learning happen ever since the time of Socrates. v But we’ve got to use all the tricks in the book to make sure that all our students actually answer the questions. v The default state is for students to be inert! We’ve got to prevent this from happening. www. phil-race. co. uk

Surface learners? Deep learners? Strategic learners? Rote learners? What kinds of learners have we got? Cueoblivious learners? Cueseeking learners? Cueconscious learners?

Don’t just say ‘any questions? ’ Have the occasional prize for a good question – e. g. a tube of Smarties! Get them to add their questions to a ‘frequently needed answers’ page on the course web pages Get them to Tweet or email questions to you over the week, and answer these in lectures Don’t just ask for questions at the end when they’re ready to pack up Getting students to ask questions in and between lectures Get them to Tweet questions to you live in the lecture If you do say ‘any questions? ’, give them a moment to think of some! Ask them all to jot down a question, note who’s writing, then ask for those questions Give them post-its to jot down questions, and send to you Get them asking questions of each other for a minute, then ask for some of the best ones

Teaching… Other people’s knowledge is just information. Teaching is helping people to turn information into knowledge… …by getting them to do things with the information… …and giving them feedback about their attempts. The ten most important words in teaching – and research – and reflection www. phil-race. co. uk

Teaching – and research – and reflection: the ten most important words v v v Why? What? Who? Where? When? How? v Which? v So what? v Wow? And the most powerful four-letter word in the English language. . . ? ? www. phil-race. co. uk

‘what’ is getting ever less important v Columbia University research published recently (2011) in Science magazine shows that people are much less prepared to remember ‘what’, but better at remembering where to access information, and how to do so. v Betsy Sparrow, one of the principal researchers, concludes that the internet has become ‘an external memory source that we can access at any time. . Just as we learn through transactive memory who knows what in our families and offices, we are learning what the computer ‘knows’ and when we should attend to where we have stored information in our computer-based memories. We are becoming symbiotic with our computer tools. Published Online July 14 2011 < Science Express Index Science DOI: 10. 1126/science. 1207745 Report: Google Effects on Memory: Cognitive Consequences of Having Information at Our Fingertips Betsy Sparrow, *, Jenny Liu, Daniel M. Wegner www. phil-race. co. uk

Teaching – and research – and reflection: the ten most important words v Why? (rationale) v What? (content) v Who? (people, you, me, them) v Where? (locations) v When? (times) v How? (processes) v Which? (decisions) v So what? (importance? ) v Wow? (impact? ) And the most powerful four-letter word in the English language. . . ? ? www. phil-race. co. uk

Coaching, We can make learning happen by: explaining, teaching 1. 2. 3. 4. 5. 6. 7. Assessing Ripples on a pond…. Striving to enhance our students’ want to learn; informed Helping students to develop ownership ofmaking the need to judgements learn; Keeping students learning by doing, practice, trial-anderror, repetition; Ensuring students get quick and useful feedback – from us and from each other; Wanting/ Helping students to make sense of what they learn. Needing Getting students deepening their learning by coaching other students, explaining things to each other. Doingdeepen their learning by Allowing students to further assessing their own learning, and assessing others’ Making sense learning – making informed judgements. Feedback

Some more support materials v Alan Hurst (2006) Towards inclusive learning for disabled students in higher education Preston: UCLAN/ HEFCE v http: //www 2. glos. ac. uk/gdn/icp/ Inclusive curriculum project, a series of downloadable booklets, Geography Discipline Network v The South West Academic Network for Disability Support (SWANDS, a HEFCE-funded project at the University of Plymouth, 1999 -2002: ‘SENDA Compliance in Higher Education’ (Judith Waterfield and Bob West, 2002), copyable, or download from… http: //www. plymouth. ac. uk/assets/SWA/Sendadoc. pdf v TECHdis (for information on making electronic materials accessible – and much more)

See also v The ‘DART’ tool from Loughborough University v http: //dart. lboro. ac. uk/tool/

Back to our intended learning outcomes To what extent do you now feel better able to. . . (two hands = much better, one hand = somewhat better, no hands = no better) 1. Make assessment more inclusive. 2. Design inclusive assessment processes and instruments with due regard for achieving validity, reliability, transparency, authenticity, and manageability

Action planning statements v v v One thing I’m going to do is… One idea I’m taking away is… I’m going to think more about… I have found out that … I’d like to know … In future, I’m not going to… 29 October 2020

Thank you… www. phil-race. co. uk Twitter: @Race. Phil e-mail: phil@phil-race. co. uk