Fit for purpose diversifying assessment to meet the

Fit for purpose: diversifying assessment to meet the needs of a 21 st century university education & 21 st century students Prof. Sue Bloxham S. bloxham@cumbria. ac. uk

Workshop outcomes Develop participants skills and knowledge to: - enhance the alignment between what you intend students to learn from your courses and the assessment methods you use to test that learning - diversify assessment methods to: • • engage students involve students in their own assessment and learning Promote assessment for learning and employability Provide formative feedback

reasons to diversify assessment methods • Meeting the different purposes of assessment • Valid assessment – alignment with learning outcomes/ aims • Tasks that assess a 21 st Century curriculum • Student involvement in assessment • Inclusivity • Student perceptions and motivation • Practicability / marking load

Certification to identify and discriminate between different levels of achievement, and between students; providing a license to practice in the case of professional programmes; enabling selection of students for further study and employment. This is assessment of learning.

Quality assurance to provide evidence for relevant stakeholders (for example, external examiners, QAA, professional bodies); to enable them to judge the appropriateness of standards on the programme. This is assessment of learning.

Student Learning to promote effective learning; formative and diagnostic; steering students’ approach to studying; giving the tutor useful information to inform changes in teaching strategies. This is assessment for learning.

Lifelong learning: sustainable assessment to achieve an understanding of standards; to learn how to make judgments; to be able to use criteria; to be able to tell when you really understand something. This is assessment as learning.

The unbalanced purposes of assessment Assessment for and as learning Certification & QA

Assessment for and as learning can: Be immediate; Regular; In-class; On-line; Involve students in their own assessment ; Uses peer assessment. Quick, cheap & low stakes Assessment for certification & QA involves: Formal feedback; 2 nd marking; Moderation; External examiners; Assessment boards. Slow, costly & high stakes

Constructive Alignment Teaching/learning activities Designed to achieve intended learning outcomes May be: teacher-controlled peer-controlled self-controlled as best suits context Learning outcomes ------Expressed using verbs that the students have to enact Verbs chosen to reflect level of achievement (for example lower level learning signalled by verbs such as describe, identify; higher level signalled by verbs such as evaluate, reflect, analyse) From Biggs, J & Tang, C (2011) Assessment of learning Assessment tasks designed to evaluate how well outcomes are demonstrated

Valid assessment – (QAA) • the range and types of assessments used measure appropriately students‘ achievement of the knowledge, skills and understanding identified as intended learning outcomes. It is important that each assessment enables students to demonstrate the extent to which they meet the intended learning outcomes in respect of both the subject and any generic skills. * • Is your assessment FIT FOR PURPOSE? * QAA Quality Code (2012)

Assessing today? How will we know if these intentions have been met? What tool or task could we use? Intended Learning outcomes Develop participants skills and knowledge to: - enhance the alignment between what you intend students to learn from their courses and the assessment methods you use to test that learning

A 21 st Century education? : The USEM account of employability S E Personal qualities, including selftheories and efficacy beliefs Skills including key skills Employability, citizenship, life, etc Subject understanding U Metacognition M Yorke & Knight 2004

21 st Century higher education • Skills for the ‘knowledge economy’* – Critical thinking and problem-solving – Collaboration across networks and leading by influence – Agility and adaptability – Initiative and entrepreneurialism – Effective oral and written communication – Accessing and analyzing information – Curiosity and imagination Do these attributes figure in Programme/ unit outcomes and do assessment tasks foster and test these broader skills and capacities? * ‘Must have’ skills for the future to tackle the ‘Global achievement gap’ (Wagner, 2008)

Characteristics of learning-oriented assessment • • • Formative Demands higher order learning Learning and assessment are integrated Students are involved in assessment It promotes thinking about the learning process; Assessment expectations should be made clear; Involves active engagement of students, developing independent learning; Tasks should be authentic and involve choice ; Tasks align with important learning outcomes Assessment should be used to evaluate teaching.

Exams • They come at the end of a course • They are not integrated into the learning, • The criteria are oblique, • They rarely result in useful feedback, • When students do badly we tend to blame the students rather than use the results to diagnose problems with our teaching.

Essays have the potential to meet many characteristics but: • Often no formative element • Questions may ask students to ‘evaluate’ or ‘critically assess’ a topic but if students can pass adequately by regurgitating others’ evaluation or criticism (from lectures or reading), they may avoid higher order learning. • Students not involved in assessment or thinking about learning • Rarely authentic • Criteria often oblique

Task In small groups, discuss the assessment methods in the module outlines that you brought along and try to allocate them to a box on this grid. Likely to promote learning & employability Well aligned with the learning outcomes (or aims) of the course Not very well aligned with the learning outcomes of the course Less likely to promote learning and employability

Teacher education: embedding formative assessment in a portfolio • First year students, assessment 4000 word portfolio; • Professional Development Activity (PDA) after each taught session; • the PDAs were used in various ways: – peer reviewing; – collating – applying research to a case or problem – sharing of work. • Summative assignment 1500 words • eight PDAs as appendices referenced in the text. • Favourable student evaluation • Higher marks, better engagment Georgia Prescott, Cumbria

Psychology Redesign e. g. ’Assess the strengths and weaknesses of Freud’s and Eysenck’s theories of personality. Are theories incompatible? • Guidance provided for tackling the question and working in a group; • Best definitions & essays posted on VLE as feedback; • Students used familiar language to discuss academic concepts – Dialogue and explanation. Nicol 2009 20 • 560 students in groups of 6 -7; • 3 week cycle culminating in 700 -800 word essay

Peer assessment in lab reports • Formative • Students develop understanding of quality in analysing and reporting science • Higher order learning, focus on the science rather than a description of the process • Students involved in assessment, developing skills of evaluation

Field-based enquiry: childhood studies 1. 2. 3. 4. 5. Introduced to lecturer’s research Exploring signs of childhood Photos posted on line – annotated by self and others 2 visits to museum, gallery, library to collect other images Round table conference facilitated by more experienced students 6. Submit summative research report Sambell, (2010).

Some other assessment methods • Writing tasks: newspaper articles, press releases, executive summaries, information sheets. • Video about a specific topic – Youtube? • Research Grant applications • Lay commentary on specialist material, e. g. journal article • Poster – presenting information clearly & concisely • Presentation – oral communication • Problems and case study analysis • Reflective Journals, Diaries & learning logs • Wikipedia entry

Opportunities and threats Task: On your tables, have a discussion about the ideas and examples so far. Make two lists: - Ideas I can take away - Barriers to changing my assessment

Engaging students in formative assessment – key requirements • It clearly feeds into summative assessment tasks; • The students must submit it in some way (bring to class, post on line, hand it in) and action is taken if they don’t; • Students receive useful feedback on it; • It is not contaminated by summative purposes.

Selling peer assessment Evidence shows students find their peers a useful and more approachable source of help with assignments but we need to stress the main value in peer assessment is standing in the shoes of the assessor – not being assessed – because: • learning about standards – absolutely crucial to making progress and understanding feedback • Seeing other ways of going about the task – develops strategies for taking action • Key employability skill – being able to judge own performance and assessing and giving feedback to others • More opportunity for dialogue • Chance for more formative feedback Peer assessment needs to become a regular feature of programmes so that it is taken seriously and taken for granted as part of learning at this level.

Staff work load A heavy staff workload in assessment is not necessarily helpful to student learning. Improving assessment needs to be accompanied by a shift in effort from tutors to students: • more student activity and engagement (doing tasks, reviewing progress, selfregulation) and less staff activity (marking) • Formative assessment has the potential to achieve both of these and is most likely to increase achievement.

Tutor friendly formative feedback • Tutor posts good examples or model answers on VLE; • Students peer assess tasks using assessment criteria; • Tutors give feedback on posters, presentations in class; • Oral/ on-line feedback to group after marking a sample • Tasks done on-line (e. g EMQs, MCQs), auto marked and give immediate feedback; • Tutors put main effort into marking drafts (agreed with examiner), just checking for change and putting mark on final piece.

In-class activity Out of class activity Example of module-level approaches: the use of exemplars annotated with feedback to encourage dialogue about assessment criteria Students write and submit individual assignments Tutor leads discussion of exemplars previously marked annotated with feedback Module timeline From Oxford Brookes FDTL project on feedback Tutor assesses assignments and prepares feedback Tutor hands back assignments and leads discussion on feedback Submission point 29

From Oxford Brookes FDTL project on feedback 1. Students draft & submit individual assignments In-class activity Out of class activity Example of module-level approaches: Generic (non-personalised) feedback on drafts plus reflective commentary 2. Tutor marks sample of assignments and prepares generic feedback 4. Students rewrite and submit assignments with reflective commentary on how they have incorporated the generic feedback 5. Tutor assesses assignments 6. Tutor hands back assignments with minimal formative feedback 3. In-class discussion of generic feedback Module timeline Submission point 30

Changing student learning through assessment design • Higher order skills may be encouraged by asking students to act on something which requires them to apply, analyse, and evaluate. – E. g. Analysis of a current news item, a piece of scientific research, a lesson, an ethical dilemma. • With such tasks, they can’t regurgitate – they have to use higher order skills because the answers are not in the public domain (plagiarism? ) • The level of guidance offered, and the level of complexity demanded, should change as they move up the levels. • Such assignments are more likely to address a range of learning outcomes rather than one or two specific topic areas. 25/10/2020

Caution • Multiple methods have been associated with negative learning outcomes for students • You may make malpractice easier • Extra workload for students

Benefits to students of moving to in-class, on-line, ongoing assessment and feedback • Immediate feedback • More feedback • Assessment & teaching/learning are integrated • Students involved in assessment – gaining better understanding of standards and own performance • Potential for greater student engagement throughout modules • More independent study • integration with work experience • Raise expectations regarding study workload Quicker, cheaper and low stakes

Task Working in groups • Briefly discuss the module outlines that you have brought along and select one to work on as a group. • Design a new assessment plan for the module. • Particularly think about assessment tasks which might encompass several of the outcomes and how formative work can be embedded in the process. Be prepared to present your assessment plan to the group

What are the unanswered questions?

Conclusion Diversifying assessment can help: - Balance the different purposes of assessment; - Make assessment more valid for a 21 st century curriculum; - Encourage and reward student engagement; - Ensure that students’ study efforts are directed towards meaningful learning; - Use tutor time most effectively for learning and limit the time absorbed by marking and QA to essential summative items;

References Bloxham, S (2014) Assessing assessment (case study by Georgia Prescott), in H. Fry et al (Eds) A handbook for teaching and learning in higher education (4 th Edn) London: Routledge. Nicol, D (2009) Assessment for learner self-regulation: enhancing achievement in the first year using learning technologies Assessment & Evaluation in Higher Education, Vol. 34, No. 3, June 2009, 335– 352 Sambell, K (2010) Enquiry-based learning and formative assessment environments: student perspectives, Practitioner Research in Higher Education, Vol 4, No 1 (2010), p 52 -61 Wagner, T. (2008) The Global Achievement Gap: Why Even Our Best Schools Don’t Teach The New Survival Skills Our Children Need—And What We Can We Do About It Basic Books. Yorke, M. & Knight, P. (2004) Embedding employability into the curriculum. Higher Education Academy General texts on assessment design • Bloxham, S & Boyd, P (2007) Developing assessment in Higher Education: a practical guide, Maidenhead, Open University Press (course reader). . • Bryan, C. & Clegg, K (2006) Innovative assessment in Higher Education. London: Routledge. • Merry, S. , Price, M. , Carless, D & Tara, M. (2013) Reconceptualising Feedback in Higher Education. London: Routledge. • Price, M. , Rust, C. , O’Donovan, B & Handley, K (2012) Asssessment Literacy: The foundation for improving student learning. Oxford: ASKE. • Sambell, K. , Mc. Dowell, L & Montgomery, C (2013) Assessment for Learning in Higher Education, London: Routledge