A Framework for Embedding the Development of Feedback

A Framework for Embedding the Development of Feedback Literacy into Subject Content Across Disciplines Kieran Balloo University of Surrey @Kieran. Balloo @Surrey. Lab k. balloo@surrey. ac. uk

Feedback literacy » Students’ active role in feedback processes requires them to be feedback literate (Carless & Boud, 2018, p. 1319) » In the short-term, feedback literacy enables students to make better use of existing feedback opportunities within the university curriculum (Molloy, Boud, & Henderson, 2019) » Longer-term, it enables students to operate more effectively in the workplace (Noble et al. 2019), and prepares them for lifelong learning (Carless & Boud, 2018) 2

Developing feedback literacy » Feedback literacy interventions often separate from core curriculum (e. g. Winstone, Mathlin, & Nash, 2019; Värlander, 2008) » Potential for combining the teaching of feedback literacy with core disciplinary content in a fusion of skills and conceptual development (Winstone & Carless, 2019) 3

Towards a framework for developing feedback literacy » If the development of feedback literacy is to be viewed as a core graduate attribute that supports students’ future work capacities, there is a case for embedding the development of student feedback literacy within the core curriculum rather than keeping this separate » Project with Naomi Winstone and David Carless » Aims: • To ascertain whether curriculum models already embed practices that develop feedback literacy • Identify what opportunities exist for integrating the development of feedback literacy with core disciplinary content in a range of subjects 4

International context of current curriculum models 5

Disciplinary context of current curriculum models Hard Soft Pure Biomedical Sciences Biosciences Chemistry Materials Mathematics, Statistics, and Operational Research Physics, Astronomy, and Astrophysics Languages, Cultures, and Societies Sociology Psychology History Geography English Applied Earth Sciences, Environmental Sciences, and Environmental Studies Computing Dentistry Medicine Veterinary Science Engineering Art and Design Law Business and Management Architecture Paramedics Social Work 6

Coding scheme (based on Carless and Boud, 2018) Component Appreciating Feedback Making Judgments Feedback literate students: Understand appreciate the role of feedback in improving work and the active learner role in these processes Recognise that feedback information comes in different forms and from different sources Use technology to access, store and revisit feedback Develop capacities to make sound academic judgments about their own work and the work of others Participate productively in peer feedback processes Refine self-evaluative capacities over time in order to make more robust judgments Maintain emotional equilibrium and avoid defensiveness when receiving critical feedback Are proactive in eliciting suggestions from peers or teachers and continuing Managing Affect dialogue with them as needed Develop habits of striving for continuous improvement on the basis of internal and external feedback Are aware of the imperative to take action in response to feedback information Draw inferences from a range of feedback experiences for the purpose of Taking Action continuous improvement Develop a repertoire of strategies for acting on feedback Example codes Strengths/weaknesses Professional development Feedforward Supervision Self-assessment Self-evaluation Self-criticism Reflection Peer feedback Peer review Peer assessment Constructive criticism Improvement Use feedback 7

National Qualifications Frameworks Appreciating Feedback Australia Hong Kong Mexico UK USA South Africa Making Judgements Managing Affect Taking Action ‘make high level, independent judgements’ ‘Exercise significant autonomy in ‘Work under the determining and achieving personal mentoring of senior and/or group outcomes’ qualified practitioners’ ‘assuming certain responsibilities with regard to the evaluation and improvement of work or study activities’ ‘evaluates the strengths and weaknesses of the process’ ‘the ability to apply, in a self-critical manner, learning strategies which effectively address his or her professional and ongoing learning needs’ 8

Subject Benchmark Statements (SBSs) Appreciating Feedback Hard Pure Biomedical Sciences Biosciences Chemistry Materials Mathematics/Statistics/Operational Research Physics, Astronomy, and Astrophysics TOTAL Hard Earth/Environmental Studies/Sciences Applied Computing Dentistry Medicine Veterinary Science Engineering TOTAL Soft Pure Languages, Cultures, and Societies Sociology Psychology History Geography English TOTAL Soft Law Business and Management Applied Architecture Paramedics Social Work Art and Design TOTAL Making Judgements ● ● Managing Affect ● ● Taking Action TOTAL 2 ● 0 6 0 10 ● ● 0 0 4 ● ● ● 6 ● ● ● 4 ● ● ● 1 ● ● 4 0 ● ● ● 5 ● 0 ● ● 4 1 ● 6 1 10 9

Subject Benchmark Statements: Making Judgements » General sense of what ‘good’ looks like • ‘an appreciation of quality’ (SBS Art and Design, Soft Applied) » Binary judgements of right/wrong • ‘Mistakes can be identified’ (SBS Materials, Hard Pure) • ‘a self-critical evaluation of effectiveness’ (SBS Computing, Hard Applied) » Peer review • ‘use the principles of peer review and quality assurance in dental practice’ (SBS Dentistry, Hard Applied) • ‘understand engage in reflective practice/audit and appraisal of their own and others' work’ (SBS Medicine, Hard Applied) » Evaluating own strengths and weakness • ‘develop a capacity for self-audit’ (SBS Veterinary Science, Hard Applied) • ‘recognise their personal strengths and needs’ (SBS Art and Design, Soft Applied) 10

Subject Benchmark Statements: Managing Affect » Avoiding defensiveness • ‘a willingness to acknowledge and correct errors’ (SBS Law, Soft Applied), • ‘[the capacity to] receive constructive criticism’ (SBS Dentistry, Hard Applied) • ‘willingness to participate in the peer-review process’ (SBS Veterinary Science, Hard Applied) » Capacity for striving for continuous improvement on the basis of feedback • ‘benefit from the critical judgements of others’ (SBS Art and Design, Soft Applied) • ‘identify and work towards targets for personal, academic, professional and career development’ (SBS Biomedical Sciences, Biosciences, Hard Pure; SBS Earth/Environmental Sciences/Studies, Hard Applied) • ‘use supervision as a tool to aid professional development’ (SBS Social Work, Soft Applied) 11

Subject Benchmark Statements: Appreciating Feedback and Taking Action » Appreciating Feedback • ‘appreciate the benefit of giving and receiving feedback’ (SBS English, Soft Pure) » Taking Action • ‘make effective use of feedback’ (SBS Law, Soft Applied) • ‘to take action to improve and enhance their capacities’ (SBS Sociology, Soft Pure) 12

Feedback literacy as a graduate attribute in frameworks » Most frameworks include the development of skills related to ‘making judgements’ » ‘Managing affect’ more prevalent in ‘applied’ disciplines » Frameworks lack focus on ‘appreciating feedback’ and ‘taking action’ • Only present in ‘soft’ disciplines and not evident in National Qualifications Frameworks • May mean students do not fully understand the value of feedback, limiting their development of other aspects of feedback literacy • Goes against ‘new paradigm’ notion that feedback needs to be used by students to improve their work in order to function as feedback (Carless, 2015) » Little evidence of feedback literacy reflected as a graduate attribute, so we identified elements of subject content that may have relevance to learning about feedback • Subject-content sections of Subject Benchmark Statements • Consultation with disciplinary experts 13

Theme 1: Responding to task briefs and client requirements » ‘prepare designs that will meet building users’ requirements’ (SBS Architecture, Soft Applied) Appreciating Feedback Making Judgements » ‘understand evaluate business, customer and user needs, including considerations such as the wider engineering context, public perception and aesthetics’ (SBS Engineering, Hard Applied) » ‘analyse the extent to which a computer-based system meets the criteria defined for its current use and future development’ (SBS Computing, Hard Applied) Managing Affect Taking Action 14

Theme 2: Reflection on and evaluation of professional practice » ‘There are plenty of examples of poor practice … when if colleagues had offered feedback to each other early on some of the poor practice could have been averted’ (Expert Response Nursing, Soft Applied) Appreciating Feedback Making Judgements » ‘the ability to evaluate systems in terms of quality attributes and possible trade-offs presented within the given problem’ (SBS Computing, Hard Applied) » ‘All students understand that whilst on placement in the NHS especially they will be getting feedback all the time often in challenging and upsetting situations. It is our role to prepare them for this and how to act upon it professionally and appropriately before, during and after placement as part of professional socialisation’ (Expert response Physiotherapy, Soft Applied) Managing Affect Taking Action 15

Theme 3: Promoting behaviour change » ‘The graduate will be able to give advice on health promotion and disease prevention’ (SBS Medicine, Hard Applied) Making Judgements » ‘develop effective helping relationships and partnerships that facilitate change for individuals, groups and organisations while maintaining appropriate personal and professional boundaries’ (SBS Social Work, Soft Applied) » ‘[subject-specific knowledge can contribute] to policy and practice, influencing behaviour and delivering positive change to environmental performance’ (SBS Earth Sciences, Hard Applied) Appreciating Feedback Taking Action » ‘[subject-specific knowledge can inform] ways of mitigating human and physical problems and of addressing new challenges’ (SBS Geography, Soft Pure) 16

Theme 4: The mechanisms of learning and supporting others to learn and develop » ‘feedback is a key component of trial and error learning - a good example is the children’s game where you move a metal loop along a wire, and have to get all the way along without touching - the feedback here is a buzzing sound or a light (often called Buzz the Wire)’ (Expert Response Psychology, Soft Pure) Making Judgements » ‘leadership and performance management: selecting appropriate leadership style for different situations; setting objectives, motivating, monitoring performance, coaching and mentoring’ (SBS Business and Management, Soft Applied) » ‘the theories underpin continuing professional development, enabling individuals to be active lifelong learners. This knowledge also equips the paramedic to become an effective teacher in a wide range of settings’ (SBS Paramedics, Soft Applied) Appreciating Feedback Taking Action 17

Theme 5: Learning about cognate concepts » Negative feedback as a regulatory mechanism in ecosystems (Expert response Biology, Hard Pure) » Feedback in physics means that the output from a system at one time is fed back into the system at a later time to affect the later output (Expert Response Physics, Hard Pure) Appreciating Feedback » Goal of physiological regulation, response to external changes to the environment - the key to adaption and survival in a changing environment (SBS Biomedical Sciences, Hard Pure) » ‘In physiology, there are feedback loops to maintain homeostasis within the body. I think this is where the term feedback can become problematic - in physiology, it's more about maintaining the status quo, rather than driving improvement’ (Expert response Veterinary Medicine, Hard Applied) Taking Action 18

Framework for embedding the development of feedback literacy into subject content across disciplines Theme Responding to task briefs and client requirements Feedback literacy may be enhanced through: explaining how feedback is important for understanding client needs drawing links between responding to criteria in an assessment brief and responding to a client’s design brief and/or other requirements drawing links between managing client criticism in order to refine work and managing assessment feedback critiques to improve academic work teaching students how to use assessment feedback in the same way as client feedback Reflection on discussing real events in the discipline where failure to seek feedback led to poor practice and evaluation of drawing links between evaluating the quality of products/processes in students’ discipline and evaluating the quality of their academic work professional preparing students for regular appraisal and evaluation that is part of professional practice instilling an approach to reflective practice where the use of feedback forms a cornerstone of developing as a practitioner Promoting highlighting the parallel between learning to give feedback to others to change their behaviour, behaviour and their own learning about using feedback to improve their work debating how a lack of response to feedback within the field reflects barriers to using feedback change supporting students to use self-feedback to change their own behaviour The mechanisms discussing the role of feedback in the processes that underpin learning of that discipline discussing the role of peer feedback in mentoring and educating others of learning and supporting highlighting how both students and teachers use feedback for their own and others’ ongoing others to learn professional development and lifelong learning and develop Learning about contrasting the concept of feedback within the discipline to the concept of feedback for learning showing how assessment feedback should be used to improve academic work, not just maintain cognate equilibrium concepts Dimensions of feedback literacy developed Appreciating feedback Making judgements Disciplines where approach may be particularly effective Soft Applied Hard Applied Managing affect Taking action Appreciating feedback Making judgements Taking action Appreciating feedback Taking action Soft Applied Hard Applied Soft Pure Soft Applied Soft Pure Hard Applied Hard Pure 19

References » Carless, D. (2015). Excellence in university assessment. London: Routledge. » Carless, D. , & Boud, D. (2018). The development of student feedback literacy: enabling uptake of feedback. Assessment & Evaluation in Higher Education, 43(8), 1315– 1325. » Molloy, E. , Boud, D. , & Henderson, M. (2019). Developing a Learning-Centred Framework for Feedback Literacy. Assessment & Evaluation in Higher Education. » Noble, C. , Sly, C. , Collier, L. , Armit, L. , Hilder, J. , & Molloy, E. (2019). It’s yours to take: generating learner feedback literacy in the workplace. Advances in Health Sciences Education. » Värlander, S. (2008). The role of students’ emotions in formal feedback situations. Teaching in Higher Education, 13(2), 145– 156. » Winstone, N. , & Carless, D. (2019). Designing effective feedback processes in higher education: A learning -focused approach. London: Routledge. » Winstone, N. E. , Mathlin, G. , & Nash, R. A. (2019). Building feedback literacy: Students’ perceptions of the Developing Engagement with Feedback Toolkit. Frontiers in Education, 4(39). » Winstone, N. E. , & Nash, R. (2016). The Developing Engagement with Feedback Toolkit (DEFT). Higher Education Academy (HEA). https: //www. heacademy. ac. uk/knowledge-hub/developing-engagementfeedback-toolkit-deft 20

Thanks for listening Any questions? k. balloo@surrey. ac. uk @Kieran. Balloo @Surrey. Lab 21