Leap Forward Training Resource Toolkit 2018 Leap Forward

Leap. Forward Training Resource Toolkit (2018) Leap. Forward Learning for Practice: Feedforward for effective transition to the workplace

2 Leap. Forward Training Resources If you choose to use these resources in your own teaching, please attribute the Leap. Forward project, University of Bristol Except where otherwise indicated, this work is licensed under a https: //creativecommons. org/licenses/by-nc/4. 0/ This license allows reusers to distribute, remix, adapt and build upon the material in any medium or format for noncommerical purposes only, and only so long as attribution is given to the creator If you have queries about the resources, please contact Sheena. Warman@bristol. ac. uk

Introduction This document is not intended for projection, but rather to support Leap. Forward training packages. The references included in this resource toolkit and the subsequent summary of each theoretical framework, practical aid, guide or model, are intended to supplement the Leap. Forward training packages (as detailed below). Staff are invited to review these references to enhance their understanding of theoretical background of the subject of feedback (and feedforward), and to apply this to assist and support in any developmental activities they may engage in with colleagues or students. Resources are organised to correspond with Workshops are follows: q Workshop 1: Understanding feedback q Workshop 2: Practicalities of workplace feedback q Workshop 3: Emotional and relational aspects of workplace feedback Legend Each item is presented as follows: • Diagram (or reference to diagram) with source • Description of framework, model or practical aid, including principles of operation • Key points, showing how the example links to other work • References 3

Leap. Forward Project: Toolkit Contents 1. Understanding feedback v Formative Feedback model & seven principles of effective feedback: Nicol & Mc. Farlane (2006) v Feedback Model: Hattie & Timperley (2007) v Feedback Dialogue: Boud & Molloy (2013) v Students’ feedback seeking behaviour: Bok et al (2013) v Responsibility-sharing in feedback in HE: Nash & Winstone (2017) v Student Feedback Literacy: Carless & Boud (2018) 2. Practicalities of workplace feedback q Ende’s Principles (1983) q Pendleton’s Rules (1984) q ALOBA: Agenda Led Outcomes Based Approach (1996) q SET-GO (2004) q Frame-based Feedback (2013) q R 2 C 2 facilitated feedback model (2015) q Reflective Conversation model q CORBS q Critical Response Process q Sandwich 3. Emotional and relational aspects of feedback • • Mindset: Dweck (2017) Thanks for the Feedback: Stone and Heen (2015) 4

5 1. Understanding feedback v Formative feedback model & seven principles of effective feedback: Nicol & Mc. Farlane (2006) v Feedback model: Hattie & Timperley (2007) v Feedback dialogue: Boud & Molloy (2013) v Students’ feedback seeking behaviour: Bok et al (2013) v Responsibility-sharing in feedback in HE: Nash & Winstone (2017) v Student feedback literacy: Carless & Boud (2018)

Formative feedback model & seven principles of good feedback practice: Nicol & Mac. Farlane-Dick (2006) See Figure 1 in Reference: A model of self-regulated learning and the feedback principles that support and develop self-regulation in students Description A conceptual model of self-regulated learning and the feedback principles that support and develop self-regulation in students. Principles: 1. Clarify what good performance is 2. Facilitate self-development 3. Deliver high quality feedback information 4. Encourage teacher and peer dialogue 5. Encourage positive motivation and self esteem 6. Provide opportunities to close the gap 7. Use feedback to improve teaching Key Points • Makes reference to students’ motivational beliefs & goal setting • Identifies importance of self-regulation References • Nicol, D. J & Mac. Farlane-Dick (2006) Formative assessment and self‐regulated learning: a model and seven principles of good feedback practice, Studies in Higher Education, 31: 2, 199 -218 6

Feedback Model: Hattie & Timperley (2007) Description The purpose of feedback in this model is to close the gap or reduce discrepancies between the current level of performance or understanding and the desired goal. Key Points • • The gaps or discrepancies can be closed or reduced by students or teachers The model works on 4 levels, although the it is recommended that the fourth level, ‘Self-Level’ is avoided; Process & Self-Regulation used together can provide more powerful, deeper learning Students can put in more effort, use more effective strategies, or modify their goals & self-regulation is encouraged Teachers can provide learning opportunities and challenging goals and can support students with learning strategies References • Hattie, J. & Timperley, H. (2007) The Power of Feedback. Review of Educational Research 77, 81 -112 7

Feedback Dialogue: Boud & Molloy (2013) Selfassessment Knowledge of expected standards External inputs (staff, peers) FEEDBACK DIALOGUE – a collaborative approach to learning Feedbackseeking behaviour adapted from Boud and Molloy 2013 References cont’d • Ajjawi, R & Boud D (2018) Examining the nature and effects of feedback dialogue. Assessment & Evaluation in Higher Education 0: 0, pages 1 -14 Description Model which suggests that dialogue between learners & teachers provides the most useful feedback opportunities Key Points • Positions the learner as a key driver of effective feedback processes and suggests that learners or students need to develop feedback-seeking behaviours • Recognises that dialogue between learners and teachers provides a collaborative partnership approach to learning • Recognises that feedback is a relational activity and that there must be trust and respect for the process to work well References • Molloy, E & Boud, D (2013) Seeking a different angle on feedback in clinical education: the learner as seeker, judge and user of performance information. Medical Education 47(3): 227 -229, 2013 • Boud, D & Molloy, E. (2013). Rethinking models of feedback for learning: The challenge of design. Assessment & Evaluation in Higher Education, 38(6), 698 -712. • Feedback in higher and professional education: understanding it & doing it well (2013) Eds Boud, D. & Molloy, E. , New York: Routledge 8

Students’ feedback-seeking behaviour: Bok et al (2013) Description • See Figure 2 in Bok et al 2013: Factors influencing feedbackseeking behaviour in clinical clerkships Model of feedback which seeks to explain student behaviour surrounding feedback based on a consideration of personal & interpersonal factors, influencing factors and the resultant feedback seeking behaviours. Considers the cost-benefit of students seeking feedback in the clinical workplace Key Points • Intentions, characteristics and behaviours of students are addressed as part of the ‘equation’ as is selfimage and self-identity • Goal setting is seen as relevant to the feedback process • The source of the feedback is relevant in terms of it being someone who is trusted and whose feedback is valued and respected • Seeks to explain student behaviour within clinical context with reference to Mindsets (Dweck), References Relationships and Emotions • Bok, H, Teunissen, P, Spruijt, A, Fokkema, J, van Beukelen, P, Jaarsma, D, & van der Vleuten, C (2013) Clarifying students’ feedback seeking • Takes the view that receiver & giver have a role to behaviour in clinical clerkships Medical Education 47: 282 -291 play in effective feedback processes • Bok, H. , Jaarsma, D. , Spruijt, A. , Van Beukelen, P. , Van Der Vleuten, C. , & Teunissen, P. (2016). Feedback-giving behaviour in performance evaluations during clinical clerkships. Medical Teacher, 38(1), 88 -95 9

Responsibility-sharing in giving and receiving feedback in HE: Nash & Winstone (2017) 10 Description See Figure 1 in Nash and Winstone (2017): Distributions of responsibility for tackling barriers to proactive recipience References • Nash, R. & Winstone, N. (2017) Responsibility sharing in the Giving and Receiving of Assessment Feedback Frontiers in Psychology, 01 September 2017, Vol. 8 • Winstone, N. et al (2017) ‘It'd be useful, but I wouldn't use it’: barriers to university students’ feedback seeking and recipience, Studies in Higher Education, 42: 11, 2026 -2041 Based on a qualitative study which identified four broad kinds of psychological barrier to students engaging with and using feedback: • Awareness: students cannot understand it or know what it’s for, or do not even realise they’ve received feedback • Cognisance: students lack knowledge of opportunities available for them to implement their feedback effectively • Agency: students feel insufficiently equipped to deal with feedback, believing skills or qualities they are advised to develop are fixed, rather than modifiable through effort • Volition: students simply lack motivation to engage with feedback, being unprepared to invest time or effort Key Points • • Responsibility for effective feedback dialogue is shared between student and educator Connections with Mindsets (Dweck), motives and motivation, and cost-benefit of seeking feedback (Bok et al, 2013)

Developing student feedback literacy: Carless & Boud (2018) See Figure 1 in Reference: Features of student feedback literacy This graphic shows how student feedback literacy encompasses appreciating feedback, making judgements, and managing affect; each of these is interrelated, and impacts on action. Description Four inter-related features are proposed as a framework underpinning students’ feedback literacy: appreciating feedback; making judgments; managing affect; and taking action Key Points • • • Feedback literate students have understanding of and appreciate the role of feedback in improving work; have developed capacities to make sound academic judgements; and managing the affective aspects of feedback Crucially, feedback literate students develop a range of strategies for acting on feedback Teachers are identified as playing important facilitating roles in promoting student feedback literacy through curriculum design, guidance and coaching. References • Carless, D, & Boud, D. (2018). The development of student feedback literacy: Enabling uptake of feedback. Assessment & Evaluation in Higher Education, 1315 -1325 11

2. Practicalities of workplace feedback q q q q q Ende’s Principles (1983) Pendleton’s Rules (1984) ALOBA: Agenda Led Outcomes Based Approach (1996) SET-GO (2004) Frame-based Feedback (2013) R 2 C 2 facilitated feedback model (2015) Reflective Conversation model CORBS Critical Response Process Sandwich 12

13 Ende’s Principles (1983) Description Feedback should be: q Undertaken with teacher & trainee as allies working towards common goal q Expected q At a mutually agreed time & place q Close in time to episode on which it is sought q Based on specific behaviour rather than general performance and should have been ideally observed at first hand q Given in small quantities and limited to remedial behaviours q Descriptive, non-evaluative and nonjudgemental q Composed of subjective data, which should be labelled as such q Given on decisions and actions and not on ones' interpretations of student motives A set of guiding principles designed to enable feedback to be constructive, balanced, behavioural and useful Key Points • • Gives due consideration to time and place of feedback session (close to the action) Recommends feedback should be based on observable behaviours (ideally observed at first hand) Pays attention to what is manageable in terms of quantity and timings of feedback Limits feedback to ‘remedial’ behaviours and does not attempt to interpret student motives References • • Ende J. Feedback in clinical medical education JAMA 1983; 250: 777– 81 Ende et al, Measuring patients’ desire for autonomy: decision making and information-seeking preferences among medical patients. J Gen Intern Med 1989; 4: 23– 30

Pendleton’s Rules for feedback (1984) 1. 2. 3. 4. 5. Clarify any points of information/fact Ask the learner what s/he did well – ensure that they identify the strengths of the performance and do not stray into weaknesses. Discuss what went well, adding your own observations (if there is a group observing the performance, ask the group what went well; again, keep them to the strengths. Ask the learner to say what went less well and what they would do differently next time. Discuss what went less well, adding your own observations and recommendations (if there is a group observing the performance, ask the group to add their observations and recommendations). 14 Description Pendleton's rules a set of guidelines for conducting feedback which can apply to group or individual feedback on performance observed at first hand or on video. Reflective conversations models build on these guidelines. • Provides a ‘safe’ structure or environment for learner • Usable for 1: 1 and group feedback • Forces learner to give strengths first, however, the opportunity for interactive discussion of topics relevant to learner may be lost by following a rigid format • Has elements of ‘judgement’ attached • Expectations of students may be negative Key Points • ‘Rules’ may be overstating the fact that this is really only guidelines • Forms the basis for reflective conversation models that additionally incorporate a planning stage References • Pendleton D, Schofield T. , Tate P. , Havelock P. The Consultation: An Approach to Learning and Teaching. Oxford: Oxford University Press; 1984.

ALOBA - Agenda-Led Outcome-Based Approach: Silverman, et al (1996) Agenda-Led, Outcome-Based Approach (ALOBA) 1. Identify learner’s desired outcomes 2. Identify learner’s agenda 3. Encourage self-assessment and self-problem solving 4. Give balanced, descriptive feedback 5. Encourage self- assessment and self-problem solving in response to feedback 6. Make offers and suggestions, generate alternatives 7. Rehearse suggestions 8. Be well-intentioned, valuing and supportive 9. Opportunistically introduce new learning material 10. Structure and summarise learning Description: A recommended approach to guide a productive and effective feedback session or dialogue. Key Points: • • • Provides a ‘safe’ environment for learner Useable for 1: 1 and group feedback Has an element of ‘judgement’, attached which may be covert Expectations of students may be negative Identifies what learner wants help with, the discussion is directed towards achieving learner's goal References • • Silverman, Kurtz & Draper (1996) The Calgary. Cambridge approach to communication skills teaching: Agenda-led, outcome-based analysis of the consultation. Educ Gen Pract , 4: 288– 299 Chowdhury & Kalu (2004) Learning to give feedback in Medical Education Obstetrician & Gynaecologist, October 2004, Vol. 6(4), pp. 243 -247 15

16 SET-GO: Chowdhury & Kalu (2004) SET-GO q What I Saw – describing what I saw q What Else did you see? What happened next q What do you Think? Reflect back to the learner q What Goals were you trying to achieve q Any Offers on how to achieve goals – suggestions regarding skills and rehearsals Description • Simple aide-memoire for use in supporting and conducting observation-based feedback sessions. Built upon Pendleton’s Rules (1984) and ALOBA Approach (Silverman, et al, 1996), developed in clinical practice Key Points • Observable behaviours & actions emphasised • Self-reflection encouraged • Goal-based learning encouraged References • Chowdhury & Kalu (2004) Learning to give feedback in Medical Education Obstetrician & Gynaecologist, October 2004, Vol. 6(4), pp. 243247

Frame-based Feedback: Rudolph et al, (2013) Feedback Algorithm Step : Describe the problem from the instructor’s perspective Tell the trainee unambiguously and specifically what, from the instructor’s personal perspective, went wrong (or right) Description Three step process which encourages the teacher to consider the learner’s point of view (or frame of reference) and tailor instruction accordingly and diagnose both the ‘what’ and the ‘why’ of task failure Key Points • Step 2: Diagnose the frames Ask questions to discover what ‘cognitive frames’ drove the trainee’s Action • Step 3: Teach to trainee’s frames Tailor instruction and discussion to the trainee’s frames • • Proposes an understanding of the learner- or student’s perspective of learning task Operates at a ‘task’ and instructional level and aims to provide actionable feedback Feedback is ‘correctional’ or ‘remedial’, with less emphasis on feedforward or improvement References • Rudolph JW, Simon R, Raemer DB, Eppich W. (2008) Debriefing as formative assessment: closing performance gaps in medical education. Academic Emergency Medicine; 15: 1110– 1116. Rudolph, J. , Raemer, D. , & Shapiro, J. (2013). We know what they did wrong, but not why : The case for ‘frame‐based’ feedback. Clinical Teacher, 10(3), 186189. 17

18 R 2 C 2: Sargeant et al, (2015) R 2 C 2: • Relationship • Reactions • Content • Coach References Sargeant, J. , et al (2015) Facilitated Reflective Performance Feedback: Developing an Evidence- and Theory-Based Model That Builds Relationship, Explores Reactions and Content, and Coaches for Performance Change (R 2 C 2) Academic Medicine 15; 90: 1698– 1706 Description A facilitated feedback model based on personcentred approaches; informed self-assessment and behaviour change foundations. Four phases: 1: Build rapport and relationship; explain the purpose of the assessment report and interview and learn about their context 2: Explore reactions to and perceptions of the data/report 3: Explore physician understanding of the content of the data/ report 4: Coach for performance change Key Points: • Facilitated model aims to engage recipients of feedback in self-reflection and deliberate conversation about performance (largely drawn from research in the clinical environment) • Takes a ‘coaching’ approach (in line with personcentred and behaviour change foundations)

19 Reflective Conversations Model DESCRIBE Relevant, Observable Behaviours What Went Well/ What Went Less Well INVITE SELF ASSESSMENT CREATE ACTION PLAN Description Pragmatic approach to conducting a performance review or feedback meeting which relies on a dialogue between recipient and provider Key Points: • Builds on Pendleton's Rules for feedback • Identifies the importance of dialogue in the feedback process • Self-assessment and self-reflection encouraged • Connections with Carless and Boud (2018) ‘taking Action’ and Sargeant et al, (2015) ‘providing actionable guidance’ • Outcome is a facilitated feedback session rather than a coaching conversation, but parallels between the two approaches NB: original source of model unclear

20 CORBS Description A simple set of guidelines intended to ensure feedback is generally useful and effective C CLEAR O OWNED R REGULAR B BALANCED S SPECIFIC Key Points • Aims to ensure feedback is balanced, timely and effective • Makes no reference to respective roles and responsibilities (ownership is not specified), but provides generic indications of what a reasonable, practicable feedback process should cover References • Maclean, S & Lloyd, I (2013) Developing Quality Practice Learning in Social Work: A Straightforward Guide for Practice Educators and Placement Supervisors NB: original source of acronym unclear

21 Critical Response Process: Lerman (2003) 4 stage process designed for use in Work in Progress Practice 1. Audience say what gave them meaning and why 2. Receiver (student) asks a specific question to focus on an aspect of the work 3. Giver (Peer, Tutor, or audience) asks a neutral question aimed at getting performer to reflect on an aspect 4. Giver (as above) offers a subjective opinion Description A suggested 4 -stage framework which guides a critical reflection process with a group of participants in a creative, practical, work-in-progress practical activity. Key Points • Provides opportunities for participants or audience to offer personal reactions to creative work • Recognises audience, receiver and giver of feedback roles • Allows for subjective views to be offered in an ordered or structured format References • Liz Lerman (2003) Liz Lerman's Critical Response Process • Lerman, L. (1984) Teaching Dance To Senior Adults. Springfield, Ill. : Charles C. Thomas.

22 Feedback Sandwich Description The sandwich feedback method involves discussing corrective feedback that is “sandwiched” between two layers of praise. Aims to be ‘balanced’ through the provision of ‘good’ and ‘bad’ (or corrective) feedback References Von Bergen, Bressler, & Campbell (2014) The sandwich feedback method: Not very tasty Journal of Behavioral Studies in Business Volume 7 Baron, R. A. (1988). Negative effects of destructive criticism: Impact on conflict, self-efficacy, and task performance. Journal of Applied Psychology, 73(2), 199 -207 Ilgen, D. R. , & Davis, C. A. (2000). Bearing bad news: Reactions to negative performance feedback. Applied Psychology: An International Review, 49(3), 550 -565 Key Points • Positive feedback or praise may be ignored or discounted, if the ‘negative’ feedback is perceived as the more useful; may be viewed as highly judgemental • Does not suggest explicitly suggest improvements or necessarily enable actionable feedback • Teacher-led (dialogue is not encouraged) • Finishes with positive rather than negative NB: original source of menthod unclear

3. Emotional and relational aspects of feedback • Dweck, C. (2017) Mindset: Changing the way you think to fulfil your potential. Robinson, London. • Dweck, C. (2006). Mindset : The new psychology of success. Random House, New York • Dweck, C. (2000) Self theories: their role in motivation, personality and development Psychology Press, Hove • Stone, D. and Heen, S. (2015) Thanks for the Feedback, Penguin 23

24 Mindset: Dweck (2017) Description Two types of ‘mindset’: growth and fixed, which can apply to all aspects of life, including education. Mindset can determine what sort of feedback seeking approach an individual takes. Key points • • Learning goal orientation (Growth mindset): a desire to learn new skills, master new activities, understand new things Performance goal orientation (Fixed mindset): winning positive judgements of your competence and avoiding negative ones Individuals can change, chose, or modify the approach they take, or train themselves to adopt a learning goal orientation (growth mindset) Growth mindset held to be most useful for effective feedback seeking behaviour References • • • Dweck, C. (2017) Mindset: Changing the way you think to fulfil your potential (revised edition) Robinson, London. Dweck, C. (2006). Mindset : The new psychology of success. Random House, New York Dweck, C. (2000) Self theories: their role in motivation, personality and development Psychology Press, Hove

Science and art of receiving feedback well: Stone & Heen (2015) Description Feedback is not a simple, scientific process but involves the management of feelings, emotions and relationships with others in the process. Key points Aims to identify the key ‘triggers’ that can help or hinder effective feedback recipience • Truth triggers – the feedback is wrong, unfair, unhelpful, “blind spots” • Relationship triggers – I can’t hear this feedback from you • Identity triggers – the feedback is threatening and I’m off balance • Useful when identifying ‘barriers’ and, by implication avoiding mistakes and engaging in better practice. • Relates to Carless & Boud (2018) active management of affect (emotions & relationships) as part of effective feedback practice. Reference • Stone, D. and Heen, S. (2015) Thanks for the Feedback, Penguin 25

26 Addendum Dec 2020 • There is more recent work focussing on the importance of the sociocultural aspects of feedback. • For a review, see Ramani, S. et al (2019) Feedback redefined: Principles and Practice. J Gen Intern Med 34 (5): 744 -9