Using Dialogue as an Assessment Tool Conference Paper

Using Dialogue as an Assessment Tool Conference Paper July 2010 ESCalate-sponsored Project RMHPilkington@uclan. ac. uk

To explore how dialogue can be used: �To assess knowledge �To assess practice �To evidence reflection on practice �To assess professional learning � 5 universities across UK �November 2009 – December 2010 Aims

Introduction to the Project Discussion of theory and support for Dialogue as an Assessment tool 3. Methodology 4. Emerging issues 1. 2. ◦ 5. training assessors, issues of judgement, evidence of reflection, evidencing scholarship, process questions, managing and facilitating dialogue Conclusions: Questions and Answers Structure of Presentation

� Using dialogue to assess students, staff dialogue to assess professional learning: competence, using standards � Transcription of recorded dialogue; sharing and reflection on process; interviews with assessment protagonists (assessors, students) � Analysis of transcripts for evidence of reflection, learning, evidence around practice, professional achievement � Review of process, challenges, benefits through interviews � Analysis of how dialogue constructed; themes; evidence to inform judgement � Recommendations to others for practice � Investigation of dialogue as tool for assessment The Project

� Critical discussions � APEL /APL processes � Reflective dialogues � Professional conversations � Dialogue with evidence � Group assessment � Presentations � Narrative, storytelling Examples of use

�Sharing, exchange �Between peers �Participatory �Open-endedness �Unknowns �Talk between 2 or more people in which thoughts feelings and ideas are expressed, questions are asked answered, or information exchanged Dialogue defined

�Trust �Empathy �Honesty �Sincerity �Openness �Learning �Self awareness �Growth �Mutual responsibility �Listen �Challenge �Interrogation �Inquiry �Exploration �Reflection and critique �Shared knowledge �Process �Multiple levels of communication Characteristic Elements

Group a Group b Group c Conversation with a purpose It has to involve moving forward Learning should emerge for all Care should be given to the different purposes of dialogue There is an inherent power differential in assessment due to the differing roles Assessment involves: quality of dialogue, evidence of understanding The key to defining assessment is about looking at what dialogic assessment is not Process versus product, exposing of the process Assessment for learning A shared purpose Genuine questions to seek unknown answers Formative for the participant (backwards and forwards) Power relationship is two-way Consideration of influence of assessment criteria / Learning outcomes Appreciative Inquiry versus negative marking Dialogue is about learning evolving Assessment is about summative element Dialogue involves complex interaction; compare with monologue Mutuality, turn-taking Different expectations can be an issue Notion of process is critical Emphasis is on growth Would students know when dialogue is in process? Defining dialogue

Written � Static, superficial, write to order, � Writing reflectively � Emergence of values � Word limit � Limited opportunity to probe, question, explore � Evidence all present � Easier to grade? A Comparison Dialogue � Multi-layered � Flexible � Easier to reflect � Engagement easier to show and explore � Opportunity to probe, question � Depth of learning � Avoids plagiarism � Natural when talking about self and practice � May need back-up evidence

� Mentor – mentee process (Brockbank & Mc. Gill) � Criterion led professional conversation (Bowen Clewley) � Appreciative Dialogue (Ghaye) � Assessment process (O’Donovan et al) � Self and Peer assessment (Yorke & Knight; Bryan & Clegg) � Socialisation, literacy (Lea & Goodfellow); � individual construction of professional knowledge (Shulman, Eraut, Boud); � Work-based, informal learning (Boud); � Values-led, professional and reflective (SEDA/UK PSF, Moon, Larrivee); � Organisational learning and change models (Pedlar, Burgoyne, Easterby-smith, Kotler) Model of UCLan dialogue

Preliminary analysis: �Construction of dialogue, facilitation �When is a dialogue? �Appreciative aspects �Evidence to support judgement �Evidence of rigour �Level of Reflection �Evidence of scholarly practice �Value for individual Interesting Outcomes:

�Assessment of informal professional learning �Creation of a learning space �Re-profiling idea of HE professional �Re-configuring teaching and learning �A career development and CPD tool �Means of organisational change Seeking case studies Conclusions

� � � � Brockbank A & Mc. Gill I 2007 Facilitating Reflective Learning in HE SRHE/OUP 2 nd Ed. Bowen Clewley L ‘Assessing against competency standards in the workplace’ in 207 -227 in Arguelles & Gonczi (2000) Easterby Smith Eraut M 1997 Professional Learning and Competence Ghaye T & Lillyman S 2006 Learning Journals and Critical Incidents RP for HE Professionals 2 nd Ed Quay Books London O’Donovan B, Price M and Rust C (2004) ‘Know what I mean? Enhancing Students understanding of assessment standards and criteria’ in Teaching in HE Vol 9, No 3, July 2004 Knight P & Yorke M (2003) Assessment Learning and Employability SRHE/OUP Boud D and Falchikov N Eds Rethinking assessment in HE 2007 Routledge Bryan C & Clegg K 2006 Innovative Assessment in HE Routledge Goodfellow R & Lea MR (2007) Challenging E-learning in the University: a literacies perspective SRHE / OUP Larrivee, Barbara(2008)'Development of a tool to assess teachers' level of reflective practice', Reflective Practice, 9: 3, 341 — 360 Moon J (2004) A Handbook of Reflective and Experiential Learning: Theory & Practice Routledge Falmer Pedler M (1996), Action Learning for Managers, The Learning Company Project Pedler M, Burgoyne J, Boydell 1997 The Learning Company Mc. Graw Hill, Berkshire Shulman LS ‘Knowledge and Teaching’ pp 61 -77 in Leach J, Moon B (eds) (1999) Learners and Pedagogy Sage/ Paul Chapman pubs References: