Open Mentor Supporting tutors with their feedback to

Open Mentor: Supporting tutors with their feedback to students Denise Whitelock Open University, Walton Hall, Milton Keynes d. m. whitelock@open. ac. uk Stuart Watt The School of Computing, The Robert Gordon University, Aberdeen s. n. k. watt@rgu. ac. uk DMW 2007 The Open University's Institute for Educational Technology

Formative feedback for tutors: Open. Mentor – a training and monitoring e-Tool? • • What implicit /explicit models do tutors have about marking assignments? What training is given to tutors about marking assignments? Do we hold the company line when marking? Tutor questionnaire DMW 2007 The Open University's Institute for Educational Technology

What is Open Mentor? “An open source mentoring tool for tutors” “Open source” = free and easy to use, and to embed in an institutions infrastructure and working practices “mentoring” = designed to help people learn how to give feedback effectively, through reflection and social networks “tutors” = primarily intended for teaching staff, but with clear applications for those involved in quality DMW 2007 The Open University's Institute for Educational Technology

Coding into Categories • Bales analysis • Psychology 1950 s • Analyses talk • Includes socio-emotive categories • Flander’s (1970)categories inappropriate as also includes classroom control DMW 2007 The Open University's Institute for Educational Technology

Bales Categories • Four main groupings • A. Positive reactions; agreeing and boosting the other person • B. Directing/teaching • C. Questions: requesting information, clarification etc • D. Negative reactions: disagreement DMW 2007 The Open University's Institute for Educational Technology

Coding the comments Bales’ Interaction Process DMW 2007 The Open University's Institute for Educational Technology

Identifying trends: H 801 Graph to show conflated Bale’s categories against mean number of incidences in H 801 scripts DMW 2007 The Open University's Institute for Educational Technology

Identifying trends: H 801 Key: A = Positive reactions B = Responses C = Questions D = Negative reactions Pie Chart to show the mean number of incidences per pass per conflated Bales' Interactional Category for all four levels of pass in H 801 scripts DMW 2007 The Open University's Institute for Educational Technology

How general is the model? • S 103 – Foundation Science Course • B 820 – Masters Business School Course DMW 2007 The Open University's Institute for Educational Technology

Identifying trends Pie Charts to show the mean number of incidences per conflated Bales Interactional Category for ‘Pass 1’ and ‘Pass 4’ in the following courses: B 820 S 103 H 801 Pass 4 Key: A = Positive reactions C = Questions B = Responses D = Negative reactions DMW 2007 The Open University's Institute for Educational Technology

Explaining Open Mentor’s Rules • Four categories • A – Positive Reactions • B – Attempted Answers • C – Questions • D – Negative Reactions DMW 2007 The Open University's Institute for Educational Technology

‘A’ - Positive Reactions DMW 2007 The Open University's Institute for Educational Technology

‘B’ Attempted Answers DMW 2007 The Open University's Institute for Educational Technology

‘C’ Questions DMW 2007 The Open University's Institute for Educational Technology

‘D’ Negative reactions DMW 2007 The Open University's Institute for Educational Technology

Example Rules • . . . you could. . . (B 4) • . . . good. . . but. . . (B 4) • . . . excellent analysis. . . (A 1) • . . . this. . . does not. . (D 10) • . . . what are. . . (C 7) • . . . in my opinion. . . (B 5) DMW 2007 The Open University's Institute for Educational Technology

Building the rule set DMW 2007 The Open University's Institute for Educational Technology

Is the rule set generic? DMW 2007 The Open University's Institute for Educational Technology

Is this a model that works in the way tutors and students expect? 47 students and 44 tutors responded to a questionnaire to test OM’s underlying tutorial model Findings suggest: Lower grades should attract more detailed comments and explanation by the tutor (students) Higher grades should attract more positive comments (students and tutors) Lower grades attract more questions and suggestions (tutors) Model supported by pedagogical study DMW 2007 The Open University's Institute for Educational Technology

How Open Mentor handles comments “Good work” A = positive reactions “Yes, well done” A = positive reactions “Yes, but is this useful? ” B = attempted answers, and not a positive reaction “Can you explain what you mean” C = questions “This does not follow” D = negative reactions DMW 2007 The Open University's Institute for Educational Technology

Open Mentor in action DMW 2007 The Open University's Institute for Educational Technology

DMW 2007 The Open University's Institute for Educational Technology

Inside Open Mentor Benchmarks Course info Web interface Analyser Assessments Extractor Classifier DMW 2007 The Open University's Institute for Educational Technology

Current and future developments Embedding in institutional practice Enhancing quality of first year provision Links to VLEs and information systems Further development Support for students Course evaluation Open Comment Ensuring emotional support as well as conceptual guidance for learners DMW 2007 The Open University's Institute for Educational Technology