AudioVisual Feedback Staff student satisfaction Student attainment George
Audio-Visual Feedback: Staff / student satisfaction & Student attainment George Kominis - Lecturer in Management Control John Kerr - Learning Innovation Officer Marcus King - Student
This Presentation… …describes a case of applying innovative technology for assessment feedback (AVF - Audio-Video Feedback) …offers empirical evidence on the effects of AVF on satisfaction & attainment …seeks to interpret these findings through a student perspective …considers possible implications for institutional changes in approach to feedback
Assessment & Feedback Q 11. I have received helpful comments on my work Business studies Management studies 2017 2016 Change 53. 0% 52. 0% 1. 0% 53. 8% 51. 0% 2. 8% 52. 4% 53. 0% -0. 6% 53. 0% 0. 0%
The Study… Our study investigated three key issues: 1. Does AVF increase student satisfaction levels compared to traditional (written) feedback? 2. Does AVF have a stronger (positive) impact on student attainment levels compared to traditional (written) feedback? 3. How is AVF received by staff who choose to implement it? The results are mixed and some are truly surprising!
The Intervention • • • AVF introduced by the Learning Innovation Officer across the College to enhance/transform feedback practices View on increasing NSS scores Over 40 staff trained on this method 10 took part in returning AVF to over 200 students Camtasia used as the main technology to deliver AVF (formative and summative) Files retuned via Moodle Assignment
Part 1: AVF & Satisfaction
Teaching staff’s perceived benefits from & shortcomings of AVF Benefits from AVF: • Focused • Shortcomings of AVF: • More time consuming • More focused on negatives Impactful • Specific • Detailed • Personalised • Tone of voice
Student satisfaction across college Students loved AVF! • 75% preferred AVF to traditional feedback • 96% thought they would act upon their feedback as a result of AVF • 96% found it easy to match specific comments to their assignments • 70% strongly agreed or agreed that as a result of this method they were spending more time thinking about their feedback.
Students’ perspective of feedback function What the students say What this means for us “Feedback is the biggest aid to my learning. I feel like good feedback is the best tool to Feedback should indicate good learning because it explains how you can improve as well as pointing out key areas students’ good practice. that you are good at or have been successful in. ” “First of all, it provides an excellent means of making me aware of any gaps in my Feedback should indicate areas knowledge. Secondly, it helps to let me know the structure required in the subject. for improvement. Thirdly, and related, it allows me to know what that particular professor is looking for and what they expect. ” “Feedback often helps me identify flaws in my work so therefore I can improve it when I come to do a similar piece. ” “Good feedback is a reference for making future improvements. The feedback is provided by professionals who can give insightful and meaningful comments that try to help make the next essay better or guide you better in how to prepare for an exam. ” Feedback should provide guidance on how to improve.
Students’ perspective on reflecting on & acting upon fback What the students say What this means for us “When researching for and writing a new assignment, I know what to better focus on or how to achieve better understanding of the topic requested from me. It made it clearer what critical thinking is and what an academic standard is for essays and assignments. ” Students can conceptualise areas for improvement and appear to be making judgments of quality regarding their own work. “Thanks to the feedback, I have identified the improvements to be made in this specific assessment. ” “I've listened to it a few times … I have taken comments into consideration and will make sure not to make the same mistakes in future essays. ” “I used the feedback by writing and down from the video and then to ensure on my next assignment Students appear to I had for this subject I made changes that were highlighted in my previous essay. ” be critically reflecting on their feedback and using it “I took notes on areas of improvement and used them to address my next assignment. I also paid to improve on future attention to the comments on areas where performance was good to make sure I used a similar assignments approach in my next assignment. ”
Students’ perceived benefits from & shortcomings of AVF Benefits from AVF: • Focused Shortcomings of AVF: • Technical issues • Impactful • Navigation • Specific • More time consuming • Detailed • More focused on negatives • Personalised • Flexibility of use • Teacher presence • Reflecting of staff effort
Improving AVF - Students’ perspective • Colour coding • Written summary • Technical accessibility • Coordination with the release of the grades • Chuncking • No replacement for face-to-face meetings
Part 2: AVF & Attainment
Research Questions (1) Does the provision of feedback affect (positively) students’ future performance? (2) Does the provision of AV feedback have a stronger effect on students’ future performance than the provision of traditional (written) feedback?
Research Design (1) Research Setting: a course assessed through 3 equivalent individual written assignments (A 1, A 2, A 3) Method: a between-subjects field experiment; students split into two equivalent, independent groups, specifically formed as to control for two extraneous variables, gender and nationality; with the experimental group receiving AV feedback, the control group receiving traditional (written) feedback; then examining the differences in subsequent performance between the two groups.
Research Design (2) Phase 1 (pre-test phase): the base-line performance for all students was established, as improvement in student performance between the first assignment (A 1) and the second assignment (A 2) in the course, after students had received feedback of traditional form after the completion of the first assignment (A 1). Phase 2 (post-test stage): the experimental group received AV feedback on their second assignment (A 2), whereas for the same assignment (A 2) the control group received traditional feedback. Student performance in the third assignment (A 3) was then measured for both groups (control and experimental) to establish whethere were any significant difference in performance between the two groups receiving different forms of feedback (AV vs. traditional feedback).
RQ 1: Impact of Feedback on Student Performance At the descriptive level, students appeared to do slightly better in A 2 than in A 1; but performance dropped slightly from A 2 to A 3. When tested for statistical significance through parametric & non-parametric tests, the differences in performance between A 1 and A 2, and between A 2 and A 3 are not significant; thereby providing evidence that there was no effect of feedback on students’ performance. This finding is counterintuitive and contradicts previous evidence that generally feedback, irrespective of its type, improves student performance.
RQ 2: Impact of AV Feedback on Student Performance The descriptive analysis reveals that AVF has, surprisingly, a negative effect on subsequent performance; this is in contrast to the traditional (written) feedback, which appears to have a marginally positive effect. Parametric and nonparametric tests show, perhaps slightly less surprisingly, that the differences in subsequent performance are not significant, neither for AVF nor for traditional feedback. 16. 1 16 15. 9 15. 8 Traditional Feedback Subgroup 15. 7 15. 6 Video Feedback Subgroup 15. 5 15. 4 15. 3 15. 2 Assignment 1 Assignment 2 Assignment 3
In Sum… Contrary to our expectations / hypotheses… • AVF does not appear to have a (positive) effect on students’ performance, which actually appears slightly reduced in our course, despite students’ enthusiastic attitudes towards it. • But, if this is any consolation, feedback in general (of all forms) does not seem to have a discernible effect on student performance in the setting and sample examined.
Disclaimer… The study’s findings need to be considered in light of the experiment’s limitations: • • • students were not randomly allocated to the two groups; although an attempt was made to control for important extraneous variables (gender and educational background), it is possible that other confounding variables, not controlled in this study, may have affected performance change (e. g. various pressures from other courses or from students’ extra-curricular activities) alongside feedback type; this threat to internal validity is inherent in quasi-experimental designs, where control over groups is generally limited. On the positive side, our findings are based on data collected from real students in a real (higher education) context, which provides some assurance about the ecological and population validity of the findings.
Thank you! Link to report - http: //eprints. gla. ac. uk/119844/ /glasgowuniversity @Uof. Glasgow Search: University of Glasgow #Uof. GWorld. Changers
- Slides: 21