Moodle Moot Japan 2018 Musashi University Balancing Realtime
Moodle. Moot Japan 2018 – Musashi University Balancing Real-time Feedback vs. Recorded Assessments Using Moodle Rubrics Finding the Balance : Timing, Tools and Teaching (TTT? ) Matt Cotter Hokusei Gakuen University Don Hinkelman Sapporo Gakuin University
Presentation Performance
Interview Performance
Group Performance
Role Play/Drama Performance
PRINCIPLE 1 Assessment of *for* learning Assessment *of* learning for testing/administration Gardner, J. (2012)
PRINCIPLE 2 ‘Endless’ feedback in formative assessment “Learners need endless feedback more than they need endless teaching. ” (Wiggins, 2012) Wiggins, G. (2012 a). “Less teaching and more feedback? ” ASCD in Service. Accessed from: http: //inservice. ascd. org/lessteaching-and-more-feedback/ Wiggins, G. (2012 b). “Seven keys to effective feedback” Educational Leadership, 70(1), pp. 10 -16.
METHODOLOGY Collaborative action research (8 year longitudinal study in one Japanese university) Oral English Presenting Class – 5 Presentations (One semester) 2010 - experimented with syllabus design; types of assessment 2011 - added video recording, hosted on You. Tube, LMS forums 2012 - designed new LMS module for video assessment 2013 - added self and peer assessment 2014 - evaluated self/peer assessment 2015 - added whole class synchronous assessment 2016 - trialed whole class synchronous assessment - added rubric pre-calibration 2017 - synchronous, in class teacher coaching focus
METHODOLOGY Start 2010 - experimented with syllabus design types of assessment 2011 - added video recording hosted on You. Tube trialed LMS forums 1. Post = Performance Feedback
2012 - designed new LMS module for video assessment 2013 - added self and peer assessment 2014 - evaluated self/peer assessment 2015 - added whole class synchronous assessment 2016 - trialed whole class synchronous assessment - added rubric pre-calibration + = Post Performance Scoring & Feedback (self, peer, teacher)
What is the Moodle Video Assessment Module? 1. Video presentation “Hi everyone. Today I’m going to talk about cute stuff!” 2. Upload to PC
3. Upload from PC to Moodle 4. Associate Video with student
5. Assess the Presentation Score Comments • Teachers and students watch videos and assess themselves and/or peers on simple/complex predetermined rubric criteria.
VIDEO ASSESSMENT MODULE - IN USE
PRELIMINARY RESULTS 2016 Quantitative
FINDINGS (UNTIL 2016) Quantitative • Students rated themselves lower than peers and teachers. • Peers scored almost the same as teachers. • Students scores are closer to teacher scores during post assessment. • Student scores are higher when scoring in real time. Qualitative • Students valued the assessment task. • Students valued teacher feedback over peer and self feedback but still valued peer feedback. • Students would rather be assessed than assess.
QUESTIONS FROM FINDINGS 1) Was the load too big for the students? • Students spent a lot of time doing 4 out of 5 presentation post assessments on Moodle (including new pre-training calibration 2) Were students were taking the process seriously? • Most assessing tasks were done out of class time for homework
2017 Asynchronous, post-performance feedback + a little! (Only 2 out of 5 presentations) Synchronous, in class teacher coaching! “Stop! Say it this way! More eye contact! Stress that word, it is important! ……” a LOT! 4 out of 5 presentations
QUALITATIVE DATA FOCUS GROUP & TEACHER’S FORUM Key Research Questions: What types of assessment and feedback are we doing? How often are we doing it? What did students actually do? What did students say at the end of the course? Deferred Research Question: What types of assessment and feedback are most effective?
QUALITATIVE DATA FOCUS GROUP & TEACHER’S FORUM Teacher C: “I often heard the words, “緊張していた ”, meaning ‘kinchou shite ita’, or ‘I was nervous’, but often with a look of accomplishment. I told them conquering nervousness was the number 1 aim, and they will be able to claim this skill when they are job hunting. ”
QUALITATIVE DATA FOCUS GROUP & TEACHER’S FORUM Teacher C: What is coaching? “There is spectrum of: 1. Teaching 2. Interacting 3. Demonstrating 4. Commenting 5. Interrupting …. that makes up coaching. . ”
QUALITATIVE DATA FOCUS GROUP & TEACHER’S FORUM 2. Interacting
QUALITATIVE DATA FOCUS GROUP & TEACHER’S FORUM 3. Commenting http: //moodle. sgu. ac. jp/mod/videoassessment/view. php? id=70900&action=assess&userid=14253
QUALITATIVE DATA FOCUS GROUP & TEACHER’S FORUM Summary: 1) Scripting 2) Good/bad Demo
QUESTIONNAIRE RESULTS (n=48) Watching
QUESTIONNAIRE RESULTS (n=48) Rating (Scoring)
QUESTIONNAIRE RESULTS (n=48) Feedback
QUESTIONNAIRE RESULTS (n=48) Teacher Tool
QUESTIONNAIRE RESULTS (n=48) Teacher Tool
LIMITATIONS AND PLANS FOR 2019 Should separate Q 3. in questionnaire…. . Synchronous (Real Time) or Asynchronous (Post. Performance) Find the right balance between timing, tools and teaching!
There is no one method! Find the balance!
- Slides: 31