Design and Implementation of a Course Review Process

Design and Implementation of a Course Review Process Adam M. Persky, Kimberly H. Deloatch, Wendy C. Cox, Mary T. Roth Mc. Clurg, Pamela U. Joyner UNC Eshelman School of Pharmacy, University of North Carolina, Chapel Hill, NC RESULTS INTRODUCTION • Quality assurance is the systematic monitoring and evaluation required to ensure standards of quality are being met. • One of the principal mechanisms for assuring quality of learning and teaching is peer review of teaching and evaluation of the curriculum (learning, teaching and assessment methods) (Dill et al. 1996; Massy et al. 1994). • Horsburgh, (1998) investigated factors that impact student learning through a quality assurance process and found that the most important factors were the curriculum, the instructors, how the teachers taught and facilitated learning, and the assessment practices they adopted. • The course review at UNC focused on: o learning objectives, o course content, o degree of student-centered activities, o assessments methods consistent with learning objectives and course goals, o consistency in coordination, and; o course policies. FIGURE 1 Office of Professional Education and Office of Assessment & Curriculum Committees Assessment Committee Full review of course 2 weeks Formal letter to course director / division chair Post semester follow-up 2 weeks Description of Process • The overall process was assessed by two methods: o the ability the process to identify areas for improvement and o an attitudinal survey completed by the course reviewers and the course directors REFERENCES • Dill DD. Is There An Academic Audit in Your Future? Reforming Quality Assurance in U. S. Higher Education. Change. 2000; 32(4): 34 -41. • Massy WF, National Center for Postsecondary Improvement SCA. Energizing Quality Work: Higher Education Quality Evaluation in Sweden and Denmark. Project 6, Quality and Productivity in Higher Education 1999. • Horsburgh M. Course approval process. Quality Assurance in Higher Education. 2000; 8(2): 96 -9. RECOMMENDATION 6 weeks 4 weeks METHODS Assessment of Process Course Directors Joint meeting to discuss reviews Annual assessment Recommendations implemented • Each required course (n=30) was reviewed by a team of two faculty members • Most of faculty were on the Curriculum or Assessment Committee. • Each team contained at least one clinical faculty member • The review and reports were based on a standardized rubric developed by the Committees which addressed: o course layout and integration, o learning outcomes, o assessment of learning, o resources and materials, and; o learner interaction • Data used in the review process included a self-reflection and teaching goals inventory, course content on Blackboard, curricular mapping information, student course evaluations, and a student completed version of the rubric. TABLE Schematic of the course review process. TABLE 2 Review of comments/plan of course director Review of comments; written reply 25 (23. 4) 19 Active Learning (increase the amount) 20 (19. 6) 16 Assessment Methods (modify, improve, provide more feedback) 16 (18. 3) 17 Content (change, update, remove, re -sequence, reduce) 11 (11. 5) 11 Course Policies (add, modify, clarify) 6 (6. 7) 6 Course Direction (ensure consistency between instructors/integrate or align content with other courses) 2 (1. 9) 2 13 (17. 3) 13 Survey results from the COURSE REVIEWERS. Data presented as % response (n=22, 60%). TABLE 3 Agree Disagree 95. 0 85. 0 Neutral 0. 0 10. 0 90. 0 10. 0 35. 0 45. 0 20. 0 55. 0 40. 0 15. 0 40. 0 95. 0 0. 0 90. 0 10. 0 TOTAL RECOMMENDATIONS NUMBER OF COURSES (N= 93) with Recommendation (% of all recommendations) Learning Objectives (update, clarify, make explicit) Other (readings, Blackboard issues, assignments) 6 weeks The review process was objective. The rubric was beneficial during the course review process. By reviewing other courses, I learned how to improve my own course or teaching. The review process helped me become more familiar with the School's curriculum. The time commitment to review a course was burdensome. The one day format to discuss all the review reports was productive. I prefer the discussion of review reports to be over two or three days not the one-day current format. The course review process is an important part of the quality assurance process for the professional program) Overall I was satisfied with the course review process from the REVIEW TEAM perspective. Number and types of recommendations made to course 1 directors Survey results from the COURSE DIRECTORS. Data presented as % response (n=13, 50%) The feedback I received was objective. The course review provided constructive feedback. The recommendations were consistent with areas already recognized by myself as course director. The time required to prepare the self-reflection documents was burdensome. I would support a course review process every 3 -4 years as part of the curricular quality assurance. For the next offering of the course, the 2010 -2011 academic year, I plan to implement the majority o. . . XXXX The course review process is an important part of the quality assurance process for the professional. . . XXXXX Overall I was satisfied with the course review process from a COURSE DIRECTOR standpoint. Agree Disagree 84. 6 0. 0 69. 2 0. 0 Neutral 15. 4 30. 8 61. 5 23. 1 15. 4 46. 2 7. 7 66. 7 25. 0 8. 3 69. 2 15. 4 84. 6 7. 7 53. 8 38. 5 CONCLUSION • The course review process was able to identify a variety of areas for course improvement (Table 1). • Faculty are supportive of the course review process (Table 2 & 3). • Review team members felt the review process helped them become familiar with the curriculum and how to improve their own courses (Table 2). • Course directors felt the feedback was objective and constructive (Table 3). • There was some feeling among course directors the review was burdensome on their time (Table 3). However the median time was 2 hours (range <1 hour to 17 hours). • Considerations for future iterations of the course review may include more incorporation of student feedback • Next course review will focus on elective courses and pharmacy practice experiences. • Assessment Committee working to identify methods for continuous course review as opposed to a cyclical process every few years.