Academic Reviews Orientation Fall 2019 Why Academic Reviews

  • Slides: 43
Download presentation
Academic Reviews Orientation Fall 2019

Academic Reviews Orientation Fall 2019

Why Academic Reviews? 1. We have to! • The academic review processes for all

Why Academic Reviews? 1. We have to! • The academic review processes for all Ontario Universities are based on the same guideline – The Quality Assurance Framework • Mandated by COU and forms the basis for: – The cyclical review of existing programs – The approval of any new programs, and – Final approval by the MTCU when considering funding for new programs

Why Academic Reviews? 2. It helps us re-think/refresh curriculum “…the exercise of having an

Why Academic Reviews? 2. It helps us re-think/refresh curriculum “…the exercise of having an Academic Review has been not only a necessary commitment on behalf of the Department and the University, but also an important restructuring and self-analysis for members of the Department. We have spent a year taking the recommendations as serious starting points to review our philosophical and academic justifications for our undergraduate curriculum. As a result, our department members are certainly more aware of the challenges we have been dealing with over the recent past, we are more sensitive to the concerns of individual faculty, and we have been more explicit as to why certain program decisions are appropriate for our students. ” Psychology Self-Study, 2014

General Overview • Reviews have taken place since 1970 s • Ontario Universities Council

General Overview • Reviews have taken place since 1970 s • Ontario Universities Council on Quality Assurance (“Quality Council”), formed in 2010 by the Ontario Council of Academic Vice-Presidents (OCAV) • Quality Assurance Framework (QAF)2010, rev. 2016 • Institutional Quality Assurance Processes (IQAP) since 2010, rev. 2016

Quality Assurance Website https: //brocku. ca/vp-academic/quality-assurance/

Quality Assurance Website https: //brocku. ca/vp-academic/quality-assurance/

BROCK’S IQAP INSTITUTIONAL QUALITY ASSURANCE PROCESSES

BROCK’S IQAP INSTITUTIONAL QUALITY ASSURANCE PROCESSES

Overview – FHB III: C. 11 – IQAP first approved by Senate (May 2010)

Overview – FHB III: C. 11 – IQAP first approved by Senate (May 2010) with a revised version May, 2016 – Focus on Learning Outcomes and U/GDLEs – Academic Review Committee (ARC) – Combine Graduate + Undergraduate (Integrated) – Quality Council – approve changes to the IQAPs, reviews new programs and undertakes an audit – Quality Council Audit – March 2013, again in 2022 -23

Cyclical Reviews • IQAP Section 2, Pages 10 -21 • Highlights: – Self-Study Manual/Template

Cyclical Reviews • IQAP Section 2, Pages 10 -21 • Highlights: – Self-Study Manual/Template (p. 13) – Evaluation Criteria (pp. 13 -15) – Review Committee (pp. 16 -17) – Comments/Responses (pp. 19 -20) – Final Assessment Report (FAR) (p. 20)

Academic Review Committee (ARC)

Academic Review Committee (ARC)

Overview • Special Committee of Senate • Responsible for the coordination, monitoring and implementation

Overview • Special Committee of Senate • Responsible for the coordination, monitoring and implementation of all aspects of the IQAP • Adherence to IQAP audited by the Quality Council every 8 years (2022) • Also provides reports to the Quality Council and Board of Trustees

Committee Membership 2019 -20 Ex officio: Chair: Provost and Vice-President, Academic (or Designate) –

Committee Membership 2019 -20 Ex officio: Chair: Provost and Vice-President, Academic (or Designate) – Greg Finn Vice-Chair: Vice-Provost and Associate Vice-President, Academic – Brian Power Appointed members: Applied Health Sciences – Maureen Connolly Education – currently vacant Goodman School of Business – Eric Dolansky Graduate Studies – Danny Cho Humanities – James Allard Mathematics and Science – Alecksandar Necakov Social Sciences – David Hayes Graduate student – Alison O’Connor Undergraduate student – Abdul Basit BUFA Observer (non-voting) – currently vacant Non Voting Members: Deans, as needed (for reviews pertinent to Programs in their Faculties) Dean of Graduate Studies (for all graduate programs)

Areas of Responsibility • Cyclical Reviews – Oversight and approval of cyclical academic review

Areas of Responsibility • Cyclical Reviews – Oversight and approval of cyclical academic review of programs within a Unit every eight years – Review of annual Implementation Reports for four years after review • New Program Approvals – Approval of new undergraduate and graduate programs (New = having substantially different program requirements and learning outcomes than existing programs) • Major Modifications – Approval of significant changes to undergraduate and graduate program requirements or learning outcomes, often with resource implications • Discontinuation of Programs

Cyclical Reviews Process • ARC reviews the Self-Study and suggests changes. When satisfactory, ARC

Cyclical Reviews Process • ARC reviews the Self-Study and suggests changes. When satisfactory, ARC approves the document for distribution to the Review Team • The Office of the AVPA organizes the site visit • ARC receives the Reviewer’s Report and requests internal responses to it from the Unit/Program, the Dean(s), and the appropriate Senate Committee (UPC or SGSC) • ARC develops the Final Assessment Report (FAR) • ARC monitors implementation of recommendations

2020 -21 Program Reviews Includes ALL degree programs, concentrations, minors, certificates and co-op where

2020 -21 Program Reviews Includes ALL degree programs, concentrations, minors, certificates and co-op where applicable in the following units: UNDERGRADUATE ONLY (U) • Interactive Arts and Sciences GRADUATE ONLY (G) • Faculty of Applied Health Sciences Graduate Programs INTEGRATED (I) • • History Geography and Tourism Studies

THE PROCESS (STEP-BY-STEP)

THE PROCESS (STEP-BY-STEP)

Process • Eight-Year Cycle • Review of Programs NOT Departments • Steps: 1. 2.

Process • Eight-Year Cycle • Review of Programs NOT Departments • Steps: 1. 2. 3. 4. 5. 6. 7. Self-Study Reviewers’ Site Visit Reviewers’ Report Responses Final Assessment Report Annual Report for 3 years Four Year Report

Step 1: Self-Study • Self-Study Committee (IQAP 2. 7) – Faculty, staff and student

Step 1: Self-Study • Self-Study Committee (IQAP 2. 7) – Faculty, staff and student participation/involvement • Program alignment with Brock’s Mission, Vision, Strategic Plan and SMA • Program history, results of previous review • Degree Level Expectations (DLEs) and Learning Outcomes • Data (with analysis) • Assessment of Resources • Potential for Growth and Improvement • Unit’s Plan • Routing: Goes to Dean(s) and then to ARC (for approval) • Confidential (to Program, Dean(s), Reviewers & ARC) • Privacy -avoid mentioning individuals except where specifically called for (e. g. , Faculty awards)

Step 2: Reviewers’ Site Visit Review Committee (IQAP 2. 8) • 2 external, one

Step 2: Reviewers’ Site Visit Review Committee (IQAP 2. 8) • 2 external, one from outside Ontario • 1 internal, from outside the Faculty, not necessary that they have knowledge of the program or discipline • “Arm’s-Length” relationship • Receive approved Self-Study (plus other University documents) • Site Visit (arranged by AVPA Office) – – Faculty, students, staff, library Provost and Vice-Provost/AVPA, Dean(s) Supporting units - Co-op, Library, others Tour of physical facilities

Step 3: Reviewers’ Report (IQAP 2. 10) • • • Executive Summary Outcome Category

Step 3: Reviewers’ Report (IQAP 2. 10) • • • Executive Summary Outcome Category Outline of Visit Program Strengths Feedback on Evaluation Criteria Other Issues Recommendations Confidential Section (if necessary) Report submitted to AVPA Office

Step 3: Reviewers’ Report • Feedback on Evaluation Criteria (IQAP 2. 5) – Objectives

Step 3: Reviewers’ Report • Feedback on Evaluation Criteria (IQAP 2. 5) – Objectives – Admission requirements – Curriculum – Teaching and Assessment – Resources – Quality Indicators – Quality Enhancement – Additional Graduate Program Criteria

Step 4: Responses (IQAP 2. 11) • Report received 30 days after site visit

Step 4: Responses (IQAP 2. 11) • Report received 30 days after site visit • AVPA Office asks for responses to the Reviewers’ Report from: • Department/Centre/Program • UPC and/or SGSC • Dean(s) “in consultation” – Recommendations and Unit Responses – Changes in organization, policy or governance – Resource Implications – Timeline for Implementation • All responses sent to AVPA’s Office

Step 5: Final Assessment Report (IQAP 2. 12) • Written by ARC • Dean(s)

Step 5: Final Assessment Report (IQAP 2. 12) • Written by ARC • Dean(s) and Unit Rep attend ARC meetings when FAR is discussed • • • Summary of the Review Strengths of Program Opportunities for Improvement Disposition of Recommendations – Accepted or NOT If accepted - Implementation Plan: – Who is responsible? – Timelines for action • Draft to Unit, Dean(s) • Approved version sent to Senate/Quality Council • Published on Brock’s Quality Assurance website

Step 6: Implementation Reports (IQAP 2. 15) • Deans consult with unit and report

Step 6: Implementation Reports (IQAP 2. 15) • Deans consult with unit and report annually for three years, on status of all Recommendations for programs reviewed from their Faculty • Unit reports four years after Senate approval of the FAR • Reports submitted to ARC • ARC approves • ARC then reports to Senate • Unit’s report posted on QA website

Abbreviated Schedule • October 2019 to October 2020 (12 months) – Orientation/Data – Collection

Abbreviated Schedule • October 2019 to October 2020 (12 months) – Orientation/Data – Collection and Analysis/Writing • September 1, 2020 – List of proposed reviewers to AVPA Office • September 1, 2020 – Self-Study to designated ARC Reader • October 15, 2019 – Completed Self-Study submitted electronically to AVPA Office • January 2021 – April 2021 – Site Visit, depending on availability of reviewers • February 2021 – May 2021 – Reviewers’ Report and Responses • May 2021/June 2021 – Final Assessment Report

THE SELF-STUDY

THE SELF-STUDY

Self-Study Template 1) Program Fact Sheet* Unit Background – – 2) Introduction and Background

Self-Study Template 1) Program Fact Sheet* Unit Background – – 2) Introduction and Background (Program NOT Department/Centre) Consistency with the University’s Mission and Academic Plan (IQAP 2. 5. 1) Evolution of the program Self-Study Process Previous Review (IQAP 2. 5. 7) = Four year report from the previous review 3) Degree Level Expectations and Learning Outcomes (IQAP 2. 5. 1) – – – 3. 1 Course Learning Outcomes Table A. 1 – Course Learning Outcomes** (IQAP 2. 5. 1) 3. 2 Program Learning Outcomes (PLO) (IQAP 2. 5. 1) 3. 3 Assessment of Learning and Improvement of the Program (IQAP 2. 5. 4) • • – 3. 4 Program Curriculum Map • – Table 3. 1 - Mapping Program Outcomes to Degree Level Expectations Table 3. 2 – PLOs to Modes of Delivery and Methods of Assessment Table 3. 3 – Program Curriculum Map 3. 5 Additional Learning Outcomes Support • NSSE, CGPSS, other relevant data if available *– data provided TO the Program by IA **– data provided BY the Program

Self-Study Template cont’d 4) Program Structure, Curriculum and Delivery Modes (IQAP 2. 5. 3,

Self-Study Template cont’d 4) Program Structure, Curriculum and Delivery Modes (IQAP 2. 5. 3, 2. 5. 6) Complete Calendar Entry – 4. 1 Program Goals (IQAP 2. 5. 1) – 4. 2 Program Options • Undergraduate –BA Honours, with Major, Pass, BA/BEd, Concentrations, Streams, Minors, …. • Graduate – MA, Thesis, MRP, Course based, Ph. D, Fields, …. . – 4. 3 Program Structure • Year 1, Year 2, Year 3, Year 4 – 4. 4 Program Curriculum • Calendar Outline • Table of course information for Program Course No. 4. 1 – Summary Course Title Contact Hrs/week Delivery Mode ERSC 1 F 01 Lecture, Lab Dynamic Earth • Current State of the Discipline • Collaborative Arrangements 4

Self-Study Template cont’d 4) Program Structure, Curriculum and Delivery Modes (cont’d) – 4. 5

Self-Study Template cont’d 4) Program Structure, Curriculum and Delivery Modes (cont’d) – 4. 5 Course Offerings (past 8 years) • Courses Offered by Unit – Undergraduate » Table 4. 2 a Course Registrations by Level » Table 4. 2 b Course FCE Enrolment by Level » Table 4. 2 c FCE Course Sections by Level » Table 4. 2 d Course Offerings and Registrations – Graduate » Table 4. 3 a Course Registrations by Level » Table 4. 3 b Course FCE Enrolment by Level » Table 4. 3 c FCE Course Sections by Level » Table 4. 3 d Course Offerings and Registrations • Additional Required Courses offered by Other Units

Self-Study Template cont’d 4) Program Structure, Curriculum and Delivery Modes (cont’d) – 4. 6

Self-Study Template cont’d 4) Program Structure, Curriculum and Delivery Modes (cont’d) – 4. 6 Modes of Delivery • Level of Interactivity • Class Size • Appropriateness and Effectiveness

Self-Study Template cont’d 5) Faculty and Staff Resources (IAQP 2. 5. 5, 2. 5.

Self-Study Template cont’d 5) Faculty and Staff Resources (IAQP 2. 5. 5, 2. 5. 6) – 5. 1 Full and Part-time Faculty – – – Table 5. 1 - Instructor Qualifications and Teaching Assignments within the Unit (5 years) Table 5. 2 – Contributions by Instructors to other Units (5 years) Table 5. 3 – Intellectual Contributions (past 8 years) – Role of Adjunct and Part-Time Faculty – 5. 2 Faculty Directly involved with the Program from other Units – Table 5. 4 - Contributions by Instructors from Other Units (5 years) – 5. 3 Permanent Staff with Teaching responsibilities associated with the Program – Table 5. 5 - Permanent Staff with Teaching Responsibilities (5 years) – 5. 4 All Other Administrative Support Staff directly associated with the Unit – Table 5. 6 - All Other Administrative Support Staff directly associated with Unit (5 years) – 5. 5 Research Grants – – Table 5. 7 – Research Grants and Contracts – External Table 5. 8 – Research Grants and Contracts – Internal – 5. 6 Faculty Awards and Distinctions – 5. 7 Financial Support for Graduate Students – Table 5. 9 – Financial Support to Graduate Students

Self-Study Template cont’d 6) Budget, Space and Other Resources (IQAP 2. 5. 6) –

Self-Study Template cont’d 6) Budget, Space and Other Resources (IQAP 2. 5. 6) – 6. 1 Operating Budget (previous 5 years, excludes FT faculty and staff salaries) (IAQP 2. 5. 2) – 6. 2 Space and Equipment – 6. 3 Other Academic Resources 7) Technological Resources and Support (IQAP 2. 5. 5) – 7. 1 ITS Report – provided by ITS – 7. 2 Educational Technology Report (CPI) - provided by CPI 8) Library Resources Library Report –provided by the appropriate Liaison Librarian NOTE: NOT written by the unit! 9) Comparators (IQAP 2. 10) – 9. 1 Undergraduate – 9. 2 Graduate

Self-Study Template cont’d 10) Admissions (IQAP 2. 5. 6 b) – 10. 1 UG

Self-Study Template cont’d 10) Admissions (IQAP 2. 5. 6 b) – 10. 1 UG and Grad admission Requirements and Qualifications of Incoming Students – 10. 2 Admission Targets and Numbers of Students Registered – – Table 10. 1 10. 2 10. 3 10. 4 UG Admission Targets and Registrations GR Admission Statistics UG Admission Averages GR Admission Averages – 10. 3 Admission Patterns – Table 10. 5 UG Admission Patterns (101 s vs 105 s) 11) Enrollments (IQAP 2. 5. 6 b) – 11. 1 UG number of Majors for past eight years – Table 11. 1 UG November 1 Headcount by Major – 11. 2 UG: Full-time Equivalent Enrolment for past eight years – Table 11. 2 UG Full Time Equivalent Students Taught – 11. 3 Grad enrolments and Projections – Table 11. 3 GR Student Headcount

Self-Study Template cont’d 12) Retention, Graduation and Times to Completion – 12. 1 UG

Self-Study Template cont’d 12) Retention, Graduation and Times to Completion – 12. 1 UG Retention and Graduation Rates (10 year Cohort) • UG Retention and Graduation Report (CSRDE Data) – 12. 2 Grad Flow-Through and Times to Completion • Table 12. 2 a – Masters Students: Flow-Through and Times to Completion • Table 12. 2 b – Doctoral Students: Flow-Through and Times to Completion 13) Student Success – 13. 1 Scholarly Success (IQAP 2. 5. 8) – 13. 2 Pathways after Graduation • UG (IQAP 2. 5. 6) - Table 13. 2. 1 Ontario University Graduate Surveys • Grad

Self-Study Template cont’d 14) Surveys – 14. 1 Current Students UG and GR –

Self-Study Template cont’d 14) Surveys – 14. 1 Current Students UG and GR – 14. 2 Recent Alumni UG and GR – 14. 3 Others Surveys 15) Potential for Growth and Improvement 16) Academic Program Plan

Self-Study Template cont’d Appendices: – Course Level Learning Outcomes – Course Outlines (the most

Self-Study Template cont’d Appendices: – Course Level Learning Outcomes – Course Outlines (the most recent course outline for each course offered over the review period) – Program(s) Complete Calendar Entry – CVs – Program Governance and Unit Rules and Regulations – Results of Current Student Survey – Results of Recent Alumni Survey – Previous Review Results

Self-Study Resources • Self-Study Template – Word Document for use in populating the Self-Study

Self-Study Resources • Self-Study Template – Word Document for use in populating the Self-Study • Table of Contents • Required sections with Headings • Self-Study Tables – Embedded in the template as Word or Excel files • Data provided for Self-Studies – IA&P • Reviewer Nomination Forms

Reflections on the Self-Study • • • Buy-in Review of the program(s), NOT the

Reflections on the Self-Study • • • Buy-in Review of the program(s), NOT the Department or people Length 100 -150 pages + Appendices Data analysis DLEs, Learning Outcomes, Assessment and Curriculum Map Organization – Simple parsimonious language, tightly organized, appropriate to section heading, • Sections on Growth and Improvement and Program Plan (15 and 16) – What is working and why? – Where improvements are needed?

How to Write a Bad Self-Study Experience has shown that nothing is more important

How to Write a Bad Self-Study Experience has shown that nothing is more important to the successful review of any program than the self-appraisal by its members. However nothing is more variable in its quality and effectiveness than that selfappraisal. Ineffective self-appraisals are: q descriptive rather than reflective, analytical, self-critical and evaluative; q loaded with data that is presented rather than analyzed; q defensive or self-justifying rather than aimed at quality improvement; q prepared in a formulaic or mechanical way, as if completing a checklist rather than demonstrating that the members of the program are sensitive to and thinking about the context, mission and objectives of the program; and, q written by the Chair without evidence of buy-in (or sometimes even knowledge) of faculty, staff and students rather than resulting from a participatory self-critical process.

DLEs vs Learning Outcomes • DLEs – Academic standards that identify the knowledge and

DLEs vs Learning Outcomes • DLEs – Academic standards that identify the knowledge and skills graduates are expected to demonstrate – Depth and breadth of knowledge, knowledge of methodologies, application of knowledge, communication skills, awareness of limits of knowledge, autonomy and professional capacity • Learning Outcomes

DLEs vs Learning Outcomes • DLEs • Learning Outcomes – Focus on student learning

DLEs vs Learning Outcomes • DLEs • Learning Outcomes – Focus on student learning and whether certain stated knowledge and skills have been assessed – For academic reviews, we refer to program level learning outcomes and course level learning outcomes to indicate the assessable knowledge, skills and values the graduate will have achieved by the end of the program or course – Curriculum Map

Health Sciences curriculum mapping retreat May 4 2017

Health Sciences curriculum mapping retreat May 4 2017

Contacts • Brian Power- Vice-Provost and AVP, Academic • Rick Guenther or Rico Natale–Institutional

Contacts • Brian Power- Vice-Provost and AVP, Academic • Rick Guenther or Rico Natale–Institutional Analysis • Trish Greydanus – Quality Assurance Coordinator and Secretary to ARC • Centre for Pedagogical Innovation – Madelyn Law – Director – Giulia Forsythe – Associate Director – Jennifer Kopczinski – Educational Developer /Curriculum Specialist

QUESTIONS / DISCUSSION

QUESTIONS / DISCUSSION