The ABCs of CDE Curriculum Development and Evaluation

The ABC’s of CDE (Curriculum Development and Evaluation) Teaching and Learning (B) Leslie, Salma and Grace June 11, 2015

Agenda Introduction Goal & objectives Baseline knowledge assessment Overview of curriculum development & evaluation Three approaches to curriculum development Six-step approach Evaluation Summary/Discussion

Workshop Goal To become familiar with the principles of curriculum development and evaluation

Workshop Objectives Attitude • Appreciate the complexity of curriculum development and evaluation Skill • Be able to actively participate in applying the six steps of curriculum development Knowledge • Identify the core principles of curriculum development and evaluation

Overview of Curriculum Development Definition Curriculum “The means and materials with which students will interact for the purpose of achieving identified educational outcomes. ” Ebert, Ebert & Bentley (2013)

‘Need to Know’ Complex task • Knowledge & skill of curriculum development process • Commitment by faculty, administration, students Three spheres • Formally endorsed curriculum • Informal curriculum • Hidden curriculum Goldenberg, Andrusyszyn, Iwasiw (2004); Bennett, Lockyer, Mann et al. (2004)

Importance of Curriculum Development Fundamental changes in medical practice • Shift from hospital-based to community-based clinical settings • Advances in information technology • Need to encourage interprofessional training • Changes in society’s expectations of physicians Key to long term change in medical education More attention on process and organization of medical education Bligh (1999); Goldenberg et al. (2004); Neufeld, Maudsley, Pickering et al. (1998)

Importance cont’d Curriculum impacts on learning experience • No difference in Board exam scores • Traditional vs innovative approach: • Poor development of moral reasoning skills • Increased stress • Less desirable mode of learning Bligh (1999)

Problem-based Learning Approach (PBL) Definition Educational method which is used to: • • Help learners identify their learning needs Motivate self-directed learning Develop problem-solving & clinical reasoning skills Work effectively in a group Barrows (1985); Walsh (2005)

Advantages • Promote self-directed learning • Motivate learners • Create interest about subject • Facilitate team work Barrows (1985)

Disadvantages • Resource intensive • Lack of traditional structure and progression • Lack of depth of acquired knowledge • Time consuming • Does not fit all learning situations Barrows (1985); Hung (2009); Robert (1997); Bradley-Levine (2014)

How to Build a PBL Curriculum 1. Why? Where? Who? 2. Set the learning goals and objectives 3. Set the stage • The scenario (the problems) • The location • The timing • The equipment (resources) • The actors Reinout (2002); Barrows (1985); Walsh (2005); Stanford Uni (2001)

PBL Curriculum The Process: • The initial activities • Introduce the problem • Self-directed study • Application of the new knowledge • Summary and integration of learning Barrows (1985)

PBL Summary • Helps learners to develop problem solving and teamwork skills • Secret of success: requires unstructured timing • Resource intensive • High recall of content, surface learning

Experiential Learning Approach • NOT at a prescription for how to approach curriculum development • IS a theory about adult learning that can be applied to curricula Li, Greenberg & Nicholls (2007)

What is Experiential Learning? • Adult learning theory developed by David Kolb • Used extensively in many professions • Learners take charge of their learning • Provides opportunity for deep learning • Discovery, self-reflection

4 Tenets of Experiential Learning Theory (ELT) • Focus on PROCESS not OUTCOMES • Holistic approach • All learning is re-learning • Moves between reflecting, acting, feeling, thinking

4 Stages of Experiential Learning Cycle Kolb (1984); Kolb & Kolb (2005)

Characteristics of Curriculum Based on ELT • Learner-centred while respecting professional standards • Variety of learning activities • Time for reflection on experience • Where learning occurs • Reflect on learning experience as well on you as a learner Kolb (1984); Kolb & Kolb (2005)

When is it Appropriate to Use ELT in Curriculum Development? • Adult learning • Need to meet standardized requirements of regulatory body while still addressing learners’ individual needs • Courses where theory and practice must come together to complete the learning process

Challenges of Using ELT in Curriculum Development • Departure from traditional lecture-style teaching • Can be met with resistance • More difficult to manage with larger groups • Effort intensive theory to put into practice Kolb & Kolb (2005); Li, Greenberg & Nicholls (2007); Smith, Emmett & Woods (2008)

A Six-Step Approach • Step 1 Problem Identification and General Needs Assessment • Step 2 Needs Assessment for Targeted Learners • Step 3 Goals and Objectives • Step 4 Educational Strategies • Step 5 Implementation • Step 6 Evaluation and Feedback

Advantages & Disadvantages Advantages • Systematically designed • Practical, feasible approach • Generic • Timeless Disadvantages • Concept is a ‘step-wise’ approach

Step 1: Problem Identification and General Needs Assessment A. B. C. D. Problem identification Identify the current approach Brainstorm the ideal solution General needs assessment

Step 1 a: Problem Identification • What is the health or learning problem? • Whom does the problem affect? • What does the problem affect? • How big is the problem?

Step 1 b: Identify the Current Approach • What is currently being done about it? • Is there existing medical curricula? • Does anyone else have the same problems? • How are they resolving them? • Where are sources of possible solutions?

Step 1 c: Brainstorm the Ideal Solution In the perfect world, • How would patients, health care professionals, educators and society solve this problem? • How would you solve this curricular problem?

Step 1 d: General Needs Assessment The difference between the actual and ideal curriculum model is the general needs assessment Patients Current Approach Ideal Approach Health Care Professionals Medical Educators Society

What are the Barriers to Solution? • What are the barriers between current and ideal solution? • Time • Personnel • Money • Knowledge • Community issues

Step 2: Targeted Needs Assessment Describes the learners and their environment • Learners • • • Previous training Existing proficiencies Past and current performance Attitudes Learning styles • Environment • Past or current curricula • Enabling factors and barriers to curriculum development • Stakeholders

Step 3: Set Goals and Objectives Goal = broad educational objective Objective = specific measurable outcomes

Writing Objectives – 5 Elements Who? Will do? How Much? Each resident will demonstrate at least once Of What? By When? of obtaining PAP smear by end of course

Writing Learner Objectives Attitude Skill Knowledge

Competency-Based Curriculum Objectives related to Can. MEDS roles framework

Other Types of Objectives • Process objectives • Outcome objectives

Step 4: Educational Strategies What How Curriculum content Curriculum methods

Guidelines to Choose Educational Methods • Understand the principles of adult learning • Feasibility • Congruence between objectives and educational methods

Congruence Between Objectives and Educational Methods Attitudes Skills Knowledge Exposure Supervised clinical experience Simulations Artificial models Readings Facilitation of reflection Role models Lecture Audiovisual materials Discussion Problem-based learning Programmed learning

Step 5: Implementation • Converts a mental exercise to reality • Resources • Political support • Administration and support staff • Barriers

Piloting and Phasing In • What are some advantages/disadvantages to piloting a curriculum? • What are some advantages/disadvantages to phasing in a curriculum?

Step 6: Evaluation and Feedback • Closes the loop in the curriculum development cycle

Curriculum Evaluation - Definition “The systematic documentation of the consequences of a curriculum and the determination of their worth in order to make decisions about them. ” Green (1977)

Purposes of Curriculum Evaluation • Determine the outcome • Decide the future of the program • Future development • Improve teaching methods • Cost-benefit analysis • Knowledge dissemination Green (1977); Herman (1987); Kern (2009)

What to Evaluate • Learners • Instructors • Curriculum material • Scope and sequence of content • Environment and facilities • The overall curriculum Green (1977)

Types of Evaluations Summative evaluation Formative evaluation • Final feedback • Evaluates overall effectiveness and impact of curriculum • Helps to decide about future of the program • Occasional or continuous feedback • Used to continually improve the curriculum during its implementation Green (1977); Herman (1987)

How to Evaluate - The Evaluation Cycle Rowel (1999)

Who Benefits? • Decision-makers • Employers • Evaluators • Scholars • Students • The profession • Academic setting Green (1977); Herman (1987)

Models of Evaluation • Bradley's effectiveness model (1985) • Tyler's objectives centred model (1950) • Stufflebeam Context, Input, Process, Product (CIPP) 1971 Glatthorn (2015)

Key Principles • Complex process • Begins with clear goals and objectives • Meets needs of learners and communities they serve • Contains evaluation/feedback mechanism • Foundation to successful learning