Performance Metrics Improving Undergraduate Science Education at UBC

Performance Metrics Improving Undergraduate Science Education at UBC Canadian Council of Deans of Science Conference Simon Peacock June 2008 FACULTY OF SCIENCE

UBC’s Carl Wieman Science Education Initiative “Science education is a science in itself, and a scientific approach can be applied to optimize its delivery and impact. ” CWSEI is an evidence-based approach to improving undergraduate science education (as opposed to approaches to teaching based on tradition or anecdote) Classroom studies Brain research Cognitive psychology FACULTY OF SCIENCE http: //www. cswei. ubc. ca

Research on Learning Results from Traditionally Taught Courses Retention of information from lecture: 10% after 15 minutes lecture: Gain in conceptual understanding: 25% Beliefs about physics and problem solving: significant drop Courses Using Research-Based Pedagogy Retention of information from lecture: More than 90 % after 2 days Gain in conceptual understanding: 50 -70% Beliefs about physics and problem solving: small improvement FACULTY OF SCIENCE

Evidence-based Scientific Approach to Education • Establish what students should learn - measurable learning goals • Scientifically measure what they are actually learning – e. g. , Force Concepts Inventory used in Physics • Implementation of teaching methods aimed at maximizing achievement – e. g. , peer instruction, personal response systems, interactive computer simulations • Scholarly dissemination and adoption • Sustainable and continued optimization FACULTY OF SCIENCE

Success requires: • Organizational change (an unnatural act) • Change at the course and departmental program level • Faculty changing the way they teach – Personality and self-awareness challenges – Cultural challenges (dept course vs. my course) • Substantial resources to effect transition FACULTY OF SCIENCE

CWSEI Investments • Funding is based on Department proposals • Most departments have opted to use Science Teaching and Learning Fellows (STLFs) • Developing clear measures of student learning (assessment of educational outcomes) • Creating and testing new educational methods and materials • Developing faculty expertise in effective education FACULTY OF SCIENCE

Performance Metrics • How many of our first-year courses specify (and distribute to students) clearning outcomes? • Longer term goal - specify learning outcomes for all courses and programs • Student retention rates and graduation rates (cohort studies) – Reasons for students leaving (e. g. , academically ineligible) – Careful analysis of inflows and outflows • Student evaluations – Review teachers with low “effectiveness” scores and require improvement plan – Focus groups and on-line evals provides rich feedback FACULTY OF SCIENCE

Performance Metrics • Scrutinize courses with anomalous failure rates and/or anomalous grade distribution. Possible factors contributing to low success rates include: – Insufficient resources – Students are insufficiently prepared (prerequisites? ) – Course is poorly taught (instructor, course activities) – Course expectations are too high* – Means of assessing performance are flawed* FACULTY OF SCIENCE

In Development • Benchmarking of course (and lab) requirements in specific programs with comparable programs at peer universities • Number of students with substantive out-ofclassroom educational experiences: e. g. , - Co-op programs (18%) - Research experiences, Honours theses - Exchange program, Service learning, … • Number of courses taught by contract faculty • Number of faculty who do not teach undergraduates in a given year FACULTY OF SCIENCE