Paola Borin Deb Dawson Florida Doci Donna Ellis

Paola Borin Deb Dawson Florida Doci Donna Ellis Lori Goff Jill Grose Sandy Hughes Erika Kustra Ken Madows Peter Wolf ISSOTL – October 2014 Documenting and Transforming Institutional Teaching Cultures

Session Overview • Review background on the political terrain impacting HE and teaching culture • Consider indicators of teaching culture • Our project on teaching culture perceptions • Next steps and conclusions 2

The Political Terrain in Ontario Productivity and Innovation , s c i r t e m , s c i Metr s c i r t e m Stren g t henin c e e r n e t f g Dif innov res of cr eativ ation ity and k nowl edge Strategic Mandates n o i t a nti 3

The value place on that institutions teaching is important We aim to contribute to to us fostering an institutional culture that values quality teaching 4

What do we mean by teaching culture? • Organizational culture = Deep structure of an organization, rooted in organizational members’ values, beliefs, and assumptions (Denison, 1996) • We believe that one fundamental way to ensure quality teaching is to foster an institutional culture that values teaching 5

Political terrains? • In small groups, introduce yourselves and discuss the political terrain impacting in your region 6

Indicators of teaching cultures Brainstorm a list of measures or indicators that might be suggestive of a teaching culture. Instructions (Part A) • One idea per post-it note please! • As many post-it notes as possible! Instructions (Part B) • Post each idea • Arrange post-it notes into themes 7

Teaching Culture – Initial Elements Broad engagement around teaching occurs Teaching is recognized in strategic initiatives and practices Infrastructure exists to support teaching Assessment of teaching is constructive and flexible Faculty are encouraged to develop as teachers http: //www. oecd. org/edu/imhe/QT%20 policies%20 and%20 practices. pdf 8

Overall Project Focus Evidence & enhance institutional teaching culture at post-secondary institutions through direct feedback from constituents and key institutional indicators in order to provide concrete feedback and recommendations for continuous improvement 9

Anticipated Project Outcomes • Develop a survey instrument that identifies and provides evidence of prevailing perceptions regarding the teaching culture among key stakeholders – the Teaching Culture Perception Survey (TCPS) • Identify key institutional indicators to triangulate and confirm teaching culture • Develop a report template that institutions would receive following the completion of the inventory • Develop a recommendation package to help institutions choose practices that enhance their teaching culture 10

The Consultative Process • Presented to Educational Developers Caucus for feedback (EDC; winter 2013) • Shared idea with Council of Ontario Universities (COU; summer 2013) • Applied for Government funds (MTCU-PIF; fall 2013) – received $175, 000 • Presented to Council of Ontario Educational Developers for feedback (fall 2013) • Updated COU; requested support (fall 2013) • Updated EDC on project progress (winter 2014) • Presented at the Society for Teaching and Learning in Higher Education (Summer 2014) 11

Phase 1 – Pilot Study – Teaching Culture Perception Survey • Aimed at examining the perceptions to develop a profile, allowing comparison between different stakeholders’ perceptions & comparison of change over time. • Questions designed to identify and validate indicators of quality teaching culture on campus. • Indicators are signals that reveal the progress or lack of progress towards a specific objective. (Chalmers 2008) 12

Phase 1 – Pilot Study – Identification and validation of indicators • Input: Resources involved in supporting an institutional program, activity or service. • Output: Reflect the quantity of outcomes, including measurable results and direct consequences of the activities implemented. (Burke, 1998) • Outcome: Focus on the quality of educational program, activity and service benefits for all stakeholders. (Warglien & Savoia, 2001) • Process: Means used to deliver educational programs, activities and services within the institutional environment. (Burke, 1998) 13

Phase 1: Pilot Study–Teaching Culture Perception Survey (TCPS) • Assessment of agreement and importance ratings 14

Phase 1 –Indicators • Examining the perceived existence (agreement ratings) and importance (importance ratings) of certain indicators related to quality teaching to develop a profile of the culture in the institution. • Triangulation of perception and facts to determine deviation from actual facts. • Identification of other indicators that speak to the culture of quality teaching. 15

Phase 1 – TCPS Lever 1: Teaching is recognized in institutional strategic initiatives & practices (10 items) …departmental administrators convey that effective teaching is a priority. …evidence of effective teaching is considered in the evaluation of faculty members’ job performance (e. g. , tenure, promotion, annual evaluations). Lever 2: Assessment of teaching is constructive and flexible (7 items) …students are encouraged to provide ongoing feedback to their teachers throughout their courses. …programs are evaluated based on student learning outcomes. 16

Phase 1 – TCPS Lever 3: Faculty are encouraged to develop as teachers (8 items) …educators are encouraged to use evidence about teaching to inform their teaching practices (e. g. , literature, communities of practice, personal reflection). …educators are encouraged to adopt a variety of teaching and learning approaches. Lever 4: Infrastructure exists to support teaching (6 items) …educators can get financial support to develop their teaching (e. g. , grant programs, teaching conferences). …learning spaces such as classrooms, labs, and/or studios are designed to facilitate learning 17

Phase 1 – TCPS Lever 5: Broad engagement around teaching occurs (8 items) …students are often included in discussions about teaching. …external stakeholders such as employers and community members are involved in initiatives that foster effective teaching across the institution. 18

Phase 1 – TCPS & Focus Groups • Two versions of the survey • Students: Graduate & undergraduate • Faculty & Administration • Pilot survey at: • Windsor (921 participants) • Western (1589 participants) • Mc. Master (1334 participants) Undergraduate Students 20% Graduates Students Facultyand & Adminsstrators Administration 42% 38% • Focus groups ran to collect feedback on the survey and input on quality teaching culture at each institution. 19

Faculty PRELIMINARY RESEARCH FINDINGS Agreement Importance 1) Recognizing effective teaching (3. 57) 2) Broad involvement around teaching (2. 94) 3) Encouraging effective teaching (2. 91) 4) Assessing teaching (2. 60) 1) Recognizing effective teaching (4. 08) 2) Assessing teaching (4. 00) 3) Encouraging effective teaching (3. 82) 20

PRELIMINARY RESEARCH FINDINGS Agreement Importance Undergraduate Students 1) Accessing 1) Prioritizing effective Infrastructure (3. 70) teaching (4. 44) 2) Recognizing effective 2) Accessing teaching (3. 69) infrastructure (4. 32) 3) Implementing effective Teaching (3. 21) teaching (4. 30) 4) Broad involvement 4) Providing feedback on around teaching (2. 96) teaching (4. 24) 5) Broad involvement around teaching (3. 76) 6) Recognizing effective teaching (3. 66) 21

Pilot Study Qualitative Results Noted indicators of a teaching culture that values teaching: Students: Faculty: • Current and supported best practices • Professors’ behaviour • Passion • Teacher accessibility • Valid assessment tools • Implementation of student feedback • Promotional incentives for teaching • • Support for teaching Infrastructure Research above teaching Teaching evaluations 22

Next Steps… TCPS üRevise levers as well as survey items ü For example, separate 1 st lever “Teaching is recognized in institutional strategic initiatives & practices” into “Institutional strategic initiatives & practices prioritize effective teaching” and “Effective teaching is recognized and rewarded” üChanged item “… effective teaching is clearly defined” to “… effective teaching is clearly defined in institutional documents” 23

Next Steps… TCPS üNeed your feedback on two items… üQ 19 At my institution. . …teaching accomplishments, contributions, and/or awards are publicized and/or celebrated. üQ 33 At my institution…student evaluations of teaching contribute to improved teaching 24

Next Steps… TCPS üDevelop version of TCPS for staff who support teaching and learning üValidation of surveys üEstablish norms for the surveys 25

Next Steps… üExpand focus of research (e. g. , outside Ontario, include colleges) üSecure funding (e. g. , SSHRC Insight Grant) üPrioritize and evaluate institutional indicators of a positive teaching culture üDevelop manual with recommendations to improve institutional teaching cultures 26

Conclusions… ü Goal of fostering institutional cultures that values quality teaching, which, ideally, translates into improved teaching & learning ü Creating means for institutions to evaluate their teaching culture (TCPS, indicators), determine areas for improvement, and use recommended strategies to realize improvements ü Will institutions adopt framework & dedicate resources to improve their teaching cultures? 27

Questions…? ? ? Lori Goff: lgoff@mcmaster. ca Florida Doci: docif@uwindsor. ca Ken Meadows: kmeadow 2@uwo. ca Webpage: qualityteachingculture. wordpress. com 28

References Burke, J. C. (1998). Performance funding indicators: Concerns, values, and models for state colleges and universities. New Directions for Institutional Research, 97, 49 – 60. Chalmers, D. (2008). Teaching and Learning Quality Indicators in Australian Universities, Conference proceedings of Institutional Management in Higher Education (IMHE), Paris France, September 8 -12. Denison, D. R. (1996). What is the difference between organizational culture and organizational climate? A native’s point of view on a decade of paradigm wars. Academy of Management Review, 21(3), 619 – 54. Hénard, F. & Roseveare, D. (2012). Fostering quality teaching in higher education: Policies and practices. France: Organization for Economic Cooperation and Development. Warglien, M. , & Savoia, M. (2001). Institutional Experiences of Quality Assessment in Higher Education - The University of Venice (Italy). Organisation for Economic Cooperation and Development (OECD). 29