From Needs Assessment to Action Planning 2019 Institute

From Needs Assessment to Action Planning 2019 Institute for New Faculty Developers at UNC Greensboro Thursday, August 1, 10: 45 am – noon Cassandra Volpe Horii, Ph. D cvh@caltech. edu

Definitions & Workshop Outcomes Needs Assessment: Action Plan: A systematic process for determining needs, or "gaps, " between current and desired conditions Steps and strategies to be used in service of meeting goal(s), which may emerge from needs assessment ● Apply evidence-based needs assessment approaches to your unique context In the middle - Defining goals and outcomes - Assessment and evaluation ● Prepare to move from needs assessment insights to action plans (and actions) with ease

“Critical engagement requires questioning, forming and challenging opinions, and feeling outrage or inspiration. It is about helping individuals find their voices and learn to trust their instincts. And it is about teaching the value of what [people] know and encouraging them to use their knowledge in the service of their academic, personal, social and political lives. ” – Amy Scharf, Critical Practices for Anti-bias Education

1. Opening Reflection PROGRAM or SERVICE CENTER CHANGE INSTITUTIONAL Individual, Departmental, Institutional… People Context Evidence Heuristic adapted from Felten, Little, Ortquist-Ahrens, & Reder, 2013

Discussion PROGRAM or SERVICE CENTER CHANGE INSTITUTIONAL Individual, Departmental, Institutional… People Context Evidence Heuristic adapted from Felten, Little, Ortquist-Ahrens, & Reder, 2013

2. Frameworks and Approaches to Needs Assessment Touchstone Story 1: Curry College

Curry College Faculty Center Website

Individual Faculty; Accreditation Seek External Funding Union Negotiations; Collaboration; Trust Student Data; External Benchmarks Four Rs: Wright, Lohe, Pinder-Grover, and Ortquist-Ahrens, 2018

2009 -10 Academic Year: A survey of faculty professional development needs and interests, open focus groups held at multiple campus locations at a variety of times, and extensive discussion with the Excellence in Teaching Committee shaped ideas for a proposal to the Davis Educational Foundation to help start a Faculty Center. Summer and Fall, 2010: Grant proposal was prepared and submitted in late September 2010. The grant proposal was shared and discussed with the Excellence in Teaching Committee again. October 2010: Davis Educational Foundation conducted a site visit, including input from co-chair of the Excellence in Teaching Committee. December 2010: Grant was awarded announced to the Curry College community. January 2011: Renovation plans were started, involving the Dean’s Office, Buildings and Grounds, and Finance. Grant materials were made available on the Faculty Portal and by email on request. February 2011: Excellence in Teaching hosted a lunch discussion about ideas for the Faculty Center. Participants responded to preliminary plans—for faculty positions, space, and programs. A summary was shared by email and online, inviting further input. This helped shape the specific call for interest in the Faculty Coordinator position. March 2011: All members of Faculty Standing Committees on Excellence in Teaching, Educational Technology, and the Undergraduate Curriculum Committee were invited to participate on the selection committee. All who desired to participate did so. . .

2. Frameworks and Approaches to Needs Assessment Touchstone Story 2: Caltech

Caltech Center for Teaching, Learning, and Outreach Website

Teaching Quality; Research Grants Minimal Infrastructure Skepticism; Close community; High-functioning/ time efficient Recent Needs Assessment for Outreach Four Rs: Wright, Lohe, Pinder-Grover, and Ortquist-Ahrens, 2018

Quick start! (within about a month of arriving) Initial goals: A. Offer focused programs and services that increase faculty, TA, and student experience with innovative and effective teaching and learning methods. B. Shape the specific mission, programs, timeline, and operational plans for the new center, with broad-based input from faculty, TAs, students, and staff.

Parallel Tracks IMMEDIATE EDUCATIONAL DEVELOPMENT NEEDS ASSESSMENT & PLANNING Facilitate access: Consult with faculty: – Start-up website – Methods, courses, curricula, innovations, grant proposals. Enhance existing programs: – Training & specialized workshops for their teaching staff (TAs, others). – – New faculty workshop(s). Provost’s Innovation in Education projects. TA orientation, development, and resources. Course ombuds and peer tutors. Expand TA professional development: – Explore needs for group TA training and practice teaching for selected courses; other advanced topics. Plan major events: Support student learning: – Divisional forums. – Caltech-wide events. – Evaluate needs for additional programming on student learning challenges and techniques, diversity in the classroom, oral presentation, public speaking. . . Promote mid-quarter feedback on teaching: – New resources & workshop(s). Support K-12 Educational Outreach: – Define, advertise, and hire a dedicated educational outreach coordinator.

Relational: Needs Assessment Interviews ● ● ● Brief discussion/introductions - CTLs, Process at Caltech; Initial Goals What is most important, valued, or unique about teaching and learning at Caltech? What is most challenging or problematic about teaching and learning at Caltech? How would you want teaching and learning to be different at Caltech, if at all? How would you like the new Caltech center for teaching & learning to make a difference for (or how would you envision the center serving): FACULTY TAs STUDENTS DEPARTMENTS OTHERS ● Are there any colleagues you could suggest for me to meet with, who might share additional ideas and perspectives?

2. Frameworks and Approaches to Needs Assessment The Four Rs: Guiding CTLs with ● ● Responsiveness Relationships Resources, and Research Wright, Lohe, Pinder-Grover, and Ortquist-Ahrens, 2018

Immediate ←→ Long-term Local ←→ Generalizable Informal ←→ Formal Communication ←→ Withholding Relational ←→ Detached Prioritization←→Exploration Existing data ←→ New data Self report ←→ Direct measure Internal data ←→ External data Four Rs: Wright, Lohe, Pinder-Grover, and Ortquist-Ahrens, 2018

Explore: Immediate ←→ Long-term Local ←→ Generalizable Relational ←→ Detached Prioritization ←→ Exploration Informal ←→ Formal Self report ←→ Direct measure Existing data Internal data ←→ New data ←→ External data

Zooming In… Needs Assessment vs. /and Research Necessary? Desirable? - IRB - Timeline - Pros/Cons Immediate ←→ Long-term Local ←→ Generalizable Relational ←→ Detached Prioritization ←→ Exploration Informal ←→ Formal Self report ←→ Direct measure Existing data Internal data ←→ New data ←→ External data

Zooming In… Interpreting surveys Immediate ←→ Long-term Local ←→ Generalizable Relational ←→ Detached Prioritization ←→ Exploration Informal ←→ Formal Self report ←→ Direct measure Existing data Internal data ←→ New data ←→ External data

Identifying both high priority and low knowledge: Item % Lower Knowledge % High Priority p-value(1) Teaching effectively 62. 3% 57. 4% . 6015 Providing constructive feedback to learners 73. 8% 47. 5% . 0061* Using effective assessments 86. 9% 45. 9% <. 0001* Enhancing small group teaching 74. 6% 45. 8% . 0041* Using emerging technology in the classroom 88. 5% 44. 3% <. 0001* Selecting appropriate teaching methods 83. 6% 41. 0% <. 0001* Enhancing my classroom teaching 73. 3% 40. 0% . 0016* Designing courses 70. 5% 27. 9% <. 0001* Developing an educational portfolio 91. 8% 26. 2% <. 0001* Behar-Horenstein et al. 2014

Zooming In… External data / benchmarks Immediate ←→ Long-term Local ←→ Generalizable Relational ←→ Detached Prioritization ←→ Exploration Informal ←→ Formal Self report ←→ Direct measure Existing data Internal data ←→ New data ←→ External data

External Frameworks: Educational Development Faculty Development Instructional Development Organizationa l Development current + future individuals cohorts courses curricula student learning structures systems leadership podnetwork. org/about-us/what-is-educational-development

External Frameworks

External Frameworks

External Data Please indicate three primary purposes that guide your program (% of respondents who listed each in top 3) 2016

External Data HERI: Eagan et al. 2016

External “Curriculum” Principle 1: Establish and support a class climate that fosters belonging for all students. Principle 2: Set explicit student expectations. Principle 3: Select course content that recognizes diversity and acknowledges barriers to inclusion. Principle 4: Design all course elements for accessibility. Principle 5: Reflect on one’s beliefs about teaching to maximize self-awareness and commitment to inclusion.

Discussion:

3. Action Plans Needs Assessment: Action Plan: A systematic process for determining needs, or "gaps, " between current and desired conditions Steps and strategies to be used in service of meeting goal(s), which may emerge from needs assessment In the middle - Defining goals and outcomes - Assessment and evaluation

3. Action Plans Common elements: - Goals & Intended Outcomes - Assessment & Evaluation plan - Sequence of steps and tasks (including contingencies and prerequisites) - Timelines - Assignments (if multiple people are involved) - Approach to coordination and communication

3. Action Plans REALITY CHECKS: ● ● Can the plan be carried out at a high level of quality? Do you have the time/money/expertise required? Are too many needs being addressed at once? Could your action plans exceed the participation capacity or “bandwidth” of your participants?

Scenario 1: Program/Service Upper administration views and program data are at odds! - In what way(s) is this scenario a “needs assessment”? - What is important to consider in your action plan? Scenario 2: Center Department chairs are on fire about an idea! Your needs assessment data adds another perspective. . . - How would you go about action planning in this case? Scenario 3: Institutional Change The president’s priority is not shared by faculty! - What does your action plan need to address in a situation like this?

Synthesis and Discussion