Report to Oklahoma Teacher and Leader Effectiveness Commission

Report to Oklahoma Teacher and Leader Effectiveness Commission Prepared By: Southern Regional Education Board Andy Baxter & Tysza Gandha January 29, 2015 Oklahoma City

Section 1 Preliminaries SREB TLE Focus Groups | January 29, 2015 | Baxter 2

Our charge To gather feedback from Oklahoma teachers, principals and central office staff on the design and implementation of Oklahoma’s Teacher and Leader Effectiveness system in the fall of 2014. SREB TLE Focus Groups | January 29, 2015 | Baxter 3

Where we went 10 locations over period September 22 -Oct 3, 2014 • • • Tulsa Ponca City Pryor Stigler Antlers • • • Woodward Elk City Lawton Ada Oklahoma City SREB TLE Focus Groups | January 29, 2015 | Baxter 4

Who talked to us 131 participants • 71 administrators • 60 teachers Representing 58 districts Balance of elementary (47), middle (30), high (31) school teachers (excludes central office staff) Balanced across subject areas Teachers with less than 5 years experience were underrepresented. SREB TLE Focus Groups | January 29, 2015 | Baxter 5

How they were selected • OSDE asked superintendents to nominate up to 3 people from a variety of roles • Nominees received the invitation to register from OSDE. SREB TLE Focus Groups | January 29, 2015 | Baxter 6

What we asked • Which aspects of TLE are meaningful and beneficial to you? • Which concern you? • Which need more clarification for you? • In many districts, the quantitative measures are new to educators. What do you think would make the transition easier for schools? • How do you think the system of multiple measures of effectiveness is better or worse than what you had before? • TLE is designed to give educators useful information about their strengths and areas for improvement. From your experience so far, how is TLE helping or hurting schools? See Appendix A of the report for the full protocol (p. 18). SREB TLE Focus Groups | January 29, 2015 | Baxter 7

Caveats to the findings • While the experiences of the focus groups were strikingly similar, we do not know the extent they are representative of other teachers. • Hereafter, whenever we say “educators, ” this means “most educators who attended the focus groups. ” SREB TLE Focus Groups | January 29, 2015 | Baxter 8

Section 2 Findings SREB TLE Focus Groups | January 29, 2015 | Baxter 9

1. The qualitative component is a significant improvement over the old system, but implementation is nascent and uneven. • Both Tulsa and Marzano rubrics were cited for deepening the conversation about instruction. • Evidence-based conversations helpful to both observers and teachers. • Helpful feedback depended greatly on the principal. • Further work is needed to align professional development to the type of teaching indicated by the rubric. • Many, but not all, principals reported difficultly finding the time needed to conduct the observations with fidelity. SREB TLE Focus Groups | January 29, 2015 | Baxter 10

Consider: • Addressing time concerns with training or sharing of best practices. • Reducing the observation load of principals in their first 1 -2 years of the system. • Rethinking the “train-the-trainer” model. Try to create ways to reach educators directly. • Creating ways that educators can share with each other successful TLE practices and learn from each other, especially across districts. SREB TLE Focus Groups | January 29, 2015 | Baxter 11

2. Many educators do not understand one or more of the quantitative components. • Many do not understand value-added. • Most teachers have not seen a value-added score, many times at the discretion of the district. • Most principals did not feel comfortable explaining valueadded to their teachers. • SLO training had not happened at time of focus groups. • Widespread fear of gaming SLOs and the OAMs. • Equally widespread confirmation of actually gaming the OAMs. SREB TLE Focus Groups | January 29, 2015 | Baxter 12

Consider: • Increasing training on value-added and exposure to the scores. It’s largely an unknown in many places. • Providing more training on what OAMs and SLOs should look like when done well. • Thinking about ways to verify the quality of the OAMs and SLOs. • With all of the quantitative data, emphasizing the importance of trends rather than snapshots. SREB TLE Focus Groups | January 29, 2015 | Baxter 13

3. Educators believe the quantitative measures are invalid, unfair, and unhelpful. • Many believe that Oklahoma’s standardized tests are not valid measures of student learning. • They believe value-added creates a two-tier system in which those in tested grades and subjects have a much more difficult time receiving high ratings than others. • They believe these measures, especially value-added, are punitive. • They believe that the focus on the quantitative measures could detract from the more positive potential of the qualitative component to improve their instruction. SREB TLE Focus Groups | January 29, 2015 | Baxter 14

Consider: • Delaying the high-stakes use of the quantitative measures until educators have had more experience with them. • Continuing to have educators implement and see their progress on the quantitative measures. Until the measures mature, use them as information for reflection by the principal and the teacher. • Using the value-added measures to think through policies beyond individual personnel evaluation (e. g. , teacher preparation, equity plans, professional development). • Dropping the qualitative/quantitative distinction of the components. For example, both the Marzano and Tulsa frameworks include sections about the teacher’s use of data. SREB TLE Focus Groups | January 29, 2015 | Baxter 15

4. Implementation of TLE has been undermined by educators’ severe distrust of OSDE. • Educators had strong feelings that OSDE was working against them rather than for them. • They cited numerous specific examples of times they were brought to the table by OSDE and then ignored. • Some of the distrust was created by frustration with inconsistency from OSDE re: deadlines, instructions on various processes, etc. • “Terms and conditions” SREB TLE Focus Groups | January 29, 2015 | Baxter 16

Consider: • Acting on ways to increase trust between OSDE and educators. A broad range of possibilities here. • Creating a small council of educator advisors who can test and respond to instructions that OSDE sends to the field (i. e. , betatesters). • Focusing on strategies to improve customer service including frequently asking for feedback and acting on it. • Annually surveying educators about their perceptions and experiences of TLE implementation. • Reporting regularly to educators and public what you are learning about the implementation of the TLE system. SREB TLE Focus Groups | January 29, 2015 | Baxter 17

Section 3 Our two cents SREB TLE Focus Groups | January 29, 2015 | Baxter 18

Oklahoma in context • The measures (rubrics, value-added, SLOs) are similar to those in many other states • Some of the frustrations are common across many states • OSDE has fewer people working on this than many states • “Train-the-trainer” models seem not to work in most places SREB TLE Focus Groups | January 29, 2015 | Baxter 19

TLE intersects with other education policy areas • Not wanting to evaluate because of teacher shortage • Conflation with A-F accountability • Compensation • The rules about student accommodations and testing SREB TLE Focus Groups | January 29, 2015 | Baxter 20

Where is Oklahoma right now? Paralyzed by fear Disequilibrium Productive Zone Limit of tolerance Threshold of learning Avoid the work Fine-tuning Time Source: Ronald A. Heifetz and Donald C. Lauire. “Mobilizing Adaptive Work: Beyond Visionary Leadershiip. ” in Jay Conger, Gretchen Spreitzer, and Edward E. Lawler III, eds. The Leader’s Change Handbook; an Essential Guide to Setting Direction and Taking Action (New York: John Wiley & Sons, 1998). SREB TLE Focus Groups | January 29, 2015 | Baxter 21

A word of thanks To the teachers, principals and administrators who gave their time to provide feedback to improve the TLE system. SREB TLE Focus Groups | January 29, 2015 | Baxter 22

Contact: Andy Baxter Vice President for Educator Effectiveness Southern Regional Education Board andy. baxter@sreb. org SREB TLE Focus Groups | January 29, 2015 | Baxter 23