EDUCATOR EVALUATIONS IN MICHIGAN Carla Howe Educational Evaluation

EDUCATOR EVALUATIONS IN MICHIGAN Carla Howe Educational Evaluation Consultant Office of Evaluation, Strategic Research, & Accountability Michigan Department of Education

Evaluation is a Collaborative Process Research tells us that evaluation systems work best when evaluation is not something we do to people, but rather when it is a shared process meant to provide feedback for learning and growth.

Positive Outcomes How can this new legislation and change be a positive for educators and students? • Will provide measurement of progress toward goals. • Allows teachers and administrators to focus on best practice and continuous improvement. • Helps reaffirm the profession by having a system in place to recognize excellence. • Helped secure Michigan’s approval of ESEA Flexibility which waived many of the unattainable targets within No Child Left Behind.

PRINCIPAL & ASSISTANT PRINCIPAL TRAINING GRANTS

Principal/Assistant Principal Training Grants – 2012 -13 SY • $1. 75 million allocated for Principal and Asst Principal Training for Conducting Educator Evaluations • Applications for Training Programs were accepted in August • Final Review of Programs is complete • MEGS+ opened in November for grant applications by district; closes December 7 • Awards in January/Funds through School Aid

Approved Training Programs A Framework for Teaching: Supporting Professional Learning (Lenawee ISD) Supporting Teacher Growth Through Evaluation (Kent ISD) Charlotte Danielson’s Framework for Teaching Proficiency Test Instrument (Teachscape) Teacher Evaluation System(s) CUES Model (Midcontinent Research for Education and Learning) Clarkston Community Schools Educator Evaluation Program (Clarkston Community Schools) Teacher Evaluation System(s) Standards-Based Model (Mid-continent Research for Education and Learning) Effective Evaluation for Educators (Jackson ISD) The Five Dimensions of Teaching and Learning (The University of Washington, Center for Educational Leadership) Evaluation Collaboration and Feedback Training to be Consistent and Support Teachers (Airport Community Schools) The Marzano Observation Protocol (Marzano Research Laboratory) Educator Evaluation: Together We Make Each Other Better (Michigan Association of Secondary School Principals MASSP) The Thoughtful Classroom (Silver Strong & Associates) Great Lakes Bay Instructional Leadership Series for Principals and Teacher Leaders (Bay-Arenac ISD) Training for Observers/Evaluators (Imlay City Community Schools)

EDUCATOR EVALUATION SYSTEMS & EFFECTIVENESS LABELS

Overview of Current Plan and Issues • November release date for the aggregate effectiveness labels by school (number of teachers reported as highly effective, minimally effective, ineffective) • Key important messages: –This is the FIRST YEAR; 800+ different systems (we have data to show this) –Districts did MASSIVE amounts of work to accomplish this –We do not believe that huge numbers of MI teachers are ineffective • Concerns –Issues with data privacy –Contextualizing the labels

Current Circumstances Our current legislation has allowed for local systems of evaluations, which has given districts flexibility to design systems that work best for them. • Over 800 systems across the state • Varying degrees of implementation across the state Public reporting of effectiveness labels is required by SFSF • Public Release in November via mischooldata. org • Teachers labels reported in aggregate by school (number of teachers in each of the four categories) • Principals/Administrators reported at the district level.

Important Context for the 2011 -12 Results • First year of implementation of NEW systems based on student growth measures • State provided student growth measures are only available in grades 4 -8 for reading and mathematics • Varying components across systems (i. e. between districts) • Varying percentages of growth across systems (i. e. between districts) • Some districts on prior contract (i. e. No new system, but reporting labels was required) • www. michigan. gov/educatorevaluations for supporting documentation

K-12 Educator Evaluation Survey • 792 districts completed the survey about their Evaluation systems from April to August • Required to be completed by SFSF • Results provide valuable insight into local systems • The types of frameworks used • The % of student growth as a component (law states “significant”, but it isn’t defined until 2013 -14) • Types of growth measures included • Types of decisions informed by the results of evaluations

# of districts 54 districts with a prior contract did not have to incorporate growth or a new system in 2011 -12 50% of reporting districts Other Frameworks reported include: Charlotte Danielson Framework AND a local component, Teacher Advancement Program, My Learning Plan, 5 Dimensions of Teaching and Learning, Local District or ISD framework, Mc. REL, STAGES, Kim Marshall Rubrics PRELIMINARY/DRAFT FINDINGS

# of districts Appropriate given the FIRST year of local evaluation systems

# of districts Others types of decisions include: Assignment to committees or roles beyond the classroom, classroom support and assistance, layoff/recall/transfer, mentoring, staff placement, scheduling, setting improvement goals, merit pay

Other Factors Reported As Part of Evaluations Absenteeism rate Professional responsibilities Classroom management Content knowledge Student achievement data Adherence to School Improvement Plans Instructional practices Commitment to School & District Use of technology Learning environment Pedagogical knowledge and Parental practice communication/involvement Professional development Relationships with students participation PRELIMINARY/DRAFT FINDINGS

Overview of Statewide Results UNDERSTANDING EDUCATOR EVALUATION LABELS IN MI

Caveat…. • Labels are not EQUAL across districts • However, we know that people will want this type of analysis and we want it done appropriately • Resources to assist with communications: –Policy brief “facts and figures” –MDE Power. Point

Statewide Results Label Number Percent Ineffective 775 0. 82% Minimally Effective 1, 998 2. 11% Effective 70, 742 74. 74% Highly Effective 21, 141 22. 33% IMPORTANT NOTES: • Based on the labels as determined by the local evaluation system; rigor of label designation is not consistent across districts • THERE is differentiation in label reporting now, 22% of teachers are reported as “highly effective” moving away from a satisfactory/unsatisfactory system • We do not believe that 1% of teachers labeled as “ineffective” is unreasonable in the first year

Impact of growth • Law required districts to implement systems based in “significant part” on student growth • How do the labels look different when the district used growth in greater percentages?

Growth and eval labels Less than 10% 11 -20% 21 -30% 31 -40% 41 -50% >50% State 0. 6% 1% 0. 5% 2% 1. 4% . 82% Min 1% Effective 2% 3% 6% 3. 5% 1% 2% Effective 83% 71% 74% 68% 82% 74% Highly 15% Effective 26% 23% 30% 27% 15% 22, 639 25, 309 5, 650 3, 143 1, 485 87, 575 Ineffective 26, 253 LESS differentiation without growth More differentiation in labels when growth counts at a higher rate

Distribution of Labels By Percent of Evaluation Based on Growth 90 83. 24 80 72. 55 67. 71 70 60 50 40 30 24. 3 20 26. 82 15. 1 10 0. 59 0. 84 1. 08 1. 07 2. 31 4. 39 0 Ineffective Minimally Effective Less than 10% 11 -40% Effective Over 40% Highly Effective

Takeaway (for messaging too) • We believe the distribution of labels (i. e. number of teachers in each category): –Is appropriate in Year 1 of implementation –Reflects differentiation (esp highly effective/effective) • BUT we also see that systems using higher proportions of growth are making those differentiations • The statewide evaluation system will move us toward more growth measures at higher rates

Who is more likely to be rated as highly effective or effective? Teachers more likely to appear in highly effective category (versus other three) and in effective category (versus other two): • Those with more time in the same district • Teachers with a professional certificate (as opposed to all others) • Those with a master’s degree or higher • Teachers in districts with growth over 40% in their system

Relationship between effectiveness labels and Priority/Focus/Reward Important to remember: • A school-level designation does not mean that all teachers within that school are in a given level of effectiveness • Example: In a Priority School, there will be effective teachers as well as ineffective teachers

Ineffective Minimally Effective Highly Effective Priority 2. 5% 6. 3% 74% 18% Focus 0. 5% 1. 5% 80% 18% Reward 0. 5% 0. 85% 74% 24% Statewide 0. 8% 2% 74% 22% Notes: There are significantly more teachers reported as ineffective and minimally effective in Priority Schools than the statewide number, and in Focus or Reward schools.

Key Takeaways from the Results 1) These results are reasonable for the first year; represent a huge effort on the part of districts 2) There is differentiation in the system; there will be more as growth becomes a higher component; but we still do not believe large numbers of Michigan teachers are “ineffective”

MCEE & THE LOOK AHEAD

MCEE Interim Progress Report Timeline for MCEE Recommendations Estimated Timeline for Completing Recommendations Month/Year Recommendation June 2012 • Observation tool(s) • Details regarding the 2012 -2013 pilot year July 2012 • Other components of teacher evaluation systems October 2012 • Student growth model November 2012 • Evaluation tool for school administrators • Details regarding the pilot of administrator evaluations • District waiver processes and principles April 2013 • Professional certificate June 2013 • Review all recommendations and adjust based on new data and information

MCEE’s Interim Progress Report Michigan Council for Educator Effectiveness (MCEE) issued its Interim Report on April 27, 2012. MCEE concluded a pilot study is imperative, saying rushing to develop a system "would be reckless, both fiscally and technically”.

Interim Assessment Timelines Grade Level(s) and Content Area Fall 2012 Spring 2013 Fall 2013 Spring 2014 Fall 2014 Spring 2015 K-2 Reading Cog Lab Item writing and committee reviews Pilot test (fixed form) Field test CAT Fully operational CAT Pretest Fully operational CAT posttest K-2 Writing Cog Lab Item writing and committee reviews Pilot test (fixed form) Field test CAT Fully operational CAT Pretest Fully operational CAT posttest K-2 Mathematics Cog Lab Item writing and committee reviews Pilot test (fixed form) Field test CAT Fully operational CAT Pretest Fully operational CAT posttest

Interim Assessment Timelines Grade Level(s) Fall 2012 and Content Area Spring 2013 Fall 2013 Spring 2014 Fall 2014 Spring 2015 High School Reading MMC course-based descriptor revisions and SBAC Interim Plans Item writing and pilot recruitment Pilot test (fixed form) Field Test (CAT) High School Writing MMC course-based descriptor revisions and SBAC Interim Plans Item writing and pilot recruitment Pilot test (fixed form) Field Test (CAT) High School Math MMC course-based descriptor revisions and SBAC Interim Plans Item writing and pilot recruitment Pilot test (fixed form) Field Test (CAT) Grades 3 -8 Science Item writing and pilot recruitment Pilot test (fixed form) Field Test CAT pretest Operational CAT posttest High School Science Item writing and pilot recruitment Pilot test (fixed form) Field Test CAT pretest Operational CAT posttest Grades 3 -8 Social Item writing and Studies pilot recruitment Field test (fixed form) Fully operational fixed form pretest Fully operational fixed form posttest High School Social Studies Field test (fixed form) Fully operational fixed form pretest Fully operational fixed form posttest Item writing and pilot recruitment

TEACHER-STUDENT DATA LINK (TSDL)

Teacher/Student Data Link: What is it? • Data initiative to link each student + assessment data to the courses he/she took and to the teachers who taught those courses • Still fairly “new; ” only in its 2 nd year of operation • Will mature in the coming years to be able to provide measures and information over time. • The “TSDL” provided by BAA is not the same as the TSDL collection, rather info from the TSDL collection + assessment results.

Student Information Course Student Assessment Data Teacher Information Student Assessment Data Student Information Course Teacher Information Student Assessment Data Teacher-Student Linked Assessment File (From BAA Secure Site)

Teacher/Student Data Link: Why is it important? • Required under State Fiscal Stabilization Fund (SFSF) as a deliverable • Required as a compliance factor in the NEW School Accountability Scorecards for 2012 -13: • Will be an “all or none” component, combined with the REP effectiveness labels reporting requirement. • At least 95% of your students must be included in your TSDL submission.

Information About the TSDL Assessment File • Functionality of the produced file is extremely limited, so districts choose which “pieces” make sense in their local context. • Generated for each educator of students in tested grades, regardless of subject taught. • BUT “growth”, or Performance Level Change (PLC), exists only for reading and mathematics for MEAP and MI-Access FI in grades 4 -8

How does the TSDL Work? • Teachers are linked to courses • Students are linked to courses • For each course taught, a teacher has a list of students who were reported as taking that course. • Spring assessment data 2011 and fall assessment data 2011 will attribute to teachers from the 2010 -2011 school year • “Feeder school” for fall assessment data

Linking Assessment Data to Students Once teachers are linked to students, the TSDL file provides: • Performance level change (PLC) for MEAP and MI-Access FI in reading and mathematics for each teacher where available (regardless of subject taught) in grades 4 -8. • Performance level in writing, science, social studies, reading and mathematics for each teacher where available (regardless of subject taught) across all tested grades.

Performance Level Change – Michigan’s Growth Model Year X Grade Y MEAP Performance Level Year X+1 Grade Y+1 MEAP Performance Level Not Proficient Partially Proficient Adv Low Mid High Mid M I I SI SI SI D M I I SI SI SI High D D M I I SI SI Partially Low Proficient High SD D D M I I SI SD SD D D M I I SI SI Low SD SD SD D D M I I SI Proficient Mid SD SD D D M I I High SD SD SD D D M I Advanced Mid SD SD SD D D M Low Not Proficient Mid

Access to TSDL data • TSDL User role must be established in the Secure Site to access the data at the district or school level • Spring Assessments/High school link available through the Secure Site as of January. • Fall Assessments (Elementary and Middle) TSDL through the Secure Site as of March.

Working with the TSDL Assessment File • District/school performs roster verification on the TSDL file • District/school needs to adjust each list based on rules like: • • student attendance subject taught match grade taught other local factors

Using PLC Data with MDE Tool • This year, the TSDL provides PLC data linked to teachers to districts for integration into local systems along with an optional tool. • These are general guidelines/suggestions— NOT requirements for reading and math in grades 4 -8

Sample Components of an Educator Evaluation Growth Component Contribution to Evaluation (as Shown Above): 20%

8 SOUND PRACTICES FOR CONDUCTING EVALUATIONS

8 Sound Practices for Conducting Evaluations 1. Develop goals/purpose/guiding principles for the evaluation system. 2. Establish clear business rules to ensure consistent practice for all. 3. Include multiple measures for the growth component. 4. Share the system, the rules, the expectations with teachers early on. 5. Be open to feedback and possible revisions. 6. Designate someone to act as principal while you conduct a formal observation. 7. Plan for the unexpected. 8. Follow through.

1. Develop goals/purpose/guiding principles for the evaluation system • Ensure that all educators know why the work is being conducted – those creating and implementing it AND those that are the recipients of its results • Consider guiding principles for all arms of the evaluation system • Helps to calibrate the work – Does this (change, rule, etc. ) align with our goals and purpose? • Helps provide a sound “defense” of criticisms • Facilitates decision making

2. Establish clear business rules to ensure consistent practice for all • Business rules ensure transparency in the system itself • By naming practices, everyone is aware of the process and how it should be unfolding • Holds all stakeholders accountable to the practices within the system • Requires follow through and direct communication

3. Include multiple measures for the growth component • The law requires multiple measures – and for good reason! • National and state assessments • Local assessments • Portfolio evidence • Determine the measures you’d like to include at the beginning of the year. • Put your teachers’ expertise to work in determining grade/subject specific practices/nuances.

4. Share the system, the rules, the expectations with teachers early on • This is good instructional practice that you would expect of your teachers in their classrooms -- model this for them. • Everyone has a clear idea of what to expect, how the work will be done, how the results will be used, and next steps for both evaluator and evaluatee. • No surprises! • Eases anxieties!

5. Be open to feedback and revisions • If what you expected to happen didn’t, look into why. • Is there a problem with the system, the instrument, or the measures? • Are the results just different than what you expected and nothing ought to change OR is there something that might need revision? • Ask questions of to understand your teachers thinking/beliefs about the system • Listen for what might be underlying their responses -- is there something you hadn’t considered? Might there be good reason to look twice?

6. Designate someone to act as principal while you conduct a formal observation • Allows you to focus, to be present • Promotes respect • Shows the teacher that you are committed to conducting a fair, thorough observation • Allows you to see their practice, value their work, and provide the basis for meaningful (even if difficult) conversation afterward • Builds trust -- he/she will be willing to listen to you, knowing that you observed and thought about their work Perception is reality sometimes – so think of it like you are being observed, too – teachers are checking in to see if you’re really paying attention or if you’re just there because you have to be – and they know how to spot that better than anyone.

7. Plan for the unexpected • Be realistic about planning observations & the pre- and post- meetings • Set up a pre-observation meeting – or conduct it via email. • I’m going to be conducting my formal evaluation of your class in the coming weeks. Please share with me three goals that you are currently working on: a teaching goal, a content goal, and a professional practice goal. . . • Give a few possible times/class hours and dates for the formal observations so that if anything urgent comes up that you CANNOT put off, you haven’t let down the teacher and given an indirect message that the teacher and his/her observation was not important.

8. Follow through • • Schedule the follow-up time when you schedule the observation so that it happens. Plan your post-observation in the same way that a teacher plans a lesson—be prepared with the content. This will help you be more direct, efficient, professional. • The candid feedback is what teachers want, but if the postobservation meeting occurs so far after the observation, you are less able to provide precise feedback about his/her practice. • Schedule for a set amount of time and stick to it. • If you ask for goals or purpose in the pre-conference, follow up on those in the post conference.

Elements of an Effective Evaluation System Allow employees to take responsibility for their own learning, growth and performance by answering three critical questions: 1. Where am I right now in my learning and performance? 2. Where should I focus next to learn, grow and improve? 3. How should I proceed to reach that next level of performance?

Questions? Contact Carla Howe Office of Evaluation, Strategic Research, & Accountability • howec 2@michigan. gov • 517 -241 -2884 • www. michigan. gov/educatorevaluations