Ongoing Lead Evaluator 2017 2018 Day 1 Todays

Ongoing Lead Evaluator 2017 -2018 Day 1

Today’s Objectives • I can…go back to the basics of APPR and 3012 -d to review my district’s compliance with the rules and regulations. • I can…identify areas where our district may have misconceptions or may wish to revisit decisions.

Warm-Up! • On a scale of 1 -10, where 1 is totally out of compliance and 10 is total compliance, where would you say your district falls when it comes to following the rules and regulations of APPR? • Share with your table.

Corrective Action 3012 -d requires an examination of APPR and score distributions. SED will have the option of imposing a Corrective Action Plan if there are significant discrepancies. Previously, Corrective Action could not impinge on anything that had been bargained. Corrective Action can now be asserted even over things that were bargained.

APPR Audits • 51 districts in NYS • 10 districts in CNY • 4 districts in OCM BOCES NYSED is informing districts this is a review process and should not be viewed as a formal audit but an opportunity to review and improve practice.

APPR Audits • SLO’s • Observation Documents • TIPs/PIPs Let’s take a look at the letter that went out to districts.

Corrective Action 3012 -d requires an examination of APPR and score distributions. SED will have the option of imposing a Corrective Action Plan if there are significant discrepancies. Previously, Corrective Action could not impinge on anything that had been bargained. Corrective Action can now be asserted even over things that were bargained.

APPR Audits A – in the column indicates there was not enough data to make it publically available.

APPR Audits • Some Trends: – Districts that were new Focus Districts tended to be selected. – Districts that didn’t report all of their data tended to be selected. – Districts where there were big discrepancies between State Provided Growth Ratings and the reported Transition Ratings tended to be selected. data. nysed. gov

APPR Reporting • A memo distributed this week indicates districts that do not submit all of the required APPR data are at risk of losing their state aid increases. – This includes original and transition ratings! Checkpoint: Is your district submitting all of the required APPR data?

10, 000 FOOT OVERVIEW

3012 -c Points Structure 100 HEDI Rating total points 60 Observations, Portfolio, Survey 40 Student Performance 20 State SLO/SPG 20 Local SLO

3012 -d Points Structure 100% Teacher Observation/Principal School Visit HEDI Scaled Scores 100% Student Performance HEDI Observation 4 points Student Performance 20 points

3012 -d Matrix for Overall Rating Teacher Observation/Principal School Visit Student Performanc e Highly Effective Developing Ineffective Highly Effective H H E D Effective H E E D Developing E E D I Ineffective D* D* I I *If a teacher is rated ineffective on the Student Performance category, and a state-designed supplemental assessment was included as an optional subcomponent of the Student Performance category, the teacher must be rated Ineffective overall.

3012 -d Matrix for Overall Rating Teacher Observation/Principal School Visit Student Performanc e Highly Effective Developing Ineffective Highly Effective H H E D Effective H E E D Developing E E D I Ineffective D* D* I I *If a teacher is rated ineffective on the Student Performance category, and a statedesigned supplemental assessment was included as an optional subcomponent of the Student Performance category, the teacher must be rated Ineffective overall.

APPR Emergency Regulation Timeline Which APPR Regulation is in Effect? Emergency Period 2015 -16 3012 -C 3012 -D Transition Period 2016 -17, 2017 -18, 2018 -19 3012 -D Full Implementation 2019 -20 and forward 3012 -D

APPR Emergency Regulation Timeline What Needs to be Reported? Emergency Period 2015 -16 Transition Period 2016 -17, 2017 -18, 2018 -19 Full Implementation Emergency Ratings Transition Ratings Original Ratings 2019 -20 and forward During the emergency and transition periods, two ratings will need to be reported for many teachers and principals regardless of 3012 -c or 3012 -d status. Transition ratings can not use any data derived from 3 -8 ELA/Math Assessments or 9 -12 State Provided Growth Scores

STATE PROVIDED GROWTH SCORES

Why Growth? One way to compare student performance on tests across teachers’ classrooms is to observe student proficiency at the end of the year. Based on this information alone, whose students had a better year? ─ Proficiency

Why Growth? By adding in prior performance (blue bars), we can see how much a student grew from last year to this year. With this additional information, now whose students had a better year? ─ Proficiency Engage. NY. org 21

How to Measure Growth? • Test score scales are not • Example: Last month I comparable across had $100 in my grades. checking account. This month I have 85 Euros. • Difficult to measure How much money have growth directly. I gained or lost? • Engage. NY. org Extra credit: I have about $11 or 8€ more this month than last. 22

How to Measure Growth? • Example: Her hair grew one inch last month. • So did his. • Whose hair growth seems more impressive? Engage. NY. org • Starting point matters. • Just knowing how much growth happened may not be enough. Need a system that provides fair comparisons. 23

GROWTH SCORES EXPLAINED

Factors Used to Define “Similar Students” in the Growth Model Grades 4– 8 ELA/Math Teachers and their Principals Similar Student Characteristics Used Academic History: • • Up to three years of student state exam scores, same subject Prior-year test score, different subject Retained in grade Average prior achievement and range around average prior score in student’s class/course (same subject) Student With Disability (SWD) • • SWD (Yes/No) SWD spends less than 40 percent of time in general education setting for the entire day Percentage of SWDs in student’s class/course • English Language Learner (ELL) Economic Disadvantage (Poverty) • • • ELL (Yes/No) New York State English as a Second Language Achievement Test scores Percentage of ELLs in student’s class/course • • Poverty (Yes/No) Percentage of students in poverty in student’s class/course *Additional characteristics may be added in the future as available and approved by the Board of Regents. Engage. NY. org 25

Student Growth Percentiles: True or False? 1. A student with an SGP of 50 performed as well or better than 50 percent of similar students. 2. A student with an SGP of 80 must be proficient. 3. A student with an SGP of 20 grew less than a student with an SGP of 60. 4. The highest SGP that a student can receive is 99. 5. A student with an SGP of 75 got about 75% of test questions correct. 6. Extra credit: A student with an SGP of 55 in math and an SGP of 50 in ELA learned more math than ELA content this year. Engage. NY. org 1. True 2. False 3. False 4. True 5. False 6. False 26

A quiz! • Which of the following teachers gets a SPG Score?

A quiz! – 3 rd Grade Common Branch Teacher with 16 students – 3 rd Grade Common Branch Teacher with 9 students

A quiz! – 3 rd Grade Common Branch Teacher with 16 students – 3 rd Grade Common Branch Teacher with 9 students 3 rd Grade Teachers Do Not Receive SPG Scores because there is no 2 nd grade exam.

A quiz! – 5 th Grade Math Teacher with 45 students

A quiz! – 5 th Grade Math Teacher with 45 students Yes! He/she teaches a course with a 4 th-8 th Grade ELA or Math Assessment and has at least 16 scores.

A quiz! – K-6 Building Principal

A quiz! – K-6 Building Principal Yes! And, if at least 30% of the students are covered by a grades 3 -8 Exam, this principal would not need any SLO.

A quiz! – 9 -12 Assistant Principal

A quiz! – 9 -12 Assistant Principal No. There actually no reporting requirements under 3012 -d for Assistant Principals. The 9 -12 Principal would receive a SPG Score.

A quiz! – Geometry Teacher

A quiz! – Geometry Teacher No. Although the results of the HS Common Core Regents are what are used for the SPG Score for the 9 -12 principal, the teachers of those courses don’t receive SPG Scores themselves.

A quiz! – Self-Contained 12: 1: 1 Teacher with 4 th-6 th graders.

A quiz! – Self-Contained 12: 1: 1 Teacher with 4 th-6 th graders. Maybe. The teacher would need at least 16 scores in the NYSTP to get a SPG Score.

A quiz! • Could a district use a SPG Score as part of the APPR rating for everyone in the building?

A quiz! • Could a district use a SPG Score as part of the APPR rating for everyone in the building? – Sure. The APPR Committee could negotiate that all teachers in the building who don’t get their own SPG Score will get whatever score the principal gets. – Except that during the transition period, SPG scores may not be used in employment decisions, including TIPs and PIPs.

Transition Period • During the transition period, no transition scores or employment decisions, including TIPs or PIPs can be based on State Provided Growth Scores or the 3 -8 ELA/Math Exams. Checkpoint: Does your district have appropriate transition scores in place?

STUDENT LEARNING OBJECTIVES

Student Learning Objectives • Considered the “Other Comparable Measures” of APPR • Used for teachers who do not have a stateprovided growth score, or whose state-provided growth score does not cover at least 50% of their students. • Meant to measure teacher performance based on students demonstrating “one year’s growth in one year’s time. ”

The 50% Rule • 50% of students must be covered. • Like classes are grouped as courses. • Start with the largest course and keep adding until you get to 50%. – All students in the course must be accounted for. • If there is a state assessment at the end of the course, that must be the basis for the SLO.

The 50% Rule HS Math Teacher • Geometry • Math Elective A 23 Ss • Geometry • Math Elective B 29 Ss • Geometry 22 Ss 25 Ss 23 Ss • What is the total number of students? • What is 50% of that total number • Are there like courses? • How many students are in each course? • Starting with the largest courses, get to 50% • What assessment would be used?

The 50% Rule HS Math Teacher • • • Math Elective A 22 Ss Math Elective A 23 Ss Math Elective B 25 Ss Math Elective B 29 Ss Geometry 23 Ss • What is the total number of students? • What is 50% of that total number • Are there like courses? • How many students are in each course? • Starting with the largest courses, get to 50% • What assessment would be used?

The 50% Rule “Test” • With your table group… • Read each of the three teacher profiles. • Identify how their Student Performance Measure will be built under 3012 -d by applying the 50% rule. Checkpoint: Are all of your teachers accurately applying the 50% Rule?

SLO True/False ① A teacher who teaches a mix of Regents and non-Regents courses writes SLOs for Regents courses first. ② All SLOs must provide some kind of baseline data, but this does not have to be a pre-test. ③ If a teacher goes over 50% of his students, he can pick which sections to include in his SLO and which to omit. ④ A teacher can adjust his/her SLO individual targets for students based on interim assessment results throughout the school year. ⑤ It’s common for teachers to have to write an SLO for more than one course/subject. ⑥ Job-alike teachers in a district must have their SLO based on the same assessment. ⑦ Common Core Regents exams have state-provided growth scores.

SLO True/False ① A teacher who teaches a mix of Regents and non-Regents courses writes SLOs for Regents courses first. ② All SLOs must provide some kind of baseline data, but this does not have to be a pre-test. ③ If a teacher goes over 50% of his students, he can pick which sections to include in his SLO and which to omit. ④ A teacher can adjust his/her SLO individual targets for students based on interim assessment results throughout the school year. ⑤ It’s common for teachers to have to write an SLO for more than one course/subject. ⑥ Job-alike teachers in a district must have their SLO based on the same assessment. ⑦ Common Core Regents exams have state-provided growth scores.

All SLOs Must Have • • • Course Population Learning Content Interval of Instructional Time Evidence (baseline/post-assessment) Baseline Data Targets HEDI Scale Checkpoint: Do your teachers adequately complete each of these Rationale sections of the SLO document?

SETTING SLO TARGETS

Plan Approval Lots of Superintendent Attestations: • Back-up SLOs are in place (no need in transition period) • All targets are 1 year’s growth • All targets are reviewed and approved Checkpoint: If your district were to be audited, would you “pass” on these 3 components?

OBSERVATIONS

2 Options for Teacher Observation/Principal School Visit Subcomponent Teacher Observation/Principal School Visit Supervisor At Least 80% Teacher Observation/Principal School Visit Independent Evaluator Supervisor At Least 10% At Least 80% Independent Evaluator At Least 10% Peer Evaluator Not more than 10%

Observable Rubric Subcomponents ● All observable Standards/Domains of the selected rubric must be addressed across the total number of observations. ○ Evaluators may select a limited number of observable rubric subcomponents for focus during a particular observation. ● All non-observable Standards/Domains of the selected rubric may be observed during pre-ob. or post-ob. conferences or in natural conversations between the teacher and evaluator and incorporated into the observation score. ○ Points can not be given based on artifacts, unless that artifact constitutes evidence of an otherwise observable rubric subcomponent (eg: lesson plan viewed during the course of the classroom observation may constitute evidence of professional planning). ● Districts may also decide how to weight the scoring of individual standards within their plans.

New York State Teaching Standards Standard I (Danielson 1) Knowledge of Students and Student Learning Standard II (Danielson 1) Knowledge of Content and Instructional Planning Standard III (Danielson 3) Instructional Practice Standard IV (Danielson 2) Learning Environment Standard V (Danielson 1, 3, 4) Assessment for Learning Standard VI (Danielson 4) Professional Responsibilities and Collaboration Standard VII (Danielson 4) Professional Growth

New York State Teaching Standards Standard I (Danielson 1) Knowledge of Students and Student Learning Standard II (Danielson 1) Knowledge of Content and Instructional Planning Standard III (Danielson 3) Instructional Practice Standard IV (Danielson 2) Learning Environment Standard V (Danielson 1, 3, 4) Assessment for Learning Standard VI (Danielson 4) Professional Responsibilities and Collaboration Standard VII (Danielson 4) Professional Growth

The Observation Portion At least two observations, one of which has to be unannounced. Total amounts, frequency, direction, and communication plans are worked out locally.

The Observation Portion At least one observation has to be completed by the principal or other trained administrator. At least one observation has to be completed by an impartial, independent trained evaluator. This observer cannot be assigned to the same school building as the teacher. *

The Observation Portion If using peer observers: • The district chooses the peer evaluator • The peer evaluator must be trained • The peer evaluator must have been rated as H or E in the previous year

Teacher Observation/Principal School Visit Scoring Ranges Minimum Maximum Highly Effective 3. 50 to 3. 75 4. 0 Effective 2. 50 to 2. 75 3. 49 to 3. 74 Developing 1. 50 to 1. 75 2. 49 to 2. 74 Ineffective 0 1. 49 to 1. 74 ● Each observation/school visit will receive an overall 1 -4 score. ● Once all evaluations are complete, the different types are combined using a weighted average to produce an overall 1 -4 Observation/School Visit score. ● The overall Observation/School Visit score is converted to a HEDI rating based on locally determined ratings consistent with the ranges listed above.

Principals’ “Observation” One observation shall be conducted by the principal's supervisor. A second observation shall be conducted by one or more impartial independent evaluators. This observer may be employed by the district but not assigned to the principal’s building (see next slide).

Principals’ “Observation” A principal’s supervisor (i. e. , the Superintendent) is a district employee and therefore assigned to a different BEDS code as the principal being evaluated—therefore it is possible for the same administrator to serve as both the supervisor and impartial evaluator for the purpose of school visits.

Observations Checkpoint: Do you have good notes on your observations? Are you rating all of the observable standards?

TIP/PIP

Improvement Plans The content of improvement plans will be determined by superintendent rather than through bargaining or negotiations. • Required for ineffective or developing • In place by October 1 st • Include: areas in need of improvement, timeline, assessment of improvement • Should be keeping progress and conclusory notes Checkpoint: Are all of these things happening

Moving Forward • Checking to ensure compliance in district. • Making note of things you might wish to change in the future. • Really checking those SLOs! Our Next Session: November 2 nd (Syracuse) or November 9 th (Cortland)