Introduction to ACE August 2014 For ACE Observers
Introduction to ACE August 2014 For ACE Observers and TNTP Instructional Staff © TNTP 2013
Objectives By the end of this session, you will: 1 Understand the purpose of assessing teacher effectiveness 2 Understand each measure used in the Assessment of Classroom Effectiveness (ACE) 3 Understand the process and tools used in ACE Observations 4 Understand ACE scoring and potential outcomes /2
Agenda Why We Care About Effective Teaching Understanding the Assessment of Classroom Effectiveness (ACE) Adding It Up: ACE Scoring /3
Nationwide, poor and minority students receive a second-rate education, with unequal access to the resource that matters most: great teachers. By the end of high school, African-American and Hispanic students read and do math at virtually the same level as eighth-grade white students. On the 2011 National Assessment of Educational Progress (NAEP), only 14% of African-American eighth-graders and 21% of Hispanic eighth graders scored at the proficient or advanced levels in math. In urban areas, half of students won’t graduate at all, closing the door to a college degree or a middle-class living. In a nation dedicated to equality and opportunity for all, poor and minority students still receive an unequal education, undermining the hopes of millions to compete for a better life. /4
Even one year with a highly effective teacher has a lifelong impact on students. Great teaching changes lives. /5
Agenda Why We Care About Effective Teaching Understanding the Assessment of Classroom Effectiveness (ACE) Adding It Up: ACE Scoring /6
Future teachers join us because they want to make a real difference for high-need students. We use ACE to help them become the best teachers they can be. We are committed to helping teachers make a difference from day one, but we have a responsibility to ensure that they remain on track to become great teachers. TNTP Academy uses the Assessment of Classroom Effectiveness (ACE) to create the fullest possible picture of all teachers’ performance using multiple measures. ACE is more than an evaluation system; it is designed to support rapid growth by ensuring that teachers always know how they’re doing and what they need to do to improve. We use ACE not only to assess teacher growth but to customize our training to meet unique developmental needs. At the end of the year, teachers must pass our ACE screen in order to be recommended for certification. Earning certification under ACE is a high bar and an honor. /7
Through ACE, we strive to create the fullest possible picture of our teachers’ performance using multiple measures. • Research shows classroom observations are valid predictors of teacher effectiveness. Classroom Observations • Teachers will be observed at least 4 times, with performance rated according to the ACE Instructional Framework. Principal Ratings • Principals will rate teacher performance relative to other new teachers they have worked with throughout their careers. • The survey uses research-tested questions predictive of teacher effectiveness. Student • If teachers work with very young students (grades K-2) or with students Surveys in moderate-to-severe special education settings, they will not administer student surveys. • The best measure of effectiveness is the academic growth of one’s students. Student Achievement • Where available, we will incorporate data from state-produced value. Data added reports to analyze teachers’ ; actual impact on student learning over the course of the school year. /8
ACE helps teachers be more successful by providing regular insights about their practice. Impact of Teacher Evaluation on Student Math Scores • Pre-evaluation Post-evaluation Personalized Support Teachers receive at least 4 ACE observations annually and get detailed reports shortly thereafter. • Student Survey reports are released in early spring, so teachers can respond to that feedback within the year. • We use ACE data to customize training and coaching to address your highpriority development needs • We share ACE data with principals to help teachers access relevant PD resources through their school and district. Research shows that teachers improve when they receive quality feedback through evaluations. Taylor, E. & Tyler, J. (2011). The effect of evaluation on performance: Evidence from longitudinal student achievement data of mid-career teachers. NBER Working Paper # 16877. Cambridge, MA: National Bureau for Economic Research. /9
Measure 1: Classroom Observations WHY? HOW? WHEN? • Research shows that rigorous observations of teachers at work are valid predictors of teacher effectiveness. • Trained ACE Observers will use our ACE Instructional Framework to evaluate teacher performance. They will be looking for hallmarks of effective teaching that boost student learning. • Teachers will receive at least 4 ACE observations, starting in October/November. • Each observation will be scored in 4 performance areas, producing an average score for the observation session. • All observations will be announced. • Observation ratings and feedback can give you useful insights on your strengths and weaknesses. NOTE: ACE observations are separate from nonevaluative observations conducted by Academy staff or any observations conducted by your district/school-based mentors or • Teachers should receive a report shortly after each observation. • Observations will last 4560 minutes or one full class period. • Each month, teachers will need to input a copy of their teaching schedule into our online tracking system so we can schedule observations. / 10
After each observation, teachers will get a report showing their scores in four performance areas, along with an overall observation score. PERFORMANCE AREA SCORE Student Engagement 3 Essential Content 2 Academic Ownership 2 Demonstration of Learning 3 OVERALL SCORE FOR THIS OBSERVATION: 2. 50 ACE observation scores will be calculated by averaging the competency scores for each competency, with the overall scores aligning to the performance levels below. Skillful Proficient Developing Minimally Effective Ineffective 4. 30 – 5. 00 3. 60 – 4. 29 2. 80 – 3. 59 2. 00 – 2. 79 1. 00 – 1. 99 / 11
Measure 2: Principal Evaluation Measurement for Principal In the spring, principals will rate. Evaluation teacher performance relative to other firstyear teachers he or she has worked with in the past. Performance will be rated on a 5 -point scale: 1. 2. 3. 4. 5. Much worse than other first-year teachers Worse than other first-year teachers About the same as other first-year teachers Better than other first-year teachers Much better than other first-year teachers Principal evaluations under ACE are entirely separate from school-based evaluations or school/district renewal decisions. / 12
Measure 3: Student Surveys WHY? HOW? WHEN? • Research from the Gates Foundation’s Measures of Effective Teaching (MET) study demonstrates that student surveys are a reliable indicator of teacher effectiveness. • Students in grades 3 -12 will take paper-based surveys. • Student surveys will be administered between February and March 2015. • Our student survey uses questions shown to be predictive of teacher impact on student learning. • Student surveys give voice to the people who have the most direct experience with teachers – the children in their classrooms every day. • Teachers working with very young students (K-2) or in moderate-tosevere settings will not administer the survey. “The average student knows effective teaching when he or she sees it. ” (“Learning about Teaching: Initial Findings from the Measures of Effective Teaching, ” Gates Foundation, December 2010) / 13
How the student survey process works 1. ADMINISTER SURVEY Students complete an anonymous, paper-based survey. The survey administration takes approximately one class period and will designed to ensure confidentiality. 2. GENERATE SCORE 3. IMPROVE PERFORMANCE Survey questions are grouped into three broad categories: Care, Control, and Challenge, with responses rated on a 5 point scale. Teachers receive formative score reports detailing their students’ perceptions. This data provides valuable insights into your strengths and weaknesses as seen by the students in your classrooms every day. Student responses will be averaged to generate a composite score in each of these categories, and those three category scores will then be averaged to generate the overall survey score. Student Survey 1– 5 Survey Score / 14
Calculating the survey score. Harry Potter Hermione Granger Ronald Weasley Cho Chang Neville Longbottom Lavender Brown Padma Patil AVERAGE CATEGORY SCORES STUDENT COMPOSITE SCORES BY CATEGORY Care Control Challenge 4 4 2. 75 3. 75 4 2. 5 4. 25 3. 5 3 4 3. 75 2. 75 4. 25 3. 25 2. 75 4. 07 3. 79 2. 82 TEACHER’S OVERALL SURVEY SCORE: 3. 56 Teacher’s overall survey scores map to the performance levels below. Skillful Proficient Developing Minimally Effective Ineffective 4. 50 – 5. 00 4. 00 – 4. 49 3. 50 – 3. 99 3. 00 – 3. 49 2. 99 or lower / 15
Past participants valued student survey feedback, indicating that it would help them improve their teaching. CHALLE NGE CARE CONTRO L • “I was glad to see my students recognized the value of the homework I assigned, that they believe that I want them to explain their thinking, and that they knew that I believed that they could get a good grade if they tried. “ • “I deeply cared for my students, but I know that I didn't show them or tell them enough. This is definitely something I will focus on next year. “ • “[After reviewing survey results, ] I am going to give the kids more input into the behavior system in my classroom, in an effort to give them ownership of their own actions as individuals and as a class. ” / 16
Measure 4: Student Achievement Data We believe that the best measure of teachers’ effectiveness is the academic growth of their students, and we are committed to using student achievement data in ACE when possible. Where available, we will use data drawn from state value-add reports to assess teacher impact on student learning. Value-added models emphasize growth rather than absolute proficiency, recognizing that students arrive with varied levels of preparedness, but all are capable of learning. Value-added models also account for factors beyond teachers’ control, including students’ past academic performance and student characteristics. When value-added reports are released by the state, we will use the ratings provided by the state to map teacher performance to our ACE rating scale. / 17
Agenda Why We Care About Effective Teaching Understanding the Assessment of Classroom Effectiveness (ACE) Adding It Up: ACE Scoring / 18
Performance on each ACE measure will be mapped to the performance scale below, earning between 1 and 5 points for each ACE component. Classroom Observations Principal Ratings Student Surveys Student Achievement 1. 00 – 1. 99 points Much worse than 1 point 2. 99 or lower 1 point Ineffective 1 point Minimally Effective 2. 00 – 2. 79 points Worse than 2 points 3. 00 – 3. 49 2 points Minimally Effective 2 points Developing 2. 80 – 3. 59 points Same as 3 points 3. 50 – 3. 99 3 points Developing 3 points Proficient 3. 60 – 4. 29 points Better than 4 points 4. 00 – 4. 49 4 points Proficient 4 points Skillful 4. 30 – 5. 00 points Much better than 5 points 4. 50 – 5. 00 5 points Skillful 5 points Ineffective Each component will be weighted and combined to calculate the final ACE rating. / 19
While ACE considers multiple measures, those measures carry different weights toward the final ACE score. Scenario 1: Scenario 2: Scenario 3: Scenario 4: Classroom Observations Principal Rating Student Surveys Student Achievement Data The weights for each ACE measure will vary, depending on the total number of measures available given one’s specific teaching placement. / 20
Adding it up: Four Scoring Scenarios Scenario 1: Classroom Observations + Principal Ratings Principal Rating Observations (40%) (60%) Scenario 2: Classroom Observations + Principal Ratings + Student Surveys Student Survey Observations s (40%) (30%) Principal Rating (30%) 0. 40 * 3. 15 points = 1. 26 points (Classroom Observations) 0. 60 * 3. 15 points = 1. 89 points + 0. 30 * 3 points = 0. 90 points (Principal Rating) (Observations) + 0. 40 * 4 points = 1. 60 points (Principal + 0. 30 * 2 points = 0. 60 points (Student Rating) Surveys) 3. 49 points ( Final ACE Score) 2. 76 points ( Final ACE Score) / 21
Adding it up: Four Scoring Scenarios, cont’d Scenario 3: Classroom Observations + Principal Ratings + Student Surveys + Student Achievement Data Student Observations Achievemen (25%) t (35%) Principal Rating Student (20%) Surveys (20%) 0. 25 * 3. 15 points = 0. 79 points (Observations) + 0. 20 * 3 points = 0. 60 points (Principal Rating) + 0. 20 * 2 points = 0. 40 points (Student Surveys) + 0. 35 * 4 points = 1. 45 points Scenario 4: Classroom Observations + Principal Ratings + Student Achievement Data Principal Observations Rating (35%) (30%) Student Achievemen t (35%) 0. 35 * 3. 15 points = 1. 10 points (Observations) + 0. 35 * 4 points = 1. 40 points (Student Achievement) + 0. 30 * 3 points = 0. 90 points (Principal Rating) / 22
There are 3 possible outcomes at the end of your first year of teaching. PASS 2. 75 – 5. 00 points • • If teachers pass ACE and complete all program and state requirements, they will be recommended for a Level 1 Certificate. If teachers will have value -added data available in the late summer, their score is provisional at this stage. They will be eligible to teach in 201415 on a Practitioner License. EXTENSION 2. 50 – 2. 74 points • • If teachers fall short of passing, but are on the right trajectory to meet that standard with additional progress, they may be granted an extension year to continue building skills. If teachers will have valueadded data available in the late summer, their score is provisional at this stage. They will be eligible to teach in 2014 -15 on a Practitioner License. REMOVAL 1. 00 – 2. 49 points • If teachers struggle and show limited prospect of improvement, they will not advance to another year of teaching. • While we expect this to be rare, it is necessary. Allowing a chronically struggling teacher to remain in the classroom isn’t good for anyone. • Removed teachers will be able to appeal. • Pending an appeal, this decision is final, even if teacher is awaiting valueadded data. / 23
Once value-add reports are released, there are two possible outcomes for teachers who received provisional ratings in the spring. PASS 2. 75 points or higher • Teachers who pass ACE and successfully complete all program and state requirements will be recommended for their Level 1 Certificate. • Passing ACE is not a job guarantee; ACE is separate from principal or district renewal decisions. EXTENSION 2. 74 points or lower • If teachers fall short of passing, but are on the right trajectory to meet that standard with additional progress, they will continue teaching on a Practitioner License and be reassessed during 2014 -2015. • Being extended through ACE is not a job guarantee; ACE is separate from principal or district renewal decisions. / 24
Recommended for Certification “with Distinction” CELEBRATING EXCEPTIONAL PERFORMANCE Our experience with ACE has shown that while many teachers get off to a strong start, some are truly exceptional. Through ACE, we aim to celebrate these exemplary new teachers by recommending them for certification “with distinction. ” To be recommended for certification with distinction, teachers must demonstrate strong performance across all various ACE measures, earning a cumulative ACE rating in the top 10% of all TNTP Academy participants in Louisiana. If teachers meet this standard, they will receive a special certificate recognizing their achievements. TNTP Academy will also notify principals and district officials about their accomplishments as a first-year teacher. / 25
- Slides: 25