TLE Value Added Training AGENDA Introduction Value Added

TLE Value Added Training

AGENDA Introduction Value Added as a Student Growth Measure What is Value Added? Building Conceptual Understanding about Value-Added Results Interpreting the Teacher Report Interpreting the Administrator Report Training on Value Added

What is value added q “Value added” is a measure of a teacher’s contributions - the amount of value he or she adds to students’ academic growth. One of multiple measures that will be used to evaluate the performance of teachers and administrators in Oklahoma’s TLE evaluation system. q Designed to isolate a teacher’s contribution from other factors that might affect a student’s scores but that are outside the teacher’s control. q

Alternative Measures of Student Growth Single-year outcomes (“snapshot” measures) Cohort-to-Cohort Comparisons Single grade-level cohort comparison from one year to the next

Single-year outcomes (“snapshot” measures) Average Grade 4 Reading Scores Minimum performance standard Teachers

Single-year outcomes (“snapshot” measures) % of students who scored proficient or higher Teachers

Shortcomings of Single-Year Outcomes Not a level playing field for teachers - Potential unfair advantage/disadvantage based on past performance of students

Shortcomings of Single-Year Outcomes Not a level playing field for teachers - Potential unfair advantage/disadvantage based on past performance of students Masks progress - No credit for students who make significant progress but fall even slightly below minimum proficiency standards

Shortcomings of Single-Year Outcomes Not a level playing field for teachers - Potential unfair advantage/disadvantage based on past performance of students Masks progress - No credit for students who make significant progress but fall even slightly below minimum proficiency standards Value added… - Accounts for prior achievement - Allows teachers to attain high ratings regardless of where their students started

Cohort-to-Cohort Comparisons Class Average Grade 5 Math Scores for Ms. Smith

Shortcomings of Cohort-to-Cohort Comparisons Measuring different sets of students - Differences in performance could be largely related to inherent differences between student groups

Shortcomings of Cohort-to-Cohort Comparisons Measuring different sets of students - Differences in performance could be largely related to inherent differences between student groups Value added… - Tracks the performance of the same group of students over time

Single-cohort comparison from one year to the next Previous Grade vs. Current Grade Average Math Scores for Mr. Jordan’s 5 th Grade Students

Shortcomings of Single Cohort Comparisons Single cohort year to year comparison based on proficiency targets: - Directly compares scores from two different tests - Does not take into account factors outside the teacher’s control

Shortcomings of Single Cohort Comparisons Single cohort year to year comparison based on proficiency targets: - Directly compares scores from two different tests - Does not take into account factors outside the teacher’s control Value added… - Estimates student growth by comparing actual performance and typical peer scores on the same test - Accounts for factors unrelated to teacher performance that may be related to student achievement

Value Added as a Student Growth Measure Levels the playing field by accounting for the impact of unequal prior achievement levels Highlights student progress and credits teachers for student gains Controls for factors unrelated to teacher performance Does not rely on direct comparison between two different tests

AGENDA Introduction Value Added as a Student Growth Measure What is Value Added? Building Conceptual Understanding about Value-Added Results Interpreting the Teacher Report Interpreting the Administrator Report Training on Value Added

• Coach John Robinson • Wilson High School • Win-Loss Record: 5 -9

Wilson High School Home of the Tigers 12 0: 05 12

Wilson High School Home of the Tigers 12 0: 00 18

• Players’ stats, win-loss record, observations from games and practice. • How can he isolate the impact of his program from other factors and get an overall picture of its effectiveness on a team full of different players?

• Players’ stats, win-loss record, observations from games and practice. • How can he isolate the impact of his program from other factors and get an overall picture of its effectiveness on a team full of different players? • Value added measures are designed to help answer these types of questions!

Examples factors that may impact players’ performance: 1. 2. 3. 4. 5. Performance last season Attendance at practice Years experience Weightlifting stats Enrollment in summer football camp

Weightlifting Stats Performance last season Years of experience Attendance at practice Enrollment in a summer camp

Andy’s actual sophomore year performance Estimated “typical performance” of similar players

Wilson High School Home of the Tigers 18 0: 00 12

What is Value Added? Wilson High School Home of the Tigers 18 0: 00 12

AGENDA Introduction Value Added as a Student Growth Measure What is Value Added? Building Conceptual Understanding about Value-Added Results Interpreting the Teacher Report Interpreting the Administrator Report Training on Value Added

Key Concepts Actual Score • The actual score a student received on the state assessment taken close to the end of instruction. Typical. Peer Score • The score estimated by considering the achievement of a student’s most similar peers in the state based on multiple prior assessments and other background characteristics. Value. Added Result • The difference between the average actual score achieved by a teacher’s students and the average typical-peer score.

Meet Ms. Alero Grade 5 Math Teacher Red River Elementary School Peabody Public Schools

OCCT Math Scores for Ms. Alero’s Student Names Actual OCCT Score Grade 4 5 James 850 830 Matthew 545 540 Olivia 645 635 Ricardo 700 730 Yolanda 495 515 AVERAGE Typical. Peer Score Grade 5

James - Actual score on grade 4 OCCT math assessment: 850 - Actual score on the grade 5 OCCT math assessment: 830 - Typical-peer score on grade 5 math OCCT assessment: …?

At present, the student characteristics used to determine typical-peer scores will include: q. Prior Achievement q. Free/Reduced Lunch Status q. Limited English Proficiency q. Individualized Education Plan (IEP) q. Race/Ethnicity q. Gender q. Mobility q. Prior attendance

All OK grade 5 students

Prior Achievement Relationship 900 OCCT Grade 5 Math 850 800 750 700 650 600 550 500 450 400 450 500 550 600 650 700 750 OCCT Grade 4 Math 800 850 900

Prior achievement Free and Reduced Lunch Status Mobility Limited English Proficiency Prior Attendance Race/Ethnicity Gender 810 815 800 805 820 825

OCCT Math Scores for Ms. Alero’s Student Names Actual OCCT Score Grade 4 Actual OCCT Score Grade 5 Typical. Peer Score Grade 5 James 850 830 800 Matthew 545 540 530 Olivia 645 635 640 Ricardo 700 730 710 Yolanda 495 515 500 650 636 AVERAGE

Ms. Alero’s Value-Added Result 650 636 Value. Added Result = +14

True or False

True or False Value added is designed to isolate a teacher’s contributions to student academic growth, or value added, from other factors that might affect a student’s scores but that are outside the teacher’s control.

True or False During the two-year value added pilot, teachers of all grades and subjects will receive a value-added result. False. Only teachers with at least 10 tested students who meet requirements for inclusion in the model in the following grades and subjects will receive a valueadded result: • Reading and Math (Grades 4 -8) • Algebra I • Geometry • Algebra II • English III

True or False The value-added model relies on a direct comparison of two different tests in order to show a teacher’s contribution to a student’s academic growth. False. The value-added model does not relay on a direct comparison of two different tests to estimate a teacher’s contribution to a student’s academic growth. A teacher’s value-added result is determined by comparing the average actual score and the average typical-peer score on the same test.

True or False The 2014 -15 school year value-added results are based on pilot data from the previous school year. True. Value-added results are based on pilot data from the previous school year.

AGENDA Introduction Value Added as a Student Growth Measure What is Value Added? Building Conceptual Understanding about Value-Added Results Interpreting the Teacher Report Interpreting the Administrator Report Training on Value Added

Interpreting the Teacher Value-Added Results Report The teacher value-added results report aims to… Provide unique insight into a teacher’s contribution to student academic growth Provide clarity on how valueadded results are estimated

Mr. Thompson’s Value-Added Results Report q Overall value-added results section q Subject-specific section(s), including Ø Value-added results for the subject Ø Relative Performance Graph Ø Student Performance vs. Teacher Value-Added Result Comparison Graph Ø Student Roster Ø Value-added by Subgroups Table

Unique Student Count (overall) The number of students overall. Each student is only counted once even if taught in multiple subjects. Student Equivalent (overall) The sum of students' dosages overall. Value Added TLE Component Score A teacher's overall value added (above) given by the TLE component score is a weighted average of the teachers' subjectspecific component scores (below), where the weight is the number of student equivalents for the subject.

Unique Student Count (subject-specific) The number of students in a subject. Student Equivalent (subject-specific) The sum of students' dosages in a subject. Average Actual Score The average of the actual scores a teacher's students received on the state assessment taken at the end of the course. Average Typical-Peer Score The average of the scores achieved by the “typical peers” of a teacher's students throughout the state. These peers are similar based on scores earned on multiple prior assessments and other background characteristics. Value-Added Result The difference between the average actual scores a teacher's students earned and the average scores achieved by their typical peers throughout the state.

Mr. Thompson’s Value-Added Result 650 644 Value. Added Result = +6 For more information, visit the video module “Building Conceptual Understanding about Value-Added Results”

Relative Performance Graph This graph shows how a teacher's value-added result and corresponding TLE component score compares to the performance of all Oklahoma teachers who received value-added results in this content area. Confidence Interval Value-added results are measured with some uncertainty. The confidence interval is the "margin of error" for the valueadded result and indicates that the value-added result is likely to fall in the shaded range with 80 percent confidence.

. Student Performance vs. Teacher Value-Added Result This graph shows a teacher's value-added result and the performance of that teacher’s students relative to the statewide average on the state assessment. VALUEADDED RESULT

% of Year The portion of the year the student was enrolled in the course a teacher taught in this subject. Student Roster The roster includes students who contribute to a teacher's value-added result for the subject. Students who are not eligible to be included in the value-added model are excluded from this list. For example, students must have valid pre-test and post-test scores to be included. The roster is based on data from roster verification, when available. Dosage % of Instruction The amount of The percentage of instructional time a responsibility for teacher spent with a instruction a student. This is equal to teacher was the percent of assigned for the instructional time student during the multiplied by the percent period he or she of the year with a teacher. was enrolled in the Dosage is used to weight course. students in the valueadded calculation. . x = Progress vs. Proficiency Comparison Graph Student Equivalent Value-Added Result

Prior Achievement Based on student test scores from the prior grade and year. For grades 4 through 8 math, Algebra I, Geometry, or Algebra II, this is another mathematics assessment. For grades 4 through 8 reading, or English III, this is another reading/ELA assessment. % of Year x % of = Dosage Instruction Progress vs. Proficiency Comparison Graph Prior Achievement Unique Student Count Student Equivalent Value-Added Result 7 *** Indicates that there is insufficient data to yield a result for this metric. Vale Added Result For more information, visit the TLE page of the OSDE website (http: //ok. gov/sde/tle) and view the videos explaining the teacher and administrator value-added results reports or contract the OSDE Customer Service Desk.

Mr. Thompson’s Value-Added Results Report q Overall value-added results section q Subject-specific section(s), including Ø Value-added results for the subject Ø Relative Performance Graph Ø Student Performance vs. Teacher Value-Added Result Comparison Graph Ø Student Roster Ø Value-added by Subgroups Table

AGENDA Introduction Value Added as a Student Growth Measure What is Value Added? Building Conceptual Understanding about Value-Added Results Interpreting the Teacher Report Interpreting the Administrator Report Training on Value Added

An administrator’s value-added results report aims to… Provide some insight into how instruction at the school has contributed to the student academic growth Provide clarification about how the results were calculated

Subjects included in school value-added results Reading 4 -8 Math 4 -8 Algebra II Algebra I Geometry School valueadded results English III

School valueadded results… • Take into account the number of students by subject and teacher • Not a simple average of the value-added results of all teachers in a school • Each full-year student is counted the same, rather than giving less weight to some students

Dr. Clark’s Value-Added Results Report • Overall Value. Added Results Section • Subject-Specific Value-Added Results Sections

Unique Student Count The number of students overall. Each student is only counted once even if taught in multiple subjects. Student Equivalent The sum of students' dosages in the subject. Value-Added VALUE- Result TLE COMPONENT STUDENT UNIQUE ADDED An administrator's school SCORE EQUIVALENT STUDENT RESULT COUNT value-added result is a weighted average of the value-added results of teachers in the school, where the weight is the number of student equivalents for each teacher.

Interpreting the Administrator Value-Added Results Report • Overall Value. Added Results Section • Subject-Specific Value-Added Results Sections