West Virginia Educator Evaluation Taskforce January 27 2014

West Virginia Educator Evaluation Taskforce January 27, 2014

Welcome • “Progress is not accomplished in one stage. ” ~ Victor Hugo, Les Miserables. • We’ve Come a Long Way! • Thank you for your continued work and dedication!

Agenda Items • • • 2012 -2013 School Year Wrap-up Teacher-Student Data Link Future School-wide Growth Measures Business Rules for School Growth Measures Online System Updates Looking Ahead / Policy 5310

Throughout the Day. . . • Each taskforce member should have a handout at their seat: – Side 1: Policy 5310 Revision Suggestions – Side 2: Parking Lot Issues • Add to this handout as the day progresses. Handout will be collected at day’s end.

2012 – 2013 SCHOOL YEAR WRAP-UP

2012 -2013 Demonstration School Year Number of Educators 50 326 Teachers Principals Councelors 4195

2012 -2013 Classroom Teachers Assigned Progression Counts 874 456 2865 Advanced Intermediate Initial

2012 -2013 Classroom Teachers 94. 8 Percent Completion 90. 4 90 100 80 60 Percent Completion 40 20 0 Self-Reflection Student Goals Evaluations

Comparison Data: Classroom Teachers Percent Completion (to Date) 94. 8 96. 6 90. 4 91. 2 100 80 60 2013 40 20 0 Self-Reflection Student Goals 2014

2012 -2013 Demonstration Sites School-wide Growth 60 51 50 48 48 47 41 40 35 Very Low 30 Typical 20 10 0 High 5 3 Math Reading

2012 -2013 State-wide School-wide Growth 300 250 224 200 254 248 200 188 159 Low 150 Typical 100 50 0 Very Low High 38 27 Math Reading

2012 -2013 Teacher Performance Ratings Distribution 2965 3000 2500 2000 Distinguished Accomplished 1500 Emerging 1000 500 0 Unsatisfactory 684 156 43 Number of Educators

ROSTER VERIFICATION AND THE TEACHER-STUDENT DATA LINK (TSDL)

WEST VIRGINIA DEPARTMENT OF EDUCATION Roster Verification Overview 2014, Battelle for Kids. All © 2014, Battelle for Kids. ©All Rights Reserved.

What is Roster Verification? • It is a way for teachers to verify their class rosters and allow schools and teachers to indicate when there are multiple professionals sharing responsibility for a student’s instruction. • A roster verification process ensures that teachers are accurately linked to the students they teach. • Roster verification will be based on the data that are submitted through the Master Schedule in WVEIS.

Why Do We Need Roster Verification? • To build trust in the system • To get buy-in from teachers – They take an active role in confirming their rosters – They can be sure that only their students are counted as part of their evaluation • To ensure validity of the student growth measures component of the evaluation system (because it measures student growth based on the students to whom the teacher provided instruction)

The Opportunity The Process Is Simple…It’s as easy as 1 -2 -3 1 - Teachers verify which students they taught 2 - Teachers verify during what months they taught those students 3 - Teachers verify the percent of instruction © 2014, Battelle for Kids. All Rights Reserved.

So, what is TSDL? ? • The Teacher-Student Data Link is the mechanism that links the students to teachers through the Roster verification process and then adds the students’ state assessment/growth data. • It is critical that the TSDL is accurate for educator evaluation purposes so that student growth measures can be accurately attributed to the teacher that taught them. • Without an accurate link, evaluation results could be flawed.

Battelle for Kids Video(s)

Potential Additional Uses • HQT • Educator ID Numbers

DETERMINING BUSINESS RULES FOR SCHOOL GROWTH

Policy 5310 Section 9. 1. c • Fifteen (15) percent of the evaluation shall be based upon student growth as measured by the student learning goals. Beginning in the 2014 -2015 school year, for educators who teach in tested grades and subjects (for which two consecutive years of summative assessment data is available), the results shall be established from a student growth percentile based on the statewide summative assessment. The guidelines for calculating the student growth percentile at the classroom level will be developed during the 20132014 school year through meaningful consultation with the Educator Evaluation Task Force and through a pilot program. For all other educators, progress toward meeting the two (2) student learning goals shall be measured using formal and/or teachercreated assessments.

Areas For Consideration • Student Enrollment & Attendance – Establishing rules which govern student enrollment and attendance issues relating to TSDL. • Data Availability – Establishing guidelines for when and how data is collected and used for TSDL. • Teacher of Record – Establishing a consistent and unified definition for use in TSDL and WVBOE policies.

Student Enrollment & Attendance • Considering student enrollment and attendance is important for properly attributing student growth at the school-wide and teacher level. • The school-wide growth measures will be standardized. • The teacher-level measures will rely on roster verification plus standard rules of thumb.

School-Wide Growth Teacher-level Growth Include Student if: 1) Enrollment = FAY (135 days enrolled in school) AND 2) Total Absences < 45 days Exclude Student if: 1) Enrollment Not Equal to FAY OR 2) Total Absences > 45 days Exclude Students w/ Following Enrollment Codes: 1) EO – Out-of-State Placement 2) EM – Mountaineer Challenge 3) EC – Homeschool Education 1) Implement Roster Verification Process 2) Recommendations are made for inclusion/exclusion rules that mirror Schoolwide Growth Rules, but allow for local control

Data Availability • To determine teacher-level growth, the rostering process plus rules regarding enrollment and attendance will be applied. • There will be an opportunity for “appeals” where a teacher could add a student to his/her roster even if he/she was not FAY or absent more than 45 days.

Data Availability • Once the rosters plus the rules have been finalized, the TSDL will be produced. • The TSDL file will be utilized to determine the 15% teacher-specific growth measure based on his/her roster. • To determine the teacher’s score, an average weighted growth score will be calculated.

Data Availability • The average weighted growth score would be calculated for teachers that fit the following criteria: • Teach mathematics, or Reading/Language Arts or both AND • Teach students in grades 4 -11 AND • Teach ten or more students

Average Weighted Growth Score • Student Growth Percentiles (SGPs) will be utilized as the basis for this average (but as we go forward, the growth measure for another assessment can also be utilized). • SGPs are the standard growth metric calculated and utilized for accountability purposes in the state of West Virginia from the general assessment. • By utilizing this metric that is known across the state by teachers and administrators, the weighted average simply utilizes a metric with which educators are familiar. • Additionally, the score relies on the complex and reliable statistics from which SGPs are derived, but does not require anything more than a standard calculator to compute!

So how is the average weighted growth score calculated?

Step 1. The number of students in each growth category (unsatisfactory, low, typical, high) populates the tool from the roster.

Step 1 • The number of students by growth category are also separated out by whether they are proficient or not proficient simply for informational purposes.

Step 2 The number of students in each category is multiplied by the weight associated with each category, resulting in a product.

Step 3 • The products are summed, or totaled.

Step 4 • The summed product is divided by the total number of students, which produces the average weighted growth score.

Step 5 Use the Key to look up the average weighted growth score to get the outcome that will be included in the Teacher Evaluation for the 15% growth component (unsatisfactory, emerging, accomplished, or distinguished).

Teacher of Record • Ohio Department of Education established 3 categories of educator for use in TSDL: 1) Teacher of Record 2) Assigned Educator 3) Contributing Professional

Definitions Each of the following definitions were composed by the Ohio Department of Education. They are offered here as examples only. -Definitions should be discussed & vetted prior to implementation into West Virginia policy.

Definitions Teacher of Record – The educator who is responsible for a significant portion of a student’s instructional time (based on enrollment) within a given subject or course that is aligned to a state assessment; or other relevant assessment in grades/subjects that do not have state assessments.

Definitions Assigned Educator – The educator assigned to a student, usually for HQT assignment purposes. In some cases, this translates into the teacher responsible for assigning a grade. Example: Educator serves as facilitator/monitor for computer-based course work, and also assigns grades.

Definitions Contributing Professional – The educator works with / has responsibility for a student and/or a teacher, and should be specifically linked with relevant students. Examples: Academic Coaches, Literacy Coaches, Remedial / Intervention Specialists.

Teacher of Record Works with Students Responsible for Students and/or Teachers Assigned to Students Identified as HQT Assigns Grades Two Teachers represent the 100% proportion of instructional time in a tested subject One Teacher represents the 100% proportion of instructional time in a Assigned Educator Contributing Professional

Teacher of Record • Recommended that the following ODE business rules are similarly applied in WV: – Teachers remain T. O. R when they host a Student Teacher – Teachers on Extended Leave still include Student Growth Measures – Late Hires still include Student Growth Measures

FUTURE SCHOOL-WIDE GROWTH MEASURES

ACT Work Keys Growth CTE Teacher Evaluation Sherri Nash, Ed. D. CTE Accountability & Support January 27, 2014

CTE Data Profile Federal & State Performance Measures -Completers -ACT Work. Keys® Reading, Math, Locating Information -Educate WV Performance Assessment -Positive Placement Jobs in Field and Continuing Education -Simulated Workplace Balanced Score Card

How Can We Measure Academic Growth in Career and Technical Education Centers and Multi-County Centers?

• Establish baseline ACT Work Keys ® levels using Key Train or WIN pre-assessments conducted in first two weeks of school. • Measure ACT Work Keys ® level growth using the final ACT Work Keys ® assessment results.

What Is ACT Work. Keys ®? ACT Work. Keys is a set of tests that measure the skills people use to learn quickly, solve problems, and develop skills needed for an occupation. ACT Work. Keys is best known for helping educators and employers talk to each other about workplace readiness, as in helping them translate a general statement like “We need more math” into a precise set of teachable skills really used in the workplace.

Work. Keys ® is an integrated system that helps business match the right people to the right positions. Research & Reporting Services Assessment Services Job Profiling Services

Job Profiling Services Working with Work. Keys ® Job Analysis Procedure experienced employees, ACT-authorized job profilers evaluate the 5 tasks, key skills, and 4 4 levels of skills required for specific 3 occupations. organization. Reading for Information Applied Mathematics Teamwork Applied Technology Required skill levels for the job are identified.

The Work. Keys ® Assessments • Are timed. Each test is 45 minutes long • Are multiple choice • Allow calculator use on the Applied Mathematics • Contain all skill levels for each skill

The Reading for Information assessment • Measures the examinee’s skill in reading and understanding work-related instructions and policies in the form of memos, bulletins, notices, letters, policy manuals, and governmental regulations. • Requires the examinee to read passages and answer questions based on actual workplace demands. HANDOUT - http: //www. act. org/workkeys/assess/reading/levels. html Level 3 Level 4 Level 5 Level 6 Level 7

The Applied Mathematics assessment • Measures the examinee’s skill in applying mathematical reasoning to work-related problems. Level 3 Level 4 • Requires the examinee to set up, calculate, Level 5 and solve the types of problems that actually occur in the workplace. Level 6 HANDOUT http: //www. act. org/workkeys/assess/math/levels. html Level 7

The Locating Information assessment • Measures the examinee’s skill in using information presented in workplace graphics such as diagrams, floor plans, tables, forms, graphs, charts, and instrument gauges. • Requires the examinee to locate, insert, compare, and summarize information in one graphic or in a group of related graphics. HANDOUT - http: //www. act. org/workkeys/assess/locate/levels. html Level 3 Level 4 Level 5 Level 6

National Workforce Readiness Credential Gold 1, 128 15. 14% Silver 4, 585 61. 54% Platinum 5 0. 07% Bronze 1, 733 23. 26%

Baseline ACT Work Keys ® WIN: The Career Readiness Courseware – OR KEY TRAIN Curriculum systems that assess and improve ACT Work Keys ® skill levels in the core areas that employers need most.

• ACT Work Keys ® Pre-Assessments • ACT Work Keys ® Levels Curriculum • Contextual Assessments

CTE Teacher Evaluation 40% 35% 30% h Growt 25% 20% 15% 10% 5% 0% Level 3 Level 4 Level 5 Level 6 Level 7

Questions? Thank You! Sherri Nash, Ed. D. sherri. nash@access. k 12. wv. us

Early Childhood Education Looking at school-wide growth for the early childhood grades: • 5% school wide growth data is based on 4 th grade assessment data. • Pk-2 Schools: A feeder school will be identified for the 5% data pull.

ONLINE SYSTEM UPDATE

Monitoring Tools • The latest Monitoring Tool established for the Educator Evaluation System is labeled in WVEIS WOW as EVALR. • EVALR enables district and school-level administrators quick access to aggregate evaluation component completion data.

EVALR Tool

Implemented System Updates • Summative Evaluations completed for a Corrective Action Plan – New Procedure. • Incomplete Evaluation / Not Evaluated Tool & Procedure. • Acceptance of previous year’s evaluation(s) for transferred educators.

WVEIS WOW Menu Educators first classified as “Not Evaluated” through Progressions link. Then a Reason is assigned through the next link. Summative Evaluations completed due to a CAP are documented here.

School Counselor Goals • Guidance is provided within the online system purposed with reconciling the online Goals Component Template with the types and content of Goals submitted by School Counselors. • Full Power. Point Available through Counselors page of Educator Evaluation Website.

Pending System Updates • Principal Tab Added to EVALR Monitoring Tool. • Not Evaluated Tab Added to EVALR Monitoring Tool. • Online Collection Tool for Paper-Based Evaluation Data.

LOOKING AHEAD Discussion of Potential Policy 5310 Revisions

Policy 5310 • Policy 5310 Revision Handout – 10 – 15 Minutes of Table Work / Discussion – Report Out to Large Group • Parking Lot Issues (if time allows) – Report Out to Large Group & Discuss

END OF DAY • Thank you for attending this meeting! • Please leave Policy 5310 Handouts on Tables – To be Collected