Educator Effectiveness State Frameworks and Local Practice CCSSO

Educator Effectiveness: State Frameworks and Local Practice(? ? ) CCSSO Annual Conference, June 2012 Juan M. D’Brot Executive Director of Assessment and Accountability West Virginia Department of Education

Assessment Continuum in WV • Promoting a balanced assessment system against WV State Standards Formative Diagnostics Interim Benchmarks Summative Assessments

Formative, Interim, and Summative Assessments • Summative – WESTEST 2 – Online Writing Assessment (PT) • Interim – Benchmarks in Math, ELA, Science, Social Studies • Formative – Acuity Item Banks in Math, ELA, Science, Social Studies – West Virginia Writes: Practice Writing Program – Other District Purchased Supports – Teacher Developed Assessments

Major Question: Purpose? • Purpose of Summative – Evaluative against a set of standards – Least flexible: long-cycle – Broad sense of the educational state of the state or district • Purpose of Interim – – Instructional, Evaluative, and/or Predictive Medium-cycle • Purpose of Formative – – Process to adjust teaching and learning Instructional diagnostics Typically classroom specific Very short-cycle

Examining the Use of Formative and Interim • High Use vs. Effective Use – Emerging case study of a district and school • Context – Targeted professional development – Dedicated school leadership and staff – State-level support around the use of formative and interim assessments • Early Results:

Grade 9 School A and B

Grade 10 School A and B

Grade 11 School A and B

WESTEST 2 Writing: At or Above Proficiency

Examining the Use of Formative and Interim • Strong support for use of writing shortcycle assessments • Currently examining use of other shortcycle (formative) and medium-cycle (interim) assessments on summative assessment results • Implications on teacher evaluation systems looking for triangulated evidence of student learning

Multiple Use Concerns • Tension between Compliance (accountability) and Continuous Improvement (short-cycle assessments) Conflicting? Adequate Yearly Progress Teacher Evaluation Systems Improvement and Growth Student Learning

Multiple Use Concerns • Tension between Compliance (accountability) and Continuous Improvement (short-cycle assessments) – Summative Assessment • Gaming • Cheating • “Teaching to the Test” – Interim/Formative Assessment • Fabricated student learning gains? • Student sorting to highlight teacher “gains”? • What does the research say? Not impact/effect, but valid link to teacher input/output? • Cognitive vs. Non-Cognitive measures? Scratching the surface

The West Virginia Case • A transition from nothing to something – Old: 5 years experience evaluation “only when the principal determines it to be necessary…” – New: Yearly evaluation for all educators, principals, and evaluators using researchbased professional standards and evidence of student learning

Developing the System • Goal: To develop a supportive system around valid data to drive improvement • Access: A variety of data to support selfreflective practice: – Summative Growth Data – Interim Benchmarks – Formative Diagnostic

Developing the System • Context: – Politics: National vs. State Pressure – Policy: Accountability – Pragmatics: Using actionable and valid data • Development: Stakeholder driven – Board of Education – SEA Staff – LEA staff – Teacher Unions – Legislators – Community Members

The System • System Components Category Input 5 Professional Teaching Standards 80% 14 Critical Standard Elements – 4 pt. rubric 5. 7%/CSE Student Learning Outcomes 20% Summative Growth Data • Math & ELA Only 5% Other Student Learning Evidence • Any Content Professionalism (Mandatory) Output 15% N/A

The System • 20% Student Learning Outcomes – Minimum 5% Student Assessment Data • School-wide Growth – Student Growth Percentiles – A host of evidence – Agreed upon with principal input – Multiple measures • • Acuity (Math, ELA, Science, Social Studies) WV Writes (Writing) Other evidence of student learning Teacher Developed Assessments

The System • Overall Ratings Category 100 Point Scale Distinguished 88 -100 Accomplished 54 -88 Emerging 33 -54 Unsatisfactory <33

Implementation • 2011 -2012 – Concluded 1 st pilot year in a small number of schools – Self-ratings – Accomplished across the board… • 2012 -2013 – 2 nd pilot year with schools in every district represented • 2013 -2014 – Full statewide implementation for all schools in the state

Implications for a “Young System” • Embedded Program Evaluation – WVDE Office of Research small scale implementation study and program evaluation – Formative data has resulted in changes to the system and program – Exploring third-party consulting for enhancements to guidance, system, and process (post-revision)

Lessons Learned • Politics vs. Pragmatism – Representative inclusion vs. Over/underrepresentation of certain views – Political support to push a bill to law • Capacity and Foresight – In-house development and reinventing the wheel? (…more politics) – Building a system for the present vs. building a system for the future – Revising the re-invention

Thank You Juan M. D’Brot (jdbrot@access. k 12. wv. us) Executive Director Office of Assessment and Accountability West Virginia Department of Education