Wyoming School Performance Rating Model Report to Wyoming

Wyoming School Performance Rating Model Report to: Wyoming State Board of Education Dr. Michael Flicek Education Consultant October 8, 2013

WAEA School Performance Levels • • 2 Exceeding Expectations Meeting Expectations Partially Meeting Expectations Not Meeting Expectations Wyoming WDE and SBE

Indicators used to Identify School Performance Level • Schools with grades 3 through 8 – Achievement – Growth – Equity • Schools with grades 9 through 12 – Achievement – Readiness – Equity 3 Wyoming WDE and SBE

Achievement – Grades 3 -8 • Assessments used in 2013 – PAWS reading – Grades 3 -8 – PAWS math – Grades 3 -8 – PAWS science – Grades 4 & 8 • Assessment to be added in 2014 – SAWS – Grades 3, 5 &7 4 Wyoming WDE and SBE

Achievement – Grade 11 • Assessment used in 2013 – ACT Subject-Area Tests • Reading • Mathematics • Science – Assessment to be added in 2014 • Writing (to begin after spring 2014 testing) 5 Wyoming WDE and SBE

Illustration of Computation of a School Achievement Score 6 Content Count of Tested Scores Count of Proficient Scores Math 80 65 Reading 80 60 Writing 40 25 Science 20 12 Column Totals 220 162 School Achievement Score 162/220 = 73. 6% Wyoming WDE and SBE

Professional Judgment Panel (PJP) • A representative group of 27 to 30 people – Representing groups prescribed by statute – Selected by the State Board of Education 7 Wyoming WDE and SBE

PJP Major Tasks • Determined the cut points for school scores on each indicator that determine if schools are: – Exceeding Targets – Meeting Targets – Below Targets 8 Wyoming WDE and SBE

Grade Bands for Achievement with PJP Established Cut Scores • Grade 3 through 6 • Low cut = 75 • High cut = 86 • Grades 7 and 8 • Low cut = 68 • High cut = 80 • Grade 11* See Oct. 7, 2013, Data Preparation Report by Flicek & Paul • Low cut = 63 • High cut = 78 Wyoming WDE and SBE

Data Preparation • WAEA Data Model – PJP version contained original ACT student proficiency cut-scores – Replaced by adjusted ACT student proficiency cut-scores in current version – Used equipercentile linking to correct cutscores for the adjusted ACT student proficiency cut-scores Wyoming WDE and SBE

Grade 11 Achievement Cut Scores Wyoming WDE and SBE

Impact of Corrected Data and. Corrected Cut Scores Wyoming WDE and SBE

Student Growth – Grades 4 -8 • Growth in reading and in math • Each student will have a student growth percentile (SGP) – Same grade in school – Similar test scores in previous years – Scores from 1 to 99 13 Wyoming WDE and SBE

School Growth – Grades 4 -8 �For accountability �School median SGP (MGP) �The SGP that ½ of students at the school scored above and ½ of students at the school scored below �PJP Established Cut Scores were: �Low cut = 45 �High cut = 60 14 Wyoming WDE and SBE

Equity – Consolidated Subgroup �Current subgroups performance will continue to be reported �For Wyoming accountability, however, a consolidated subgroup will be used �The consolidated subgroup will be all students who scored below proficient on the previous year’s achievement test �Minimum n size for subgroup accountability = 15 15 Wyoming WDE and SBE

Equity – Grades 4 -8 • Adequate Growth Percentile (AGP) – The SGP on this year’s test, the student needs to be on track for being proficient within 3 years or by the end of grade 8 – These are computed for all students in grades 4 through 8 with more than one previous PAWS test • School’s Equity Score – The percent of students at the school who are considered on track to become proficient within 3 years (i. e. , for whom their SGP equaled or exceeded their AGP) • Equity cut scores were: – Low cut = 40 – High cut = 55 16 Wyoming WDE and SBE

Equity – Grade 11 • Policy objective: To encourage a focus on improving performance of the most highrisk students • This will be applied to schools which had at least 15 students assessed on the ACT that were at the school for a full academic year 17 Wyoming WDE and SBE

High School Equity 2013 • The percent of not proficient test scores in reading and math on the 2012 PAWS was computed • The percent of not proficient test score in reading and math on the 2013 ACT subject area test was computed • The change in the percent of not proficient test scores from 2012 to 2013 was computed Wyoming WDE and SBE

High School Equity 2013 • The schools were placed into one of three (approximately equal) categories for percent improvement – Decrease in % Not Proficient (Exceeding Targets) (decrease of -3. 2% or more) – Minimal Change in % Not Proficient (Meeting Targets) (-3. 2 to +3. 4) – Increase in % Not Proficient (Not Meeting Targets) (increase of 3. 4 or more) Wyoming WDE and SBE

Correction • Reading and Math only • ACT data with adjusted student performance levels was used • 16 schools exceeding targets • 20 schools meeting targets • 17 schools not meeting targets Wyoming WDE and SBE

High School Equity in 2014 • A consolidated subgroup will be established based upon Plan scores from 2013 for the 2013 -14 grade 11 students • The change in performance of this subgroup from the 2013 PLAN to the 2014 ACT will become the metric • This metric has been shown to perform in a manner similar to the 3 -8 equity indicator Wyoming WDE and SBE

Readiness – Grades 9 -12 • In 2013 • Performance on ACT Suite (Explore, Plan & ACT) • Graduation index • Indicators to be added in 2014 • Grade 9 credits earned • Hathaway eligibility 2 2 Wyoming WDE and SBE

ACT Suite – Average Index Score for all Tested Students Composite Score Ranges Wyoming ACT Explore ACT Plan ACT Test Readiness Grade 9 Grade 10 Grade 11 Levels Level 1 1 -14 1 -15 1 -16 Index Points 20 Level 2 15 -17 16 -18 17 -20 50 Level 3 18 -20 19 -21 21 -24 80 Level 4 21 -25 22 -32 25 -36 100 • Aligned with Hathaway Scholarship eligibility cut-points • Each student receives an index score • The average of the index scores for all students will be the school score 2 3 Wyoming WDE and SBE

Graduation Index 2013 Criteria Numbers Student Result Points* 1 Diploma Earned in Four Years Diploma Earned in More than Four Years Continued Enrollment*** Noncompleter 100 2 3 4 100 50 • The index points were established by the PJP 2 4 Wyoming WDE and SBE 0

Graduation Index 2014 Criteria Numbers Student Result Points* 1 100 4 Diploma Earned in Four Years Alternate Standards Completion Diploma Earned in More than Four Years Continued Enrollment 5 Noncompleter 0 2 3 100 50 • The index points were established by the PJP. 2 5 Wyoming WDE and SBE

Grade 9 Credits Earned - 2014 • The percent of students who completed grade 9 with one fourth of the credits required to receive a diploma 2 6 Wyoming WDE and SBE

Hathaway Scholarship Eligibility Index - 2014 Student Eligibility Level Points Not Eligible 0 Provisional 70 Opportunity 80 Performance 90 Honors 100 • The school’s score will be the mean of the student points for the graduating class at the school • The school receives an index score for each graduate • The average of the index scores for all students will be the school score 2 7 Wyoming WDE and SBE

Computation of Total School Readiness Score for 2013 Subindicator Hypothetical Score for a School Example Subindicator Weight (School Score * Weight) ACT Suite Index 55 . 40 22 Graduation Index 67 . 60 40. 2 School Readiness Score (Sum of Subindicator Weighted Scores) = Total Readiness Cut Scores • Low Cut = 68 • High Cut = 80 2 8 Wyoming WDE and SBE 62. 2

Illustration of Computation of Total School Readiness Score for 2014 Subindicator Hypothetical Score for a School Example Subindicator Weight (School Score * Weight) ACT Suite Index 55 . 20 11 Grade 9 % On Track 72 . 10 7. 2 Graduation Index 67 . 25 16. 75 Hathaway Eligibility Index 58 . 45 26. 1 School Readiness Score (Sum of Subindicator Weighted Scores) = 2 9 Wyoming WDE and SBE 61. 05

Decision Table for Schools with Three Indicators for Grades 3 -8 Achievement Below Achievement Meeting Achievement Exceeding Growth Below NOT PARTIALLY Equity Below PARTIALLY MEETING Growth Meeting Growth Exceeding PARTIALLY MEETING Growth Below PARTIALLY MEETING Equity Meeting PARTIALLY MEETING Growth Meeting Growth Exceeding PARTIALLY MEETING EXCEEDING Growth Below PARTIALLY MEETING EXCEEDING MEETING EXCEEDING Equity Exceeding Growth Meeting Growth Exceeding 30

Performance Level Descriptions (For Schools with Grades 3 -8) • • 31 EXCEEDING EXPECTATIONS: Schools in this category, which is reserved for schools considered models of performance, have demonstrated high growth overall, have average to high levels of achievement (proficiency rates) overall, and excel in promoting equity based on growth for students with prior below proficient performance. MEETING EXPECTATIONS: Schools in this category have demonstrated acceptable levels of achievement and growth overall and are showing acceptable progress in promoting equity based on growth for students with prior below proficient performance. PARTIALLY MEETING EXPECTATIONS: Schools in this category have demonstrated either acceptable levels of growth or acceptable levels of achievement overall. Schools in this category may or may not show acceptable performance in promoting equity based on growth for students with prior below proficient performance. NOT MEETING EXPECTATIONS: This category is reserved for schools with unacceptable performance on many or most indicators. For schools in this category improvement is a priority. These schools have low levels of achievement overall and demonstrate low to average growth overall and fall short of producing growth for below proficient students that will move them toward proficiency. Wyoming WDE and SBE

Number of Wyoming Schools with Grades 3 -8 with Each Pattern (2013) Achievement Below Achievement Meeting Achievement Exceeding Growth Below 24 9 0 Equity Below 16 15 0 Growth Meeting Growth Exceeding 0 1 1 Growth Below 5 4 0 Equity Meeting 14 33 3 Growth Meeting Growth Exceeding 2 6 0 Growth Below 0 1 0 0 8 3 2 3 4 Equity Exceeding Growth Meeting Growth Exceeding 32

Decision Table for Schools with Two Indicators for Grades 3 -8 3 3 Achievement Below Achievement Meeting Achievement Exceeding Growth Below NOT PARTIALLY Growth Meeting PARTIALLY MEETING Growth Exceeding PARTIALLY MEETING EXCEEDING Wyoming WDE and SBE

Number of Wyoming Schools with Grades 3 -8 With Each Pattern (2013) 3 4 Achievement Below Achievement Meeting Achievement Exceeding Growth Below 13 10 1 Growth Meeting 8 22 15 Growth Exceeding 1 10 14 Wyoming WDE and SBE

Overall Impact for Grade 3 -8 Schools

Decision Table for Schools with Three Indicators for High Schools Equity Below Readiness Below Readiness Meeting Readiness Exceeding Readiness Below Readiness Equity Meeting Readiness Exceeding Readiness Below Readiness Equity Exceeding Meeting Readiness Exceeding Achievement Below Achievement Meeting Achievement Exceeding NOT PARTIALLY MEETING PARTIALLY MEETING PARTIALLY MEETING EXCEEDING PARTIALLY MEETING EXCEEDING 36

Performance Level Descriptions (For High Schools) • • EXCEEDING EXPECTATIONS: Schools in this category, which is reserved for schools considered models of performance, have demonstrated average to high levels of achievement (proficiency rates) overall, have high performance on graduation rates and other readiness indicators and have narrow and/or improving achievement gaps for students with below proficient performance. MEETING EXPECTATIONS: Schools in this category have demonstrated either high levels of achievement overall or high performance on graduation rates and other readiness indicators and are showing acceptable performance in promoting equity based on the magnitude and/or improvement of the achievement gap for students with below proficient performance. PARTIALLY MEETING EXPECTATIONS: Schools in this category have demonstrated either acceptable levels of achievement overall or acceptably performance on graduation rates and other readiness indicators. Schools in this category may or may not demonstrate acceptable performance for promoting equity based on the size of the achievement gap or improvement in the achievement gap for students with below proficient performance. NOT MEETING EXPECTATIONS: This category is reserved for schools with unacceptable performance on many or most indicators. For schools in this category improvement is a priority. These schools typically have low levels of achievement fall short of expectations on graduation and other readiness indicators and have large achievement gaps that show little improvement. Wyoming WDE and SBE

Number of Wyoming High Schools with Each Pattern (2013) Achievement Below Achievement Meeting Achieve 1 ment Excee 1 ding Readiness Below Readiness Meeting Readiness Exceeding 4 1 0 1 6 1 1 2 1 Readiness Below Readiness Equity Meeting Readiness Exceeding 3 0 0 5 7 0 0 1 4 1 3 0 1 8 2 0 0 1 Equity Below Readiness Below Readiness Equity Exceeding Meeting Readiness Exceeding 38

Decision Table for High Schools with Two Indicators Achievement Below Meeting Exceeding 3 9 Readiness Below NOT PARTIALLY Readiness Meeting PARTIALLY MEETING Readiness Exceeding PARTIALLY MEETING EXCEEDING Wyoming WDE and SBE

Number of Wyoming High Schools with Each Pattern (2013) Achievement Below t Meeting Exceeding 4 0 Readiness Below 13 0 1 Readiness Meeting 5 3 0 Readiness Exceeding 0 2 1 Wyoming WDE and SBE

Overall Impact for High Schools (PJP Version)

Overall Impact for High Schools (Corrected) Wyoming WDE and SBE

Participation Rate Impact • Schools with grades 3 -8 • All schools had participation rates of 98% or higher in 2013 • One small school was docked a performance level for having less than 95% participation when a prior year was included in an attempt to meet the minimum n requirement • This school went from “meeting” to “partially meeting” Wyoming WDE and SBE

High School Participation Rate • 11 of 84 high schools had participation rates on the ACT suite of between 90% and <95% • 3 of these were already “not meeting expectations” • 4 dropped from “meeting” to “partially meeting” • 4 dropped from “partially meeting” to “not meeting” Wyoming WDE and SBE

Participation Rate “Not Met” • 12 high schools had less than 90% participation rate on the ACT suite of tests • 8 of these were already “not meeting” • 3 dropped from “partially meeting” to “not meeting” • 1 dropped from “meeting” to “not meeting” Wyoming WDE and SBE

The Result • Grades 3 -8 did well on participation rate • 13 of 84 high schools (16%) had lower performance levels because of poor participation rate on the ACT suite of tests • An additional 10 of 84 high schools (12%) had less than 95% participation rate on the ACT Suite but were already “not meeting” • 28% of high schools had participation Rate Problems. Wyoming WDE and SBE

Final High School Impact (PJP) Wyoming WDE and SBE

Final Impact High Schools (Corrected) Wyoming WDE and SBE

Final Wyoming Result Wyoming WDE and SBE

Contact Information • Michael Flicek • mikefli@msn. com • 307 -259 -3963 • John Paul • John. paul@wyo. gov • 307 -777 -8771 Wyoming WDE and SBE