California Educational Research Association Analysis Measurement and Accountability

California Educational Research Association Analysis, Measurement, and Accountability Reporting Division November 30, 2016 CALIFORNIA DEPARTMENT OF EDUCATION Tom Torlakson, State Superintendent of Public Instruction Printed by the Riverside County Office of Education

Polling Questions TOM TORLAKSON State Superintendent of Public Instruction • Throughout this presentation we will be asking you a variety of questions to help guide further development of the accountability system. You will need to use your phone to respond to the questions. • The polling URL is: http: //etc. ch/LTc 4 2

Polling Question TOM TORLAKSON State Superintendent of Public Instruction • What organization are you affiliated with? a) b) c) d) e) f) County Office District Office Research Organization Charter School CDE Other 3

Agenda TOM TORLAKSON State Superintendent of Public Instruction • Framework for the New Accountability and Continuous Improvement System • Overview of the New Accountability System • Local Control Funding Formula (LCFF) Evaluation Rubrics : Top Level Data Display • Overview of State Indicators • Status of the Academic Indicator 4

Agenda (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • Status, Change, and Assigning a Color: The California Model • An In-depth Look into State Indicators: – Graduation Rate Indicator – Suspension Rate Indicator – English Learner Indicator • An In-depth Look into the College/Career Indicator (local indicator for initial release) 5

Agenda (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • In-Depth Look at Performance Categories • Overview of Local Indicators • Timeline 6

TOM TORLAKSON State Superintendent of Public Instruction Framework for the New Accountability and Continuous Improvement System 7

Accountability Framework TOM TORLAKSON State Superintendent of Public Instruction • • • Performance Equity Improvement Transparency One single cohort system to meet local, state, and federal needs Preparing All Students for College, Career, Life, and Leadership in the 21 st Century, Superintendent’s Task Force on Accountability and Continuous Improvement Report, May 2016. 8

Accountability and Continuous Improvement System: Components TOM TORLAKSON State Superintendent of Public Instruction 1. Local Control and Accountability Plan (LCAP) and Annual Update – Address eight state priorities (ten for counties) – Outline plans for expenditures – Report on progress 2. LCFF Evaluation Rubrics – State and local indicators – Performance standards 3. Support and Assistance System – Designed to support continuous improvement 9

TOM TORLAKSON State Superintendent of Public Instruction Overview of the New Accountability System 10

New Accountability System TOM TORLAKSON State Superintendent of Public Instruction • On September 8, 2016, California’s State Board of Education (SBE) approved a new multiple measures accountability system. • Rather than having two separate accountability systems (state and federal) as in prior years, the new integrated system captures local, state, and federal requirements. 11

New Accountability System (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • The new accountability system is based on multiple measures which will provide a more complete picture of what contributes to a positive educational experience for students. 12

New Accountability System (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • The multiple measures are based on the LCFF state priorities and are divided into two types of indicators: – State indicators – Local indicators 13

LCFF Evaluation Rubrics TOM TORLAKSON State Superintendent of Public Instruction LEAs = local educational agencies • Both state and local indicators will be reported through the LCFF Evaluation Rubrics. • The statutory purposes of these rubrics: 1. Support LEAs in identifying strengths, weaknesses, and areas for improvement 2. Assist in determining whether LEAs are eligible for technical assistance 3. Assist the State Superintendent of Public Instruction in determining if LEAs are eligible for more intensive state support/intervention 14

Polling Question TOM TORLAKSON State Superintendent of Public Instruction • Would you like to see the Top. Level Data Display for the new multiple measures accountability system? A. Yes B. No 15

TOM TORLAKSON State Superintendent of Public Instruction Local Control Funding Formula Evaluation Rubrics: Top Level Data Display 16

Currently There are Four Reports TOM TORLAKSON State Superintendent of Public Instruction 1. 2. 3. 4. Equity Report All Student Performance Report Status/Change Report Indicator Cluster Report 17

TOM TORLAKSON State Superintendent of Public Instruction Overview of State Indicators 18

State Indicators TOM TORLAKSON State Superintendent of Public Instruction • Academic Indicator: – English language arts/literacy (ELA) assessment – Mathematics assessment • Graduation Rate Indicator • Suspension Rate Indicator • English Learner Indicator (ELI)— measures progress of English learners (ELs) 19

Future State Indicators TOM TORLAKSON State Superintendent of Public Instruction • Chronic Absenteeism Rate • College/Career Indicator (measures postsecondary preparedness) 20

TOM TORLAKSON State Superintendent of Public Instruction Status of the Academic Indicator 21

November SBE Meeting TOM TORLAKSON State Superintendent of Public Instruction SBE=State Board of Education • At the November 2016 SBE meeting, the proposed performance standards recommended for the Academic Indicator were based on the percent of students who scored “Standard Met” or “Standard Exceeded” on the Smarter Balanced Assessments for grades three through eight. – Grade eleven assessment results are captured in the College/Career 22 Indicator.

November SBE Meeting (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • The SBE decided not to release the Academic Indicator using “Standard Met” or “Standard Exceeded” voicing concern that this closely paralleled the Adequate Yearly Progress (AYP) methodology, which would reward schools focusing only on those students who were closest to proficient. 23

November SBE Meeting (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • Rather, the SBE wants to encourage districts and schools to improve the academic achievement of all students. • Therefore, the SBE requested CDE staff to work on a methodology that uses scale scores. CDE= California Department of Education 24

Next Steps TOM TORLAKSON State Superintendent of Public Instruction • CDE staff met with the testing vendor, Educational Testing Service (ETS), to ensure the validity of using scale scores to measure distance from a standard point. • Based on the information obtained from ETS staff, CDE staff will continue to work with the Technical Design Group (TDG) to explore options for this proposed methodology. 25

Next Steps (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • In January 2017, the CDE plans to bring the a proposed methodology to the SBE with four possible options for the Academic Indicator. – Distance from Met – Distance from Nearly Met – Distance from the Statewide Average Score (by grade) – Distance from the Lowest Possible Scale Score 26

TOM TORLAKSON State Superintendent of Public Instruction 27

Next Steps (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • Pending SBE’s decisions at this meeting, the CDE will move forward with the methodology and one of proposed criteria or pursue other options. 28

Next Steps (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • In addition, the SBE will be discussing the definition of the EL student group in the Academic Indicator at their January 2017 meeting. – In prior accountability reports, the ELA and mathematics results for the EL student group included results for both EL students and reclassified 29 students.

Next Steps (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • However, the SBE has asked CDE staff to explore removing reclassified fluent English proficient (RFEP) students from the EL student group. • CDE staff are exploring multiple options. 30

Polling Question TOM TORLAKSON State Superintendent of Public Instruction • For the Academic Indicator, which criteria do you prefer for the proposed methodology? a) Distance From Met b) Distance From Nearly Met c) Distance from the Lowest Possible Scale Score d) Distance from the Statewide Average Score (by grade) 31

Polling Question TOM TORLAKSON State Superintendent of Public Instruction • For the Academic Indicator, which EL student group definition do you prefer? a) ELs plus 4 -years of RFEP (maximum amount of time allowed under the Every Student Succeeds Act (ESSA) b) ELs plus 2 -Years of RFEP c) ELs only 32

Polling Question TOM TORLAKSON State Superintendent of Public Instruction • On a scale of one to five, with five being the highest, how familiar are you with the 5 X 5 color grid to determine the five performance categories (red, orange, yellow, green, and blue)? 33

TOM TORLAKSON State Superintendent of Public Instruction Status, Change, and Assigning a Color: The California Model 34

The California Model TOM TORLAKSON State Superintendent of Public Instruction • The California Model combines five “Status” and five “Change” levels which results in a performance category (or color) for each of the state indicators. • The model provides equal weight to both Status and Change. 35

Five Status and Five Change Levels TOM TORLAKSON State Superintendent of Public Instruction Five Status Levels Five Change Levels Very High Increased Significantly High Increased Medium Maintained Low Declined Very Low Declined Significantly 36

Status and Change TOM TORLAKSON State Superintendent of Public Instruction • Status is based on current year performance. • Change is the difference between prior year performance and current year performance. – Exception! The Graduation Rate Indicator is the only state indicator that uses a multi-year average rather than prior year data. 37

Methodology TOM TORLAKSON State Superintendent of Public Instruction • To determine overall performance, the percentile distributions for Status and Change were examined for each indicator: • For Status, LEAs and schools were ordered from highest to lowest and four cut points were selected based on the distribution of all LEAs and schools. These cut points created the five Status Levels. 38

Methodology (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • For Change, LEAs and schools were ordered separately from highest to lowest for positive change and lowest to highest for negative change. • Cut points were determined separately for positive and negative change. A total of four cuts were selected which created five Change Levels. 39

Methodology (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • Cut points will remain in place for a select number of years (e. g. , 3 to 5 years) to be determined by the SBE. 40

Performance Category TOM TORLAKSON State Superintendent of Public Instruction • The combination of Status and Change results in a performance category that is assigned a color for each indicator: Blue Green Yellow Orange Highest Red Lowest 41

Who Gets a Performance Category? TOM TORLAKSON State Superintendent of Public Instruction • The model will be applied to all LEAs, schools, and student groups with 30 or more students. • The data used to determine “ 30 or more” differs for each indicator. • While a performance category (or color) will not be determined for LEAs, schools, or student groups with fewer than 30 students, the Status and Change data will be reported for groups with 11 -29 42 students.

Who Gets a Performance Category? (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction – In these instances, an asterisk (*) will be reported to indicate that because there are fewer than 30 students, the LEA/school will not receive a performance category. • Data for fewer than 11 students will not be reported due to privacy reasons and a “- - -” will be displayed. • Data that is not yet available will be noted with an N/A. 43

TOM TORLAKSON State Superintendent of Public Instruction An In-Depth Look into the State Indicators One of the goals of this presentation is to provide the necessary information for you to calculate performance category (or color) for each state indicator. 44

Before We Dive In…. TOM TORLAKSON State Superintendent of Public Instruction LEA Data Charter Schools • Because all charter schools are treated as LEAs under the LCFF, charter school data are not rolled up to their authorizing agency’s data. Alternative Schools • Since a separate accountability system will be developed for alternative schools, all alternative schools (i. e. , Alternative Schools Accountability Model [ASAM]) are also 45 excluded from LEA-level data.

Before We Dive In… (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • Therefore, the LEA-level data for graduation and suspension rates will not match the data reported on the CDE Data. Quest Web page. 46

Polling Question TOM TORLAKSON State Superintendent of Public Instruction • Should alternative schools’ results be included in the LEA-level data even though an alternative accountability system will be developed? a) Yes, include b) No, do not include 47

TOM TORLAKSON State Superintendent of Public Instruction Graduation Rate Indicator 48

Who Will Receive a Graduation Rate Indicator? TOM TORLAKSON State Superintendent of Public Instruction • This indicator applies to LEAs, schools, and student groups that have 30 or more students in the four-year graduation cohort. Note: Students who earn a Special Education Certificate of Completion or a General Education Degree (GED) are not counted as high school graduates but are included in the denominator. 49

Status TOM TORLAKSON State Superintendent of Public Instruction • For the initial release of the LCFF Evaluation Rubrics, the 2014– 15 cohort graduation rate, or the class of 2015 graduation data, will be used to determine Status. 50

Change TOM TORLAKSON State Superintendent of Public Instruction • Prior three-years of four-year cohort graduation data were used to calculate three-year weighted average to determine Change. • Note: This is the only state indicator that uses a three-year weighted average to calculate Change. 51

Three-Year Weighted Average Formula TOM TORLAKSON State Superintendent of Public Instruction Class of 2012 Graduates + Class of 2013 Graduates + Class of 2014 Graduates divided by Students in the Class of 2012 + Students in the Class of 2013 + Students in the Class of 2014 52

Three-Year Weighted Average Formula (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • It is important to note that if the LEA, school, or student group does not have cohort data for all prior three graduating classes, then the weighted average for Change was calculated using the one or two years of available cohort data. 53

Change Formula TOM TORLAKSON State Superintendent of Public Instruction Current Status (Class of 2015) minus Three-Year Weighted Average 54

Determine Performance Category (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction Status: 89. 4% Change: +4. 6% 55

Future Plans TOM TORLAKSON State Superintendent of Public Instruction • At the SBE’s request, CDE staff will work with the TDG to analyze and investigate the incorporation of the fiveyear cohort graduation rate into this indicator. • Based on the current timeline, implementation of the four-and five-year cohort graduation rates may occur in 2018– 19. 56

TOM TORLAKSON State Superintendent of Public Instruction Suspension Rate Indicator 57

Who Will Receive a Suspension Rate Indicator? TOM TORLAKSON State Superintendent of Public Instruction • LEAs, schools, and student groups that have 30 or more students enrolled will receive a Suspension Rate Indicator. • The enrollment data are obtained from the California Longitudinal Pupil Achievement Data System (CALPADS). 58

Suspension Rate Rules TOM TORLAKSON State Superintendent of Public Instruction • If a student is suspended multiple times (at the same school or district), the student is counted as being suspended only once. • LEA Example: If a student was suspended: • Five times at School A, • Twice at School B, and • Twice at School C The student would be counted as being 59 suspended once at the LEA.

Status and Change TOM TORLAKSON State Superintendent of Public Instruction Status: • The 2014– 15 suspension rate will be used for Status for the initial release of the LCFF Evaluation Rubrics. Change: • Change uses current and prior year suspension rates. Change Formula: 2014– 15 rate minus 2013– 14 rate 60

Key Differences in Suspension Cut Scores TOM TORLAKSON State Superintendent of Public Instruction • Data simulations revealed that suspension data varies widely among LEA and school type. Therefore, multiple suspension cut scores were set for LEAs and schools based on their type. • This resulted in six different sets of cut scores: – Three at LEA-level: Elementary, High, and Unified – Three at School-level: Elementary, Middle, and High 61

Key Differences in Suspension Cut Scores (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • Having six different sets of cut scores also means six separate 5 x 5 color grids based on district or school type. 62

Impact of Multiple Cut Scores TOM TORLAKSON State Superintendent of Public Instruction LCAP= Local Control and Accountability Plan ESSA= Every Student Succeeds Act • Since cut scores were set separately for LEAs and schools, charter schools and single school districts could receive two performance categories (or two colors): (1) at the LEA-level and (2) at the school-level because: – Charter schools are treated as LEAs under the LCAP, and – Single school districts are treated as schools under the ESSA 63

Impact of Multiple Cut Scores (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • Because the LEA and school performance categories (colors) could be inconsistent, the SBE approved holding charter schools and single school districts accountable for the suspension rate cut scores using the school-level cut scores. • Therefore, charter schools and single school districts will only be held accountable for their school-level performance category (color). 64

Key Differences in the Suspension Goal TOM TORLAKSON State Superintendent of Public Instruction • It is important to remember that for this indicator, the goal is reversed. • For all other state indicators, the desired outcome and goal is to achieve a high percent for Status and Change. 65

Key Differences in the Suspension Goal (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • However, the desired outcome and goal for the Suspension Rate Indicator is to have a low suspension rate, which translates to a low percent for Status and Change. • For this reason, the Status and Change levels on the 5 x 5 color grids are in reverse order compared to the grids for the other indicators. 66

Key Differences in 5 x 5 Grid TOM TORLAKSON State Superintendent of Public Instruction Level Increased Significantly Increased Maintained Declined Significantly Very Low Yellow Green Blue Orange Yellow Green Blue Orange Yellow Green High Red Orange Yellow Red Red Orange Yellow Low Medium Very High 67

Assignment of Performance Category TOM TORLAKSON State Superintendent of Public Instruction • Schools that did not certify (or submit) suspension data in the CALPADS are automatically assigned the Orange performance category. 68

Determine Performance Category (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction Status: 0. 5% Change: +0. 2% 69

Polling Question TOM TORLAKSON State Superintendent of Public Instruction Currently, a student who is suspended multiple times at the same school (or same district) is only counted once in the suspension rate. • How should suspensions be counted in the Suspension Indicator? a) Include only the student’s first suspension b) Include all of the student’s suspensions 70

TOM TORLAKSON State Superintendent of Public Instruction English Learner Indicator (ELI) 71

Who Will Receive an ELI? TOM TORLAKSON State Superintendent of Public Instruction • This indicator applies to LEAs and schools that have 30 or more students who took the annual California English Language Development Test (CELDT). • Note: Because 86. 2% of schools have no significant, or only one significant race/ethnic student group within the EL group, student group data will not be reported for the ELI. 72

ELI Data Sources TOM TORLAKSON State Superintendent of Public Instruction • The ELI determines progress through the use of two data sources: 1. Annual CELDT results, and 2. EL reclassification 73

CELDT Data TOM TORLAKSON State Superintendent of Public Instruction • The CELDT has five overall performance levels: – Beginning – Early Intermediate – Early Advanced – Advanced 74

CELDT Data (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • Because the CELDT Intermediate performance level has a large range of scale scores, many students stay in the intermediate level for multiple years. As a result, stakeholders advised, and the SBE approved, that this level be divided into two, for accountability purposes only, to recognize the substantial growth that can be made within this particular level. 75

CELDT Data (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • Therefore, the ELI uses six overall CELDT performance levels: – – – Beginning Early Intermediate Low Intermediate High Intermediate Early Advanced 76

ELI Model TOM TORLAKSON State Superintendent of Public Instruction • Annual CELDT takers must advance at least one CELDT performance level from prior year to current year to be included in the numerator of the ELI calculation. Examples: Prior Overall CELDT Performance Current Overall CELDT Performance Does the LEA or School Receive Credit? Early Intermediate Low Intermediate Yes High Intermediate Advanced Yes Beginning No 77

ELI Model (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • Students who scored Early Advanced or Advanced Proficient in the prior year and maintained that performance level for the current year will be included in the numerator for the ELI calculation. This is the only set of annual CELDT test takers who are not required to advance one CELDT performance level. • ELs who were reclassified in the prior year will also be included in the numerator and denominator for the ELI calculation. 78

ELI Model (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction Previous CELDT Overall Level Beginning Current CELDT Overall Level Early Intermediate Low Intermediate High Intermediate Early Advanced or Adv Not Proficient Early Advanced or Adv Proficient + Students Reclassified in Prior Year 79

ELI Formula: Numerator TOM TORLAKSON State Superintendent of Public Instruction • Annual CELDT test takers who: – Increased at least one CELDT level compared to the prior year – Maintained Early Advanced/Advanced English Proficient • ELs who were reclassified in the prior year 80

ELI Formula: Denominator TOM TORLAKSON State Superintendent of Public Instruction • Total number of annual CELDT test takers • ELs who were reclassified in the prior year 81

Status and Change TOM TORLAKSON State Superintendent of Public Instruction Status • The percent of ELs who moved up a performance level from the 2014 to 2015 CELDT plus ELs who were reclassified between July 1, 2013 to June 30, 2014. Change • Difference in Status from current year to prior year. 82

Assignment of Performance Category TOM TORLAKSON State Superintendent of Public Instruction • Schools that did not test at least 50 percent of their EL population in the CELDT are automatically assigned an Orange performance category. • Determination of the 50 percent is based on the EL demographic data reported for mathematics in the Smarter Balanced Assessment file from the testing vendor. 83

Example TOM TORLAKSON State Superintendent of Public Instruction Step 1: Percent of annual CELDT test takers who advanced at least one performance level (or maintained Early Advanced/Advanced English Proficient) on the 2015 overall CELDT compared to the 2014 overall CELDT Step 2: Number of ELs who were reclassified in prior year (2013– 14) Step 3: Add reclassified students to the numerator and denominator of Step 1 and calculate the rate. 210/250 = 84% 20 students 230/270 = 85% 84

Determine Performance Category TOM TORLAKSON State Superintendent of Public Instruction Status: 85% Change: +2% 85

Activity TOM TORLAKSON State Superintendent of Public Instruction Each table has a sheet of paper containing the directions for your task. Your task is determine the performance category based on the data provided, and to populate the LCFF Evaluation Rubrics posted around the room with your performance category (color) results. You have 10 minutes! 86

TOM TORLAKSON State Superintendent of Public Instruction An In-Depth Look into the College/Career Indicator 87

What is the Purpose of the CCI? TOM TORLAKSON State Superintendent of Public Instruction • The College/Career Indicator (CCI) was designed to emphasize that a high school diploma should represent completion of rigorous course work that prepared students for postsecondary. 88

CCI Measures TOM TORLAKSON State Superintendent of Public Instruction • The CCI currently contains the following measures: – Advanced Placement (AP) exam results – Dual Enrollment – Early Assessment Program (EAP) results for ELA and mathematics (Grade 11) – a-g completion – Career Technical Education (CTE) pathway completion 89

CCI Measures (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • Potential measures for the release in 2017– 18: – International Baccalaureate (IB) – State Seal of Biliteracy – Golden State Seal Merit Diploma – IB Career–related Programme – Articulated CTE Pathway 90

Students with Disabilities TOM TORLAKSON State Superintendent of Public Instruction • Students with the most severe cognitive disabilities (i. e. , students who take the California Alternate Assessment) are removed from the calculation of the CCI. 91

First a Local Indicator TOM TORLAKSON State Superintendent of Public Instruction • Although the CCI was planned to be reported as a state indicator, the SBE approved: – Reporting the CCI as a local indicator for the initial release of the LCFF Evaluation Rubrics based on Status only because the current CCI does not contain the Smarter Balanced results. – Note: The CDE will pre-populate the CCI data. 92

Then a State Indicator TOM TORLAKSON State Superintendent of Public Instruction • When the LCFF Evaluation Rubrics are released in fall 2017, the CCI will be reported as a state indicator based on Status only because only one year of Smarter Balanced results will be available. • The first time grade eleven Smarter Balanced Assessment results will be available to calculate both Status and Change for the CCI is in 2018– 19 (Class of 2016 and 2017). 93

CCI Formula TOM TORLAKSON State Superintendent of Public Instruction Students Who Receive a High School Diploma and Meet the CCI Benchmark of “Prepared” divided by Current Year Graduation Cohort 94

Status Only TOM TORLAKSON State Superintendent of Public Instruction • For the initial release of the CCI (as a local indicator), Status will be calculated based on the Class of 2014. The following years of data will be used for the CCI measures: CCI Measure AP exam Grade 11 EAP results Data Source Year of Data The College Board 2014 STAR file From Educational Testing Service (ETS) a-g completion CALPADS CTE pathway completion CALPADS Dual enrollment CALPADS 2013 2014 (Yes/No filled in by LEA ) Last 3 yrs in high school (2012, 2013, 2014) 95

TOM TORLAKSON State Superintendent of Public Instruction In-Depth Look at Performance Categories 96

Green is the Target TOM TORLAKSON State Superintendent of Public Instruction • The target performance category (or color) is Green for all LEAs, schools, and student groups. • Performance categories (or colors) tell more than just the current year’s status; they reflect information about status and change. In other words, if your school is improving, that’s already reflected in the performance category. 97

TOM TORLAKSON State Superintendent of Public Instruction The Performance Categories Already Reflect Change…. . • For some indicators, earning Red one year and Orange the next year may not reflect improvement! 98

Moving From RED to ORANGE Does Not Always Mean Improvement! TOM TORLAKSON State Superintendent of Public Instruction Example: An LEA has a 2015– 16 graduation rate of 82. 0%. The LEA’s prior three-year average graduation rate is 89. 0% (Change = -7%). As a result, the performance category is: Red The 2016– 17 graduation rate is 79. 0%. The updated prior 3 -year average graduation rate is 83. 0% (Change = -4). As a result the performance category is: Orange 99

The Reverse is also True TOM TORLAKSON State Superintendent of Public Instruction Example: An LEA has a 2015– 16 graduation rate of 92. 0%. The district’s prior 3 -year average graduation rate is 85. 0% (Change = +7%). As a result, the performance category is: Blue The 2016– 17 graduation rate is 94. 0%. The updated prior 3 -year average graduation rate is 91. 0% (Change = +3). As a result, the performance category is: Green 100

Takeaways From These Examples TOM TORLAKSON State Superintendent of Public Instruction • Performance category already reflects change, so: – Talking about how a performance category changes over time may not make sense. – Discussions about trends should focus on trends in the underlying data. – A red, orange, or yellow performance category means that there is more work to be done. A green or blue performance category means that the trajectory of performance is fine. 101

TOM TORLAKSON State Superintendent of Public Instruction Overview of Local Indicators 102

Local Indicators TOM TORLAKSON State Superintendent of Public Instruction • Basic (Priority 1): – Appropriately assigned teachers – Access to curriculum-aligned instructional materials – Safe, clean, and functional school facilities • Implementation of State Academic Standards (Priority 2) • Parent Engagement (Priority 3) • School Climate (Priority 6) 103

Local Indicators (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction For county offices of education (COEs) only: • Coordination of Services for Expelled Students (Priority 9) • Coordination of Services for Foster Youth (Priority 10) 104

Other Local Indicators TOM TORLAKSON State Superintendent of Public Instruction • The following local indicators will be populated by the CDE: – College/Career Indicator (initially will be reported in the rubrics as a local indicator for informational purposes only. ) – Grade eleven Smarter Balanced Assessment results. 105

Meeting Standards TOM TORLAKSON State Superintendent of Public Instruction • For each local indicator, LEAs are responsible for: – Using locally available information to measure progress relative to performance standards and criteria adopted by the SBE. – Reporting the results at a regularly scheduled meeting of the local governing board. – Reporting the results through the Evaluation Rubrics. 106

Meeting Standards (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • LEAs will use the following standards to assess performance on the local indicators: – Met – Not Met for Two or More Years • These results will be populated by the LEAs in the LCFF Evaluation Rubrics. 107

SBE November Decisions TOM TORLAKSON State Superintendent of Public Instruction At the November 2016 SBE meeting, the SBE approved the following tools for the Local indicators: • Basic (Priority 1): – These data are collected through the School Accountability Report Card (SARC). 108

SBE November Decisions (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction – LEAs will use locally available information, including data reported through the SARC, to provide evidence of progress. – The Web-based interface system for the Evaluation Rubrics is being developed based on the same data system that supports the SARC template. 109

SBE November Decisions (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction – The goal is for the rubrics system to auto-populate the necessary SARC data for LEAs that use the SARC template. 110

SBE November Decisions (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • School Climate (Priority 6): – At least once every other year, LEAs administer a survey in at least one grade within each grade span served (K-5, 6 -8 and 9 -12). – Provide a narrative summary of the local survey and the analyses of the survey in a text box provided in the Evaluation Rubrics Web-based user interface. 111

SBE November Decisions (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • Coordination of Services for Expelled Students (Priority 9): – COEs determine progress using the self-reflection tool adopted by the SBE. – COEs reports results through the local data selection option in the Evaluation Rubrics Web-based user interface. 112

SBE November Decisions (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • Coordination of Services for Foster Youth (Priority 10) – COEs determine progress using the self-reflection tool adopted by the SBE. – COEs reports results through the local data selection option in the Evaluation Rubrics Web-based user interface. 113

Next Steps for Local Indicators TOM TORLAKSON State Superintendent of Public Instruction • The CDE will bring recommended standards for the Implementation of State Academic Standards (Priority 2) and Parent Engagement (Priority 3) to the SBE in January 2017.

TOM TORLAKSON State Superintendent of Public Instruction Timeline 115

Timeline TOM TORLAKSON State Superintendent of Public Instruction • The LCFF Evaluation Rubrics will be released in 2017 to LEAs first for their preview. • LEA superintendents have been notified by the CDE to designate a “LCFF Rubrics Coordinator” to compile and coordinate feedback and/or responses from stakeholders.

Timeline (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • The LCFF Rubrics Coordinators will be provided access to the LCFF Evaluation Rubrics when they are made available. • After LEAs preview the LCFF Evaluation Rubrics, the public release will occur. • The CDE is releasing the data to provide LEAs and the public an opportunity to become familiar with the LCFF Evaluation Rubrics before they are used to determine technical assistance or intervention. 117

Timeline (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • Beginning in the 2017– 18 school year, the data reported in the LCFF Evaluation Rubrics will be used to determine LEA eligibility for technical assistance or interventions. • In addition, the release of the Evaluation Rubrics in the fall each year will make data available for LEAs to use in the development of their LCAPs. 118

Timeline (Cont. ) TOM TORLAKSON State Superintendent of Public Instruction • Because one of the goals of the new accountability system is continuous improvement, the SBE has established an annual review cycle beginning in March 2017, with any potential action taken no later than September of each year, to continually evolve the Evaluation Rubrics based on user experiences and stakeholder feedback, as well as the availability of 119 new data.

TOM TORLAKSON State Superintendent of Public Instruction Questions? 120

Contact Information TOM TORLAKSON State Superintendent of Public Instruction • Questions regarding the Evaluation Rubrics should be directed to the Academic Accountability Unit by phone at 916 -319 -0863 or by email lcffrubrics@cde. ca. gov 121