what is the Stanford Education Data Archive SEDA

The Problem • Although we test almost all students, states use different tests in

Our Goal • To provide estimates of test score distributions for all school districts

Three primary steps 1. Estimate district average test scores (relative to their state) from

“Coarsened” proficiency data Level 2 Level 3 Level 4 Level 5 Level 1 far

State A, Grade 3, Math, 2009 Distric Level t 1 Level 2 Level 3

Linking Rescale district distributions (dotted lines), such that the overall state distribution matches the

National average NAEP math scores Grade 8 2009 2010 2011 2012 2013 2014 2015

Three measures of school performance • Average test scores • Change in average test

Average test scores, by grade and year © sean f. reardon, 2017

Average test scores, by grade and year Average is 5. 2 over all grades

Test score change (across years w/in grades) © sean f. reardon, 2017

Test score change (across years w/in grades) +0. 8 from 2009 to 2014 +0.

Test score growth (across grades w/in cohort) © sean f. reardon, 2017

Test score growth (across grades w/in cohort) +5. 9 from 3 rd to 8

Test score growth (across grades w/in cohort) +2. 5 from 6 th to 8

$for more information: Stanford Education Data Archive (SEDA) (https: \seda. stanford. edu) sedasupport@stanford. edu$

Caveats about SEDA 1. Many factors affect students test scores, including children’s home, neighborhood,

Achievement Source Data • EDFacts Data Initiative: – Proficiency count data for each US

What is a school district in SEDA? • A SEDA data school district is

sean©f. sean reardon © 2018 f. reardon, 2017

Slides: 36

Download presentation

what is the Stanford Education Data Archive (SEDA)? …and what can you do with it? sean f. reardon may, 2018

The Problem • Although we test almost all students, states use different tests in each grade and subject – 50 states x 6 grades x 2 subjects ~ 600 different tests/year – Some overlap (e. g. , NECAP, PARCC, SBAC), but not at the national level – And each states sets its own proficiency levels (which vary across grades/subject/years as well) • This makes comparing test performance across states, grades, and years almost

Our Goal • To provide estimates of test score distributions for all school districts on state accountability tests that are: – Comparable across states (years and grades) – Standardized to interpretable units for both researchers and practitioners – Easily merged with data from other sources © sean f. reardon, 2017

Three primary steps 1. Estimate district average test scores (relative to their state) from coarsened proficiency data. 2. Use State NAEP to link these district average test scores to the common national scale (NAEP). 3. Standardize the NAEP scale so it is more easily interpretable. © sean f. reardon, 2017

“Coarsened” proficiency data Level 2 Level 3 Level 4 Level 5 Level 1 far below basic proficient advanced 15 12 42 12 19 © sean f. reardon, 2017

State A, Grade 3, Math, 2009 Distric Level t 1 Level 2 Level 3 Level 4 Level 5 A 1 21 37 33 17 . A 2 45 64 34 0 . A 3 0 34 56 23 . State B, Grade 3, Math, 2009 Distric Level t 1 Level 2 Level 3 Level 4 Level 5 B 1 17 43 56 7 3 B 2 0 5 12 33 58 B 3 2 35 78 60 0 © sean f. reardon, 2017

Linking Rescale district distributions (dotted lines), such that the overall state distribution matches the state distribution on NAEP State A Distribution (Solid Line) District Distributions (Dotted Lines) State A NAEP Distribution (Orange Line) District NAEP-Linked Distributions (Dotted Lines) © sean f. reardon, 2017

Linking Rescale district distributions (dotted lines), such that the overall state distribution matches the state distribution on NAEP State B Distribution (Solid Line) District Distributions (Dotted Lines) State B NAEP Distribution (Purple Line) District NAEP-Linked Distributions (Dotted Lines) © sean f. reardon, 2017

National average NAEP math scores Grade 8 2009 2010 2011 2012 2013 2014 2015 280. 1 281. 4 282. 7 280. 4 238. 1 239. 2 240. 4 239. 1 7 6 5 4 3 © sean f. reardon, 2017

National average NAEP math scores Grade 8 2009 2010 2011 2012 2013 2014 2015 280. 1 281. 4 7 282. 7 280. 4 NAEP score of 282. 7 = “grade 8” level 6 5 4 3 238. 1 239. 2 240. 4 239. 1 NAEP score of 238. 1 = “grade 4” level © sean f. reardon, 2017

National average NAEP math scores Grade 8 2009 2010 2011 2012 2013 2014 2015 280. 1 7 6 5 4 3 238. 1 281. 4 282. 7 Change from grade 4 2009 to grade 8 2013 is 44. 6 points So each “grade level” 239. 2 difference 240. 4 corresponds to about 11 NAEP score points 280. 4 239. 1 © sean f. reardon, 2017

Three measures of school performance • Average test scores • Change in average test scores –(across years within grades) • Growth in average test scores –(across grades within cohorts) © sean f. reardon, 2017

Average test scores, by grade and year © sean f. reardon, 2017

Average test scores, by grade and year Average is 5. 2 over all grades and years. This is 0. 3 below national average (5. 5) © sean f. reardon, 2017

Test score growth (across grades w/in cohort) +2. 5 from 6 th to 8 th grades +5. 9 from 3 rd to 8 th grades +2. 3 from 3 rd to 5 th grades © sean f. reardon, 2017

$for more information: Stanford Education Data Archive (SEDA) (https: \seda. stanford. edu) sedasupport@stanford. edu$

Caveats about SEDA 1. Many factors affect students test scores, including children’s home, neighborhood, pre-school, after-school, and K-12 schooling experiences. 2. SEDA estimates are measures of performance. They are affected by what students have been taught and have learned, how motivated they are to perform on standardized tests, etc. 3. Test performance is not the educational outcome we care about; but it is a reasonable proxy for the extent of opportunity and achievement. 4. The SEDA achievement estimates are not exact and should not be used to make fine distinctions among school districts (particularly across different states). © sean f. reardon, 2017

Achievement Source Data • EDFacts Data Initiative: – Proficiency count data for each US public school. – Available for: • 7 Academic Years: 2008/09 – 2014/15 • 6 Grades: 3 – 8 • 2 Subjects: Math and Reading/English Language Arts © sean f. reardon, 2017

What is a school district in SEDA? • A SEDA data school district is a geographic region. – The set of public (charter and non-charter) schools which are located within the geographic boundaries of a traditional public school district. – Not necessarily identical to the set of schools over which the district has administrative authority. • Enables linkage of test scores to the demographic and socioeconomic information. © sean f. reardon, 2017