Linking Partnership for Assessment of Readiness for College

Linking Partnership for Assessment of Readiness for College and Careers and Measures of Academic Progress Helen Y. Wang, Ph. D. , Huafang Zhao, Ph. D. , and Kecia L. Addison, Ph. D. Office of Shared Accountability Montgomery County Public Schools, MD RESULTS BACKGROUND This linking study addressed how Measures of Academic Progress (MAP) data can serve as indicators of success on summative assessments of Partnership for Assessment of Readiness for College and Careers (PARCC) for Grades 3– 8 in Montgomery County Public Schools (MCPS) in Maryland. For more than a decade, MCPS has been administering MAP to elementary and middle school students three times a year (fall, winter, and spring) to inform instruction and monitor student progress on state accountability assessments. Therefore, with a recent shift of the state assessment to PARCC, it became necessary to examine the relationship of MAP with PARCC. Incorporating correlation, logistic regression, and equipercentile linking procedures, this study used a singlegroup linking method to investigate predictive and concurrent validity evidence of MAP and PARCC in reading and mathematics for Grade 3– 8 students. Specifically, the linking study was to 1) identify threshold scores on fall MAP associated with high probabilities (65% and 75%) for meeting the college and career readiness benchmark (performance level 4) on PARCC, and 2) develop concordance tables for spring MAP and PARCC scores. The study results can provide school staff with informative data to identify students at risk of not being on track of attaining college and career readiness and hence make additional instructional support available for the students in need. They can also reinforce the district’s decision to continue the use of MAP as an instructional monitoring tool and help the district improve its accountability system relevant to budget decision. Design and Procedures The Single-group Design, a common approach for a linking study that requires a sample of students who took both assessments under even though the two assessments are different in test-specific constructs, was used. This method allowed researchers to control for differential proficiency of examinees (Dorans et al. , 2010). Pearson correlation analysis was used to measure strength and direction of the relationship of fall and spring MAP RIT scores with PARCC scale scores within each content area and grade level. RESULTS Correlation between MAP and PARCC A demonstrated strong positive correlation provided predictive validity evidence for fall MAP RIT scores and PARCC scale scores, while a demonstrated strong positive correlation also provided concurrent validity evidence for the two assessments. All the correlation coefficients were between 0. 80 and 0, 90 in both reading and mathematics across MAP test administrations and grade levels (Table 2). The correlation between spring MAP RIT scores and PARCC scale scores was even stronger compared to that for the fall. The Logistic regression procedure was applied to identify threshold RIT scores on fall MAP that predicted 65% and 75% probabilities of attaining performance level 4 on PARCC for each content area and grade level, with estimation of prediction accuracy. The equipercentile linking procedure a statistical procedure that bridges the scores from one test to another through the corresponding percentile ranks on the two tests (Holland & Dorans, 2006), was used to establish concordance tables for spring MAP RIT scores and PARCC scale scores. Linear interpolation procedure was used when test scores were not continuous. Conclusions RESEARCH QUESTIONS This study addressed the following research questions pertinent to reading and mathematics in Grades 3– 8. 1. How did fall/spring MAP RIT scores correlate to PARCC scale scores? 2. What were thresholds on fall MAP RIT scores that were associated with 65% and 75% probability (likelihood) of meeting the college and career readiness benchmark on PARCC? How accurately did fall MAP RIT scores predict college and career readiness on PARCC? 3. How did spring MAP RIT scores correspond to PARCC scale scores? SAMPLE Linking samples were constructed based on MAP test administrations (fall and spring) and content area (reading and mathematics) for Grades 3– 8, including students taking both MAP and PARCC assessments in 2014– 2015 (Table 1). All Grade 3– 8 students in the district took fall and spring MAP-Reading (MAP-R). All elementary school students, but only half of middle school students took fall and spring MAP- Mathematics (MAP-M). Middle school students who did not take MAP-M were not included in relevant mathematics samples. Middle school students who took PARCC Algebra 1 or Algebra 2 instead of grade-level PARCC assessments were also excluded from the mathematics samples due to a lack of comparable assessments for MAP. Therefore, the analytical results for middle school mathematics may not be generalized to all middle school students in the district. Prediction of college and career readiness on PARCC based on fall MAP RIT cut scores were identified to predict college and career readiness on PARCC with high probability of 65% and 75%. For example, a fall MAP-R RIT score of 205 in Grade 3 is associated with 75% probability of meeting the benchmark on PARCC English Language Arts/Literature (ELA) (Table 3). Prediction accuracy was then examined to determine whether observed cut scores precisely estimated students’ actual performance based on their predicted performance on PARCC. For both reading and math, the rate of accurate prediction was greater than 80% across the grade levels (Tables 4 and 5). Prediction accuracy was also estimated for Grades 3– 8 reading and mathematics by student subgroups. A similar pattern was observed across content and grade levels. Taking Grade 3 reading as an example, the prediction accuracy related to MAP-R RIT cut scores was 80. 3% for all students, 74. 8% for White, 75. 9% for Asian, 83. 9% for Black or African American, and 86. 7% for Hispanic/Latino students. The prediction accuracy appeared to be higher for Black or African American and Hispanic/Latino students than for their peers of other races. In Grade 3, the accurate estimation rates ranged from 86. 7% to 91. 9% for students with limited English proficiency (LEP), or for those receiving Free and Reduced-price Meals System or special education services. • Demonstrated strong predictive and concurrent validity evidence for MAP and PARCC assessments in Grades 3– 8 reading and mathematics. • The predictive study results with high accuracy support the continuous use of MAP assessments in the district for predicting PARCC success. • The results can provide guidance for instructional practice to improve student academic performance toward meeting the college and career readiness benchmark on PARCC. • Given that MAP assessments are vertically equated, the concordance tables may also provide guidance to school administrators and teachers for instructional planning; the earlier a student reaches the spring MAP RIT threshold score during the year (e. g. , in fall or winter), the more likely s/he will be college and career ready. • The concordance tables allow the district to use the MAP and PARCC scores interchangeably for its accountability system when necessary REFERENCES Dorans, N. J. , Moses, T. P. & Eignor, D. R. (2010). Principles and practices of test score equating. Educational Testing Service, Princeton: New Jersey. Holland, P. W. & Dorans, N. J. (2006). Linking and equating. Educational Measurement (4 th edition) edited by Robert Brennan. Praeger Publisher, Westport: CT.