Multiple Measures Assessment Project Math English Models from

Multiple Measures Assessment Project: Math & English Models from Phase II Multiple Measures Pilot College Convening - North Cañada College Terrence Willett October 28, 2015

MMAP variables for predicting CC success English • • Cumulative HS GPA AP English class grade Score on English CST Grade in last English class Math • Cumulative HS GPA • Grade in Algebra I, Algebra II, Statistics, Trigonometry, Calculus • Taking a challenging CST • Score on math CST

Why Decision Trees? • • • Provide easy-to follow and easy-to-code rule sets Create discrete categories No distributional or relational assumptions Visual presentation of decisions/decision-making They’re fun!

Increasing Homogeneity with each split ABZ ZAB Root Node BZA Branch Internal Node AAA ABA BAB ZZZ BBB Leaf Node

Statistics Tree – Direct Matriculants

Statistics Tree – NDM

English Decision Rules Direct matriculants Non-direct matriculants Transfer-level (criterion = 0. 70 / 0. 70) HS 11 GPA >= 2. 6 HS 12 GPA >= 2. 6 One-level below (criterion = 0. 65 / 0. 65) HS 11 GPA >= 2. 3 HS 12 GPA >= 2. 2 & HS 12 English course GPs >= 1. 8 Two-levels below (criterion = 0. 60 / 0. 55) HS 11 GPA >= 2. 0 HS 12 GPA >= 1. 8 & HS 12 English course GPs >= 1. 1 HS 12 GPA >= 1. 8 & CST >= 288 Three-levels below (criterion = 0. 55 / 0. 55) HS 11 GPA >=1. 4 HS 12 GPA >=1. 5 & CST>=268 HS 12 GPA >=1. 7 Four-levels below Everyone else

Level (CB 21 code, criterion) Hardcoded Pre. Requisite Algebra II Pre-Calculus (criterion = 0. 70) (or better) Direct Matriculants (Up through 11 th grade, no CST) Non-Direct Matriculants With CST Non-Direct Matriculants Without CST HS GPA>=3. 4 HS GPA>=3. 3 HS GPA>=2. 6 AND Calculus HS GPA>=3 AND Statistics (criterion=0. 70) Algebra I (or better) HS_11_GPA >= 3. 0 CST>=340 HS GPA>=3 AND Calculus C or better HS_12_GPA >= 3. 0 HS_11_GPA >= 2. 3 AND Pre-Calculus with C or better Algebra I General Education Math (or better) (Transfer Level) (criterion=0. 70) HS GPA >=3. 3 HS GPA>=3. 2 OR HS GPA>=2. 9 AND Stat C or better Algebra II (or better) HS GPA >=3. 2 HS GPA>=3. 2 College Algebra (criterion=0. 70) HS GPA >=2. 9 AND Pre-Calculus C or better

Higher level math rules Hardcoded Pre- Direct Matriculants Level (CB 21 code, criterion) Requisite (Up through 11 th grade, no CST) Non-Direct Matriculants With CST Non-Direct Matriculants Without CST Calculus I Pre-Calculus or HS GPA>=3. 6 (criterion = 0. 70) Trigonometry (or better) HS GPA>=3. 2 AND Pre. Calculus C or better HS GPA>=3. 1 AND Calculus HS GPA>=3. 5

Algebra II (CB 21=A, criterion=0. 65) HS 11 GPA >=2. 8 HS 12 GPA >=2. 9 HS 12 GPA >=2. 5 AND Algebra II CST >= 302 HS 12 GPA >=2. 5 AND Pre-Calculus with C or better Algebra I HS 11 GPA>=2. 4 (CB 21=B, criterion = 0. 55) HS 12 GPA>=2. 5 HS 12 GPA>=2. 3 AND Algebra II C or better Pre-Algebra (CB 21=C, criterion 0. 50) HS 11 GPA>=2. 0 HS 12 GPA >=2. 1 Arithmetic (CB 21=D) Everyone else Basic Skills Math Decision Rules HS 12 GPA>=2. 0 AND Algebra I CST >= 302 HS 12 GPA >=2. 1 HS 12 GPA >=1. 6 AND Algebra 1 Cluster 1 (Number Properties, Operations, and Linear Equations) >= 8 Everyone else

Correlations with College Grade: 11 th grade Cumulative GPA vs. Accuplacer English Transfer 1 level below 2 levels below 3 levels below 4 levels below 11 th Grade GPA Accuplacer 0. 27 0. 24 0. 25 0. 18 0. 21 0. 10 0. 12 0. 07 Math Transfer - STEM Transfer – Stats Transfer – LAM 1 level below 2 levels below 3 levels below 4 levels below 11 th Grade GPA Accuplacer 0. 24 0. 31 0. 26 0. 28 0. 26 0. 23 0. 19 0. 16 0. 09 0. 21 0. 11 0. 05

Placement into Transfer-level with Current Placement Systems vs. Strong Multiple Measures (Statewide) 100% 90% 80% 70% 61% 60% 50% 42% 38% 31% 30% 20% 10% 0% English (n=103, 510) Current Math (n=143, 253) Disjunctive MM

Transfer Level English Placement Current Disjunctive MM 100% 90% 80% 74% 73% 70% 60% 50% 41% 40% 30% 53% 51% 24% 30% 20% 10% 0% Afr Am Latino Asian White

Transfer Level Math Placement Current 100% Disjunctive MM 90% 80% 65% 70% 60% 53% 50% 41% 40% 32% 30% 20% 51% 15% 22% 21% 10% 0% Afr Am Latino Asian White

Correlation between HSPGA and 1 st CC English grade How long is High School GPA good for? Decay function for the predictive utility of HSGPA on English grade 0, 4 0, 35 R 2 = 0, 6201 0, 3 0, 25 0, 2 0, 15 0, 1 0, 05 R 2 = 0, 8336 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 Semesters of delay (approx. 6 months each) HS 11 GPA HS 12 GPA Linear(HS 11 GPA) Linear(HS 12 GPA) Accuplacer

MMAP Research Team Contacts Mallory Newell The RP Group newellmallory@deanza. edu John Hetts Educational Results Partnership jhetts@edresults. org Terrence Willett The RP Group twillett@rpgroup. org Ken Sorey Educational Results Partnership ken@edresults. org Craig Hayward The RP Group chayward@rpgroup. org Daniel Lamoree Educational Results Partnership dlamoree@edresults. org http: //www. rpgroup. org/projects/multiple-measures-assessment-project

Predicting transfer-level grades Test Correlation with GPs Corrected correlation CTEP Grammar . 10 . 16 CTEP Reading . 06 . 09 CTEP Syntax . 08 . 12 Math Assessment Test . 21 . 29 11 th Grade GPA - English . 38 . 39 11 th Grade GPA – Math . 37 . 38

Cost Old model Semesters Units Cost to get all to Engl 1 A Placement at Transfer level Acceleration MIH + Acceleration 3 2 1 10 6 4 $1, 677, 552 $1, 084, 682 $208, 851 18% 29% 57%