Doing Whats Right Means Telling the Right Story

Doing What's Right Means Telling the Right Story: Reporting Student Equity Completion Outcomes Thursday, March 17, 2016 10: 00 AM

Agenda • Background • Cohort options and examples • Methods and technology • Questions 2

Rules of the Road

Who we are • Brad Phillips, Ph. D President/CEO • Mary Kay Patton, MPPA Project and Data Quality Manager • John Watson, Ph. D Sr. Director Analytics 4

Poll: Who is with us today? Which of the following best describes you… a. District/campus administrator b. Educational researcher c. Student equity coordinator d. IT or database personnel e. Faculty member f. Other 5

What we do • IEBC helps education stakeholders use data and information to make informed decisions, improve practice, and increase student success. • IEBC has extensive experience with a collaborative, action research-based model that links data across educational segments. 6

Takeaways • Learn how some cohort definitions may exclude the very students you are trying to address • Understand methods for building student cohorts for student equity analysis and why using one method over another really matters • Review tools and techniques used for developing metrics for cohort analysis 7

Background 8

Background • We have our inclusive, equity hat on today • The focus today is on the completion metrics • Much of the source for this data, as recommended by the CCCCO is from the ARCC report (Student Success Scorecard) – Never intended for this use • If we want to assume that all students (or almost all) come to college hoping to achieve, behavioral metrics are not appropriate • A way to check this: “Are all students equally represented in the cohort? ” • If not, there is a problem with how the cohort is constructed 9

Examples of Alternate Cohorts 10

Introduction • Data set used comes from CA community colleges who have participated in our CA Transitions project • We start with FTIC students in the 08 -09 academic year • We are developing the cohort with an eye on only completion • Our base cohort references the Student Success design: 11

Altering Filters Distinct Student Counts, FTIC, academic year 08 -09 Had a minimum of 6 units 59, 656 12

Altering Filters Distinct Student Counts, FTIC, academic year 08 -09 Had a minimum of 6 units Attempted credit courses earning some but fewer than 6 units 59, 656 6, 836 11% more 13

Altering Filters Distinct Student Counts, FTIC, academic year 08 -09 Had a minimum of 6 units 59, 656 Attempted credit courses earning some but fewer than 6 units Attempted credit courses, but received 0 units Total amended cohort: 6, 836 11% more 26, 475 44% more 92, 967 14

Student Equity • • Race/Ethnicity Age Gender Financial Aid DSPS Veterans Foster Youth 15

Demographic Changes • After developing several alternate cohorts, we began review of various characteristics, looking for differences in demographics that may be hidden in the alternate cohorts • Initially looked at race/ethnicity within the cohort of students who registered for credit courses but did not earn the 6 or more units. • The following findings are from a subset of the districts we generated alternate cohorts for. • Note that these characteristics are not present in all districts. Individual district demographics and other characteristics likely affect individual findings 16

Demographic Changes >= 6 units earned 9% Black 23% Hispanic 17

Demographic Changes. 1 to 5. 9 units earned 16% Black 26% Hispanic 18

Demographic Changes 0 units earned 15% Black 26% Hispanic 19

Demographic Changes Percentage composition, select ethnicities Asian Black Hispanic White Original Alternate Cohort cohort Change 10% 4% -6% 9% 15% 6% 23% 26% 3% 42% 37% -5% Range of Change across districts -11. 1% to 2. 4% -0. 9% to 6. 2% -15. 2% 1 to 16. 2% -5. 9% to 8. 7% 2 Notes: Most districts showed increases in percentage of black and Hispanic students in alternate cohorts and decreases in percentage of white and Asian students. 1 This min was only found at one district. Other districts were in 0% to max range. 2 This max was only found at one district. Other districts were in -5. 9 to 0% range. 20

Methods and Technology 21

Developing alternate cohorts • Identifying questions of interest early helps to shape the alternate cohort effort • In this work, we used Microsoft SQL, Tableau Desktop and Excel. We also have utilized R and STATA and Dundas BI in other projects. • Limitations to data available: Locally held and CCCCO referential files don’t provide the complete picture due to activity outside of local district

Developing alternate cohorts Method used • • Use known data set: referential files Initial exploration based on informed question or hunch Developed initial cohort, and alternate models Validate cohort results against other sources Review cohort results to address questions of interest Develop metrics for cohorts that best fit local needs Develop final output

Developing alternate cohorts Data sourced and loaded Cohort option 1: credit threshold Cohort tables created Cohort option 2: student goals Cohort option 3: Attempt Eng. /math Evaluate results Are they more inclusive and show disparity? Cohort option 4: Other No? OK, let’s slice this a different way. Yes. Develop report Share results 24

Follow-up 25

Things to think about With our equity hat on we recommend: • Need to focus on correctly and accurately understanding this population of interest • Investigate your district’s data using these alternative cohort definitions (or others) • Discuss these finding with the Student Equity workgroup at your college

Questions and Feedback

IEBC Contact Information Cohort-generation algorithms, metric and output technical design John Watson Email: jwatson@iebcnow. org Phone: (530) 204 -7129 Questions about CCCCO data, data quality, cohort-definitions Mary Kay Patton Email: mkpatton@iebcnow. org Phone: (916) 995 -3183 General questions about student equity Brad Phillips Email: bphillips@iebcnow. org Phone: (619) 252 -8503 For more information about IEBC Analytics, visit www. iebcnow. org and click on Our Services > Analytics

End.