Using Data to Drive AWD Program Improvement Sheila

Using Data to Drive AWD Program Improvement Sheila Thompson, AWD Coordinator C-TEC of Licking County August 1, 2014

Data and Compliance What data do we already collect for our funding and accreditation agencies? – Perkins/OBR/HEI – Accreditation (ACCSC, COE) – State Agencies/Boards (Public Safety, COS, Medical Board) – Financial Aid (Title IV, Pell, Loans, ODJFS, VA, WIA)

Using the Data You Already Collect Which performance indicators are on target? – For Perkins/HEI/OBR? – For ACCSC/COE? – For state boards? Where are the data gaps? – By student, by program, and by CTPD What other information could inform improvement? – – Advisory Committee Recommendations Student satisfaction surveys Stakeholder and client surveys Instructors, Coordinators, FA and Student Services Staff

Comparison of Performance Standards Performance Standard Perkins/HEI/OBR ACCSC COE Work Keys 91% for CTPD N/A Aggregate by CTPD, 70% Report and Justify by Program 70% by Program Student Completion Retention or Transfer 83. 5% by CTPD Depends on program length 60% by Program Student Placement 79% Related OR Unrelated, by CTPD 68% RELATED ONLY, by Program 70% RELATED ONLY, by Program Technical Skill Attainment Credential, Certificate or License

Perkins/HEI/OBR Data • CTPD Level Data (aggregate) • Based on State/Federal Targets, with Local Targets “negotiated” each year – Technical Skill Attainment – Credential, Certificate or Degree – Student Retention or Transfer – Student Placement – Nontraditional Participation – Nontraditional Completion

ACCSC and COE Data • Program Level Data • Timelines based on program length and accreditation dates, NOT fiscal or calendar year • Retention and Placement based on Benchmarks, not Targets – Enrollment – Retention/Completion – Credentialing – Placement

CTPD vs. Program Data CTPD Data: Why: Drives Perkins/HEI/OBR performance and approvals Program Data: Why: Drives COE and ACCSC accreditation approvals Ability to collect/use: Easy to find, if not always timely Ability to collect/use: Can be difficult to collect, verify, analyze and improve over time Relevance: Difficult to analyze and act on beyond CTPD level Relevance: Most important data you can collect, analyze, and act on

Data and Improvement To what extent does our required data drive improvement? • Enrollment • Retention/Completion • Credentialing • Placement

Enrollment Data • Drives FTEs (AWD funding) • Drives Title IV and other FA resources • Becomes the starting point for your concentrators/completers • Used to evaluate program retention rates • Used in ED Gainful Employment Disclosures

Perkins/OBR Enrollment Data

HEI Enrollment Data

Factors Impacting Enrollment • • Marketing Program/Center Reputation Program Calendar, Length, or Design Enrollment Process – Prerequisites – High School Diploma/GED – Work Keys – Financial Aid Approval and Program Cost

Retention and Completion Data • FUNDING (Pell, Title IV, WIA) • Subject to ACCSC, COE Accreditation Benchmarks • Program Level is more important than CTPD level

Perkins Retention Data

CTPD Retention Data

HEI Completion Data by Area

Factors Impacting Retention and Completion – Satisfactory Academic Progress (SAP) • Attendance • Grades – Student Barriers (transportation, child care, conflicting work schedules, drug and alcohol use, mental illness) – Program/Instructor Issues – Leave of Absence (LOA) – Setting the bar too low for program enrollment

Credentialing Data • Perkins/HEI/OBR Performance Target • Accreditation Benchmark (COE, ACCSC) • Board/Agency Benchmark (Public Safety, State Medical Board, Ohio Dept. of Health) • Required or highly recommended for employment

Perkins Technical Skill Attainment Data

HEI Technical Skill Attainment Data

HEI Report Work Keys • Applied Math • Locating Information

Perkins Credentialing Data

HEI Credentialing Data

HEI Credentialing Data

COE Credentialing Chart

Credentialing: ACCSC

Factors Impacting Credentialing Rates • Misalignment between curriculum, instruction, and credentialing exam • Students refusing to take the exam • Students unable to take exam (criminal background, did not meet licensure requirements) • Student lacks required academic or technical skills • Inaccurate reporting • Access to exam results

Placement Data • Perkins, Accreditation, FA requirement • Used in ED Gainful Employment Disclosures • For Accreditation purposes, RELATED employment is more important than ANY OTHER PLACEMENT CATEGORY

Perkins Placement Data

COE Placement Chart

Factors Impacting Placement Rates Student Factors: – Did not pass required credentials – Employability skills (work ethic, personal barriers) – Lack of interest in related field School Factors: – Inconsistent or incomplete reporting – Lack of emphasis on follow up and placement services Labor Market Factors: – Industry demand/current openings in related field

Using the Data You Already Collect Which performance indicators are on target? – For Perkins/HEI/OBR? – For ACCSC/COE? – For state boards? Where are the data gaps? – By student, by program, and by CTPD What other information could inform improvement? – – Advisory Committee Recommendations Student satisfaction surveys Stakeholder and client surveys Instructors, Coordinators, FA and Student Services Staff

Implications for AWD Culture • More accountability by program and instructor • Better system for collecting, analyzing and acting on data • More frequent and deeper communication within and among program staff • Increased collaboration among intake, testing, instructional, financial aid, student services, and administrative staff • Action plans (Institutional Assessment and Improvement, CIP, CCIP) that are actually used and understood

Rewards for Using Data to Improve • More collaboration among staff • Fewer “findings” or “opportunities for improvement” • Improved programming and services for students • Improved relationships with external stakeholders • Move from “good” to “great”!