Evaluation of the American Apprenticeship Initiative AAI Overview

Evaluation of the American Apprenticeship Initiative (AAI) Overview for AAI Grantees June 22, 2016

Presentation Overview § Our team and roles § Research questions § Study overview § Contact information § Questions Abt Associates | pg 2

Our Team and Roles § EVALUATION • Mc. Bassi & Company – U. S. Department of Labor § GRANTS • Employment and Training Administration (ETA) • Chief Evaluation Office (CEO) – Abt Associates = prime – Office of Apprenticeship National Office staff and Federal Project Officers – AAI Grantees • Urban Institute • MEF Associates • W. E. Upjohn Institute • Capital Research Corporation • George Washington University Abt Associates | pg 3

What Questions Will the Evaluation Address? § AAI evaluation team “rolled up” DOL interests into 6 overarching questions to focus evaluation – Do benefits accrue to employers of apprenticeship? – On the supply side, what are AAI grantees doing to generate apprenticeship slots? – On the demand side, what strategies do AAI grantees and employers use to identify strong candidates for apprenticeship? – What does an AAI apprenticeship look like? – What are the outcomes for apprentices? – What innovations and lessons form the basis for broader change and sustainability that encourages employers to adopt apprenticeships? § Four sub-studies will answer questions Abt Associates | pg 4

Knowledge Development § Goal: understand AAI grantee plans to implement apprenticeship and inform the evaluation design § Includes all 46 grantees § Key activities: – Review the Quarterly Performance Reporting (QPR) system – Review and summarize grantee applications – Clarification calls with all grantees – Site visits to subset of grantees – Summarize findings Abt Associates | pg 5

Knowledge Development Tasks and Timeline § Review and summarize grantee applications (June) § Schedule and hold clarification calls; recommend grantees for site visits (July) § Conduct site visits to 8 grantees (August) § Prepare knowledge development summary memo for DOL (September) – Finalize research questions – Summary of grantee characteristics – Assessment of what findings imply for evaluation design Abt Associates | pg 6

Implementation Study § Goal: Describe how AAI programs develop, operate and mature § Includes all 46 grantees § Answers multiple study questions: – What are AAI grantees doing to generate apprenticeship slots? – What strategies do AAI grantees and employers use to identify strong candidates for apprenticeship? – What does an AAI apprenticeship look like? – What innovations and lessons form the basis for broader change and sustainability that encourages employers to adopt apprenticeships? Abt Associates | pg 7

Implementation Study (cont. ) § Key dimensions to explore – Program context – Underrepresented groups and recruitment – Employer engagement and employer perceptions – Key organizational partnerships – Training strategy – Assessment of implementation and lessons learned § Data sources – Grantee documents – QPR – Two rounds of site visits to 6 to 10 grantees (4 = demonstration sites) – Potential sources: Grantee survey, employer survey, participant survey Abt Associates | pg 8

Outcomes Study § Goal: Document range of in-program and postprogram outcomes for apprenticeship participants § Includes all 46 grantees § Answers multiple study questions: – What strategies do AAI grantees and employers use to identify strong candidates for apprenticeship? – What does an AAI apprenticeship look like? – What are the outcomes for apprentices? Abt Associates | pg 9

Outcomes Study (cont. ) § Key dimensions to explore – Hourly wages and progression – Retention and completion rates for apprenticeship and preapprenticeship – Skills acquired – Credentials earned (ST apprenticeships) or worked towards (LT ones) – Post apprenticeship employment (for ST apprenticeships) § Data sources – QPR data – Quarterly wage records – Potential source: participant survey Abt Associates | pg 10

Other Evaluation Activities § Return on Investment Study (TBD) – Goal: Estimate the benefits and costs of apprenticeship to employers – Data sources: employer survey, site visits, QPR – Next step: memo to DOL that describes different design options and associated data sources § Demonstration (TBD) – Proposed two options; may identify more during knowledge development – Data sources: grantee and employer administrative data Abt Associates | pg 11

Additional Information DOL/ETA Abt Associates Michelle Ennis Contracting Officer’s Representative Ennis. Michelle@dol. gov (202) 693 -3636 Karen Gardiner Project Director karen_gardiner@abtassoc. com (301) 347 -5116 DOL/CEO Elizabeth Copson Deputy Project Director elizabeth_copson@abtassoc. com (301) 347 -5113 Janet Javar Senior Evaluation Specialist Javar. Janet. O@dol. gov (202) 693 -5954 MEF Associates Mike Fishman Knowledge Development Lead mike. fishman@mefassociates. com (703) 838 -2724 Abt Associates | pg 12

Questions Abt Associates | pg 13