Poll Question 1 Yes read them both I

Poll Question 1. Yes, read them both I have read Training and Employment Guidance Letters 718 and 23 -19 on data validation 2. Yes, read at least one 3. Not yet, it’s on my desk 4. Wait, there’s a TEGL? Insert your response in the chat room. 1

Poll Question 1. Yes 2. Not yet, it’s on my desk I have reviewed my state’s Quarterly Report Analysis 3. No 4. No, I’m not responsible for reviewing it Insert your response in the chat room. 2

Data Integrity Technical Assistance Tool: Report Analysis for WIOA Titles I and III August 18, 2020

Today’s Moderator Christina Eckenroth Workforce Analyst U. S. Department of Labor Employment and Training Administration 4

Today’s Speakers Cesar Acevedo Workforce Analyst U. S. Department of Labor Employment and Training Administration Robert Hoekstra Program Analyst U. S. Department of Labor Employment and Training Administration 5

Today’s Objectives § Overview of Quarterly Report Analysis (QRA) pilot § Example QRA § Pilot next steps § Feedback and Discussion 6

Background ØWIOA section 116(d)(5) requires the Departments to establish data validation guidelines to ensure the information contained in program reports is valid and reliable. Ø Joint Training and Employment Guidance (TEGL) 7 -18 provides States with a general framework for data validation on the performance accountability. ØDOL only data validation guidance TEGL 23 -19. 7

The Quarterly Report Analysis An extension of our datadriven technical assistance. An ETA-generated review of grantee quarterly data. Ensure data consistency and accurate reporting. 8

The Quarterly Report Analysis Addresses statutory and policy requirements for data validation Responds to technical assistance requests 9

ETA’s Data Integrity Approach Standard Data Validation Quarterly Report Analysis Report validation—WIPS calculates state performance reports to ensure standardized application of formula calculations Parameters: State data validation requirements Addresses data expectations where logical validations cannot be applied Valid value edit checks that ensure data are submitted according to the PIRL Set monitoring and technical assistance expectations Presents data graphically WIPS Logical validations enforce consistency across PIRL elements 10

Informs Data Quality REDUCE ERRORS Highlights areas of concern for additional investigation through monitoring, or technical assistance activities. SUPPORT SUCCESS 11

Data elements Selected elements across programs related to eligibility, performance outcomes, and other program requirements Subject to revision based on feedback and assessment “Targets” Reference points to serve as a technical assistance guide to demonstrate where data is in relation to expected reporting based on requirements and current reporting by all states Initial targets will be based on assessment of national trends and will be revised as necessary 12

Example Ø QRA products include a sheet with charts displaying the state’s performance relative to all states and a sheet with detailed results including numerator and denominator counts. Ø Both sheets include visual indications to highlight when the state’s results fail to meet, or exceed ETA’s expectations 13

Foster Continuous Improvement Quarterly Report Submission Implement TA & Improve Future Data Submissions ETA Issues QRA Results State Feedback 14

Implementation Share Results with States July 2020 Identify Pilot States July-August 2020 Engage in Feedback ETA will iterate and revise the QRA products based on feedback throughout the pilot period. 15

Expectations – DOL Disseminate state-specific QRA results to appropriate staff in each state. Review the QRA results to inform: conversations with states regarding their data integrity efforts and regular grant monitoring Notify states of any especially notable or troublesome results Receive/review any state responses to individual metric results Provide feedback 16

Expectations – Grantees All States: Review their state’s QRA results Use the information to review data integrity efforts Make necessary changes to improve QRA results Volunteer Pilot States: Commit to review the QRA results in a timely manner Provide feedback on the QRA product itself For each failed metric provide region with responses for: Problem Type Solve by Quarter Resolution 17

Questions and Discussion 18

Resources: Training and Employment Guidance Letter 7 -18 Guidance for Validating Jointly Required Performance Data Submitted under the Workforce Innovation and Opportunity Act (WIOA) published DOL Performance Website Data Integrity Resources www. dol. gov/agencies/eta/performance/reporting/ data-integrity December 19, 2018 https: //wdr. doleta. gov/directives/corr_doc. cfm? doc n=4255 Training and Employment Guidance Letter 23 -19 Guidance for Validating Required Performance Data Submitted by Grant Recipients of U. S. Department of Labor (DOL) Workforce Programs published June 18, 2020 https: //wdr. doleta. gov/directives/corr_doc. cfm? DO CN=9155 19

Contact Us ETAPerforms@dol. gov 20

Thank You! Need help? Email: Support@workforce. GPS. org