Updates on the Career Technical Education Incentive Grant

Updates on the Career Technical Education Incentive Grant and the Kindergarten through Grade Twelve Strong Workforce Program Posted by California Department of Education September 25, 2020 STATE BOARD OF EDUCATION Linda Darling-Hammond, State Board President CALIFORNIA DEPARTMENT OF EDUCATION Tony Thurmond, State Superintendent of Public Instruction CALIFORNIA COMMUNITY COLLEGES CHANCELLOR’S OFFICE Eloy Ortiz Oakley, Chancellor 1

Overview • The 2020– 21 Career Technical Education Incentive Grant (CTEIG) Application • The 2020– 21 Kindergarten through Grade Twelve Strong Workforce Program (K– 12 SWP) Application • Summary Data Reports on the CTEIG and the K– 12 SWP • Memorandum of Understanding (MOU) on Data Sharing between the California Department of Education (CDE) and the California Community Colleges Chancellor’s Office (CCCCO) • Recommendations on the 2020 CTEIG and K– 12 SWP Metrics 2

Career Technical Education Incentive Grant The purpose of this program is to encourage, maintain, and strengthen the delivery of highquality career technical education (CTE) programs. California Education Code, sections 53070– 53076. 4 3

FY 2020– 21 CTEIG Application (1) • Application released Monday, August 21, 2020 • Updates: o Criteria for renewal grants adhere to California Education Code o Implements streamlined process for renewal grants o Allows applicants to explain unique conditions o Electronic submittal process 4

FY 2020– 21 CTEIG Application (2) • Updates continued: o. Includes the 12 Essential Elements of a High. Quality College and Career Pathway as a reference o. Includes Technical Assistance Provider and Pathway Coordinator contact information o. Frequently Asked Questions (FAQs) created and posted 5

CTEIG Webinars (1) Provided three webinars which included: • ADA funding distribution per statute (<140 Small, 141– 549 Medium, >549 Large) • Renewal grant information – looking at multiple measures including expenditures and data reported • Application review process, with a total of 52 points possible, and 27 points to be eligible for funding • Appeals • Payment Procedures 6

CTEIG Webinars (2) • Reviewed two application process procedures: 1. LEAs that received 2019– 20 funds 2. LEAs that did not receive 2019– 20 funds • • High-Quality CTE Program Evaluation and Plan Budget Narrative Worksheet Three-Year Budget Plan Inclusion of 2019– 20 Local Control Accountability Plan (LCAP) pages with CTE references • Application preparation and submission information • FAQs 7

CTEIG Application Timeline • Applications due September 17, 2020 • Proposed Allocations to November 2020 State Board of Education Meeting • Grant Award Notifications out to LEAs in November 2020 • Funds to LEAs in December 2020 8

Member Comments Members, Please use the “Raise Hand” feature in Zoom which can be found in the “Participant” tab. Staff will call your name so you can make your comment. Please remember to lower your hand place yourself on mute after you have completed your comment. 9

Updates to the 2020– 21 SWP K– 12 Application and Measuring the Work of K– 12 Round 1 Grantees September 25, 2020 10

K– 12 SWP 2020 Timeline (1) DATE EVENT August 27 – October 16, 2020 K– 12 SWP Online Application submission window open September 2, 2020 Bidders’ Conference Webinar Regional Engagement Information Sessions NOVA platform applicant training Questions Submission Deadline to K 12 SWP@cccco. edu September 2020 September 18, 2020 September 30, 2020 11

K– 12 SWP 2020 Timeline (2) DATE EVENT October 16, 2020 K– 12 SWP Applications due in NOVA system October 19 – November 20, 2020 Regional K– 12 Selection Committees review applications and submit nominations of proposed grantees to CCCCO November 25, 2020 K– 12 SWP preliminary awards announced 12

Round 3 Updates and Changes 1. Streamlined the RFA 2. Simplified the format and upload of the work plan 3. Expanded character count and added a budget narrative 4. Revised ADA per SB 117 5. Ensure that match funds are 100% financial 13

K– 12 SWP RFA: Round 3 Updates and Changes • Feedback solicited from multiple stakeholders • Criteria for implementing suggested changes: • • • Reduce the administrative burden on applicants Adhere to Education Codes Minimize costs associated with changing online platform Provide year-to-year continuity for applicants Improve format to effectively describe proposed plans and programs 14

Measuring the Work of K– 12 SWP Grantees Round 1 LEA Survey Overview 15

K– 12 SWP LEA Interim Measures Reporting Timeline (1) February–July 2020 - Gathering Stakeholder Input • K– 12 LEA Leaders • K– 12 SWP Key Talent (especially K– 14 TAPs) • Regional Consortia Leads • CCCCO June–July 2020 - Preparing Survey and Materials • Design survey tool that can be matched with NOVA data 16

K– 12 SWP LEA Interim Measures Reporting Timeline (2) August 2020 - Survey is Live • LEAs will access online data links distributed by regional consortia September 30, 2020 - Interim Data Reports Due October 2020 - Data Analysis (West. Ed/CCCCO) November 2020 - Report completed 17

K– 12 SWP Interim Measures (1) https: //bit. ly/2 Xlk. VT 2 Data to be collected by Round 1 LEA Grantees (September 2020): • Types of activities supported by the K– 12 SWP grant in 2019– 2020 • Total number of students enrolled in K– 12 SWP-supported CTE pathways, by sector in 2019– 2020 • Total number of students enrolled in K– 12 SWP-supported early college credit in (dual enrollment, high school articulation) offerings in 2019– 2020 18

K– 12 SWP LEA Interim Measures (1) INTENT: Improve our understanding of the impact of the grant over the course of the funding cycles PLATFORM: Via online survey that requires plain ID numbers and pathway name in order to match with data in NOVA STRUCTURE: Survey aligns with the purpose and the strategies to glean knowledge about specific approaches 19

K– 12 SWP LEA Interim Measures (2) 2020 Survey will tell us: • Types of activities supported by the K– 12 SWP in the 2019– 20 school year (Round 1) • Total number of students enrolled by the K– 12 SWPsupported CTE pathways by CDE sector • Total number of students enrolled in K– 12 SWPsupported early college credit offerings in 2019– 20 20

LEA Survey Basics Due September 30, 2020 via online survey ➢Each Round 1 K– 12 SWP LEA lead and partner(s) is required to respond ➢There are up to 14 questions. Estimate 30– 45 minutes to complete (if all data are organized in advance). ➢West. Ed provides bi-weekly updates to K– 14 TAPs on data completion rates by region. First update was September 8 th. 21

What will LEAs need to complete? (1) • • K– 12 SWP Grant Plan via NOVA, including Plan ID number, grant title and school district/LEA name Official CDE school codes for each school engaging in K– 12 SWP -funded projects called the CDS. The CDS code is a unique, 14 -digit code representing the county, district and school. You can find CDS codes in this CDE finder tool: https: //www. cde. ca. gov/schooldirectory 22

What will LEAs need to complete? (2) • • The types of strategies LEA is engaged in as indicated in their K– 12 SWP grant application: 1) Curriculum & Instruction 2) Career Exploration 3) Postsecondary Transition and/or 4) Work-based Learning The courses included in K– 12 SWP-funded pathways by CALPADS course codes from the 2019– 20 school year 23

Support for LEAs Key Talent: Tiered System of Support ØK– 14 TAPs and K– 12 PCs received training in how to complete the survey, how to access data, and how to support LEAs to complete the survey. ØK– 14 TAPs received additional training on how to support K– 12 PCs ØK– 12 PCs to work directly with LEAs to ensure receipt and completion of survey. 24

Survey Limitations Provides a descriptive understanding of program implementation, not student level outcome measures. Provides a broad set of activities that LEAs are engaging in across pathways, not details about the strategies by student. By providing data on pathway courses, specifically course codes, it will show number of students benefitting by SWP work but will not show student progression through a sequence of courses. 25

Where to find the survey and instructions LEA SURVEY https: //bit. ly/K 12 SWPLEAData. Collection. R 1 INSTRUCTIONAL GUIDE https: //bit. ly/K 12 SWPLEAData. Guide 26

Member Comments (2) Members, Please use the “Raise Hand” feature in Zoom which can be found in the “Participant” tab. Staff will call your name so you can make your comment. Please remember to lower your hand place yourself on mute after you have completed your comment. 27

Summarizing CTEIG and K– 12 SWP Data Reports • Review of qualitative data • Distribution of CTEIG expenditures • Follow up data for CTE completers 28

Methodology Used for Analyzing CTEIG Qualitative Data (1) • Based on the Tool and the LEA’s answers to each of the 15 questions from 2018– 19 and 2019– 20 • First 15 charts provided in Attachment 1 show LEAs scored based on a point system of 0– 3 o 0 = Not yet in practice o 1 = Emerging practice o 2 = Quality practice o 3 = Exemplary practice 29

Methodology Used for Analyzing CTEIG Qualitative Data (2) For each of the 15 questions, the data was organized as follows: • Large Category >550 ADA • Medium Category 141– 549 ADA • Small Category <140 ADA • Overall Sample Average • Statewide Average 30

CTEIG Qualitative Data Results (1) • When reviewing 2018– 19 and 2019– 20 data, areas of strengths (defined as an average score greater than or equal to two on the self-evaluation in both years) are: • ADA Category – Less than or equal to 140 o Career Exploration/Guidance (Figure 2) o Support services for students (Figure 3 A) o Industry/Labor partnerships (Figure 5 A) o Experiential learning (Figure 5 B) o Alignment to labor market demand (Figure 7 A) o Professional development (Figure 9 B) 31

CTEIG Qualitative Data Results (2) • When reviewing 2018– 19 and 2019– 20 data, areas of strengths defined as an average score greater than or equal to two on self-evaluation in both years) are: • ADA Category – >140 to <=550 o Targeted equity and access for all students (Figure 10) • ADA Category – >550 o No criteria on self-evaluation is higher than two 32

CTEIG Qualitative Data Results (3) • When reviewing 2018– 19 and 2019– 20 data, areas of strengths defined as an average score greater than or equal to two on self-evaluation in both years) are: • Overall Sample Average o Targeted equity and access for all students (Figure 10) • Overall State Average o Staff by skilled teachers (Figure 9 A) 33

CTEIG Expenditures – Figures 11. and 12. Reported Expenditures 2018– 19 2019– 20 Increase/Decrease Salaries (1000– 3000) 19% 20% 1% Supplies (4000) Program Support/Services (5000) Industry Equipment/Facilities (6000) 48% 24% (-24%) 5% 7% 2% 28% 49% 21% 34

Follow Up Data - Summary (1) • Whether funded by either or both programs, nine out of ten CTE completers move on to postsecondary. • When funded by either CTEIG or K– 12 SWP but not both, marketing, hospitality, and health sciences are the top three industry sectors. • When funded by both CTEIG and K– 12 SWP, marketing, and hospitality are the top two industry sectors but followed closely by agriculture, health sciences, and public services. 35

Follow Up Data - Summary (2) • When funded by either CTEIG or K– 12 SWP but not both, around 80 percent of CTE completers are not acquiring third-party credentials. • When funded by both CTEIG and K– 12 SWP, about twothirds of CTE completers are acquiring a third-party credential. 36

Member Comments (3) Members, Please use the “Raise Hand” feature in Zoom which can be found in the “Participant” tab. Staff will call your name so you can make your comment. Please remember to lower your hand place yourself on mute after you have completed your comment. 37

MOU on Data Sharing Between the CDE and CCCCO 38

The CTEIG and K– 12 SWP MOU (1)

4 0 The CTEIG and K– 12 SWP MOU (2) • Focuses on the two funding programs • Student data is limited to grades nine through twelve • Program Data to be shared is CTE • Transfer of Data is from the CDE to CCCCO 40

4 1 The CTEIG and K– 12 SWP MOU (3) • Discussion occurring at three levels: program, data sharing, and legal. • Ongoing meetings between agencies to revisit status of all data sharing agreements including the CTEIG and K– 12 SWP MOU. • Intent: MOU in place by the FY 2021– 22 application cycle. 41

Member Comments (4) Members, Please use the “Raise Hand” feature in Zoom which can be found in the “Participant” tab. Staff will call your name so you can make your comment. Please remember to lower your hand place yourself on mute after you have completed your comment. 42

Recommendations on the 2020 CTEIG and K– 12 SWP Metrics 43

CTEIG and K– 12 SWP Metrics Discussion • Providing reasons for why the same recommendations are being made for CTEIG and K– 12 SWP metrics • Methodological and data collection considerations • Step-by-step explanation of the recommendation for each metric and underlying rationale 44

Reading the Matrix of Metrics Document (Attachment 3) • Columns A-F are CTEIG; Columns G-K are K– 12 SWP • Exact same information in Columns A-F as in Columns G-K • Row 2– 11 describes information on each metric • Columns C and H indicates proposed revision in metrics • Columns E and J indicate methodology applied • Columns D and G request acceptance of the proposed changes in metrics (Columns C and H) 45

Metric Recommendations for CTEIG and K– 12 SWP • The student population of interest (grades nine through twelve CTE students completing an industry pathway) • Potential CTEIG and the K– 12 SWP grantees coming from the same pool of LEA applicants • Both programs’ data are reported through CALPADS • Methodological precision and refinement allows for analytics within and across programs 46

Methodological and Data Collection Considerations (1) Definitions: • Four-Year Adjusted Graduation Cohort An unduplicated aggregate count of students in the adjusted cohort for the graduating class. . • CTE Pathway Completer (Denominator for all Metrics) A CTE student who completes at least 300 hours of course sequence in an industry pathway, and the sequence includes a capstone course; and the CTE student receives a grade of C- or better in the capstone course. 47

4 8 Methodological and Data Collection Considerations (2) • Indicators expressed in absolute numbers or as percentages; latter is generally used • To express as percentage, need numerator and denominator • Explanation of the use of CALPADS as the only data collection source going forward for consistency • Translating the metrics list in the legislation to indicators 48

Recommendation One Metric (from legislation): The number of pupils completing CTE coursework. Recommendation 1: Change Metric to the number of pupils completing a CTE industry pathway. (Row 2, Columns C and H) Rationale: Permits defining the denominator for all indicators consistently 49

Recommendation Two Metric (from legislation): The high school graduation rate. Recommendation 2: No change to Metric Row 3, Columns C and H. Note the indicator refers to the number of CTE completers who graduate in four years with a regular high school diploma (Row 3, Columns E and J) 50

Recommendation Three (1) Metric (from legislation): The number of pupils meeting academic and career-readiness standards as defined in the College/Career Indicator associated with the California School Dashboard. Recommendation 3: Subject to the determination by the CWPJAC (Row 4, Columns E and J). 51

Recommendation Three (2) Options for operationalizing this metric to include the following: 1. Use the College/Career Indicator (CCI) definition the California School Dashboard. 2. Assessment proficiency scores for English Language Arts, Math, and Science. Recommend either, but if 1 is used then follow the CCI criteria. Row 4, Columns E and J. 52

Recommendation Four Metric (from legislation): The number of pupils obtaining an industry-recognized credential, certificate, license, or other measure of technical skill attainment. Recommendation 4: Change Metric to the number of pupils obtaining an industry-recognized certification. (Row 5, Columns C and H) 53

Recommendation Five Metric (from legislation): The number of pupils obtaining an industry-recognized credential, certificate, license, or other measure of technical skill attainment. Recommendation 5: Change Metric to the number of pupils completing work-based learning experiences. (Row 6, Columns C and H) 54

Recommendation Six Metric (from legislation): The number of pupils obtaining an industry-recognized credential, certificate, license, or other measure of technical skill attainment. Recommendation 6: Change Metric to the number of CTE completers finishing college coursework. (Row 7, Columns C and H) 55

Recommendations 4, 5, and 6 Metric (from legislation): The number of pupils obtaining an industry-recognized credential, certificate, license, or other measure of technical skill attainment. Operationalize this metric to include the following: Use the CCI definition the California School Dashboard. Recommend sub-dividing the compound metric into three separate indicators. Rows 5, 6, and 7 in Columns E and J 56

Recommendation Seven Metric: The number of former pupils employed and the types of businesses in which they are employed. Recommendation 7: No change to Metric Row 8, Columns C and H. 57

Recommendation Eight • Metric: (vi)The number of former pupils enrolled in each of the following: (I) A postsecondary educational institution. (II) A state apprenticeship program. (III) A form of job training other than a state apprenticeship program. Recommendation 8: No change to Metric. The metric itself is sub-divided into three separate indicators, which are to be collected within CALPADS (Rows 9, 10, and 11, Columns E and J). 58

Recommendation Summary (1) Change to Metric through redefining • Number of pupils completing a CTE industry pathway. Change to Metric through sub-division • Number and percentage of CTE completers who receive an industry-recognized certification. • Number and percentage of CTE completers who complete a work-based learning experience. • Number and percentage of CTE completers finishing college coursework. 59

Recommendation Summary (2) Change to Metric through Choice of alternatives • Operationalizing College and Career Readiness or Assessment Scores No change to following Metrics: • The high school graduation rate. • The number of former pupils employed and the types of businesses in which they are employed. • The number of former pupils enrolled in a postsecondary educational institution. 60

Recommendation Summary (3) No change to following Metrics (continued): • The number of former pupils enrolled in a state apprenticeship program. • The number of former pupils enrolled in a form of job training other than a state apprenticeship program. 61

Recommendation The staff of the CCCCO and the CDE recommend that the CWPJAC review and approve the revisions in the 2020 Matrix of Metrics (Attachment 3), and to allow staff to make any necessary, non-substantive clarifying edits to the Matrix of Metrics. 62

Member Comments (5) Members, Please use the “Raise Hand” feature in Zoom which can be found in the “Participant” tab. Staff will call your name so you can make your comment. Please remember to lower your hand place yourself on mute after you have completed your comment. 63

Public Comment Zoom Register at: https: //us 02 web. zoom. us/webinar/register/WN_v. CW 1 KTt. MRXamr. PX Z 0 k. EQQA When logging into Zoom please use your first and last name to provide public comment. Email Path 2 Work@cde. ca. gov and include the following: 1) your name; 2) your affiliation, if any; 3) the agenda item number (i. e. , Item 01); and 4) your public comment. Phone 712 -432 -0075, Participant Access Code: 651905# Press *6 during the public comment period to be added to the queue. 64