OSEP Initiatives on Early Childhood Outcomes Kathy Hebbeler

OSEP Initiatives on Early Childhood Outcomes Kathy Hebbeler Early Childhood Outcomes Center SRI International Presented at ECEA-SCASS Meeting Savannah, Georgia October, 2010

Topics • What state are required to report • State approaches • Most recent data • Child Outcomes Measurement Framework Early Childhood Outcomes Center 2

Reporting Requirement for Early Intervention and Preschool Special Education Early Childhood Outcomes Center 3

Why does the federal government want data on child outcomes? • Government Performance and Results Act (GPRA) • Program Assessment Rating Tool (PART) • Individuals with Disabilities Education Act (IDEA) Early Childhood Outcomes Center

Government Performance and Results Act (GPRA) passed in 1993 l Requires goals and indicators be established for IDEA l Indicators and data collection for school age population included data on outcomes l Previously, for early childhood data had been collected on: • Number of children served (Part C) • Settings (both Part C and 619

OSEP: PART evaluation results (2002) l 130 programs examined in 2002; 50% programs had no performance data l Programs looking at inputs, not results l Part C and Section 619 § No long-term child outcome goals or data § Department of Education needs to develop a strategy to collect annual performance data in a timely manner Early Childhood Outcomes Center

IDEA 2004 SEC. 616. <<NOTE: 20 USC 1416. >> MONITORING, TECHNICAL ASSISTANCE, AND ENFORCEMENT. ``(a) Federal and State Monitoring. …. . (2) Focused monitoring. --The primary focus of Federal and State monitoring activities described in paragraph (1) shall be on-- (A) improving educational results and functional outcomes for all children with disabilities;

OSEP Reporting Requirements: Child Outcomes – Positive social emotional skills (including positive social relationships) – Acquisition and use of knowledge and skills (including early language/ communication and early literacy) – Use of appropriate behaviors to meet their needs 8

What States Report: OSEP Reporting Categories Percentage of children who: a. Did not improve functioning b. Improved functioning, but not sufficient to move nearer to functioning comparable to same-aged peers c. Improved functioning to a level nearer to sameaged peers but did not reach it d. Improved functioning to reach a level comparable to same-aged peers e. Maintained functioning at a level comparable to same-aged peers 3 outcomes x 5 “measures” = 15 numbers

Developmental Trajectories 70 Functioning like same aged peers Growth in Outcome 60 50 Improved functioning to that of same aged peers 40 Moved closer to functioning like same aged peers 30 Improved functioning, no change in trajectory 20 Did not improve functioning 10 0 0 6 12 18 24 30 36 42 48 54 60 66 Age in Months Early Childhood Outcomes Center 10

Reporting details • Progress for all children who exited between July 1, 2009 and June 30, 2010 • Stayed in the program at least 6 months • Data will be reported to OSEP in February 2011 • Data reported for the first time for children who exited in 07 -08 year. Early Childhood Outcomes Center 11

The Summary Statements 1. Of those children who entered the program below age expectations in each outcome, the percent who substantially increased their rate of growth by the time they turned 3 [6] years of age or exited the program. 2. The percent of children who were functioning within age expectations in each outcome by the time they turned 3 [6] years of age or exited the program. 12

Formula for SS 1 (c+d)/(a+b+c+d) 13 13

Formula for SS 2 (d+e)/(a+b+c+d+e) 14

State approaches Early Childhood Outcomes Center 15

Early Childhood Outcomes Center 16

Early Childhood Outcomes Center 17

State Approaches to Measuring Child Outcomes Part C (56 states/jur) 7/56 (13%) Preschool (59 states/jur) 9/59 (15%) Publishers’ online analysis 3/56 (5%) 6/59 (10%) COSF 7 pt. scale Other 41/56 (73%) 5/56 (9%) 38/59 (64%) 7/59 (10%) Approach One tool statewide

Child Outcomes Rating Form (COSF) • 7 -point rating scale with defined criteria for each point • Criteria describe child’s functioning relative to same aged peers • Child’s team uses multiple sources of information to assign rating • Rating assigned at program entry and program exit Early Childhood Outcomes Center 19

The State Data for ‘ 08 -’ 09 Early Childhood Outcomes Center 20

Early Intervention (Part C) Outcomes A, B and C Average of State Percentages a-e 40 35 N=66, 000 Outcome A 30 Outcome B 25 Outcome C 20 15 10 5 0 a b c d e

Summary Statement 1: Percent of children who increased their rate of growth while in Part C 80 70 60 50 baseline 40 target 09 target 10 30 20 10 0 Outcome A Outcome B Outcome C

Summary Statement 2: Percent of children who exited Part C at age expectations 70 60 50 baseline target 09 target 10 40 30 20 10 0 Outcome A Outcome B Outcome C

Early Childhood Special Education (Section 619) Outcomes A, B, and C Average of State Percentages a-e 40 N=113, 700 35 30 Outcome A 25 Outcome B Outcome C 20 15 10 5 0 a b c d e

Summary Statement 1: Percent of children who increased their rate of growth while in 619 90 80 70 baseline target 09 target 10 60 50 40 30 20 10 0 Outcome A Outcome B Outcome C

Summary Statement 2: Percent of children who exited 619 at age expectations 80 70 60 50 baseline target 09 target 10 40 30 20 10 0 Outcome A Outcome B Outcome C

A Framework and Self Assessment for Building a Child Outcomes Measurement System Early Childhood Outcomes Center 27

Purpose of the Framework • Provide a common language for ECO and other TA providers to use in discussing COMSs with states. • Provide a organizing structure of categorizing resources and state examples related to implementation of a COMS. • Serve as the organizing structure for the self assessment Early Childhood Outcomes Center 28

Framework and Self-Assessment • FRAMEWORK – Set of components and quality indicators – Provides the structure for the self-assessment • SELF-ASSESSMENT – Scale that provides criteria for levels of implementation within each quality indicator – Rating assigned based on level of implementation within each indicator Early Childhood Outcomes Center 29

Process for Framework Development • Built off what we had learned from ECO work with states and previous ECO conceptual framework • Literature review • Repeated discussion and review internally and with 7 Partner States 30

Framework Partner States State Part C California 31 619 X Colorado X X Delaware X X Maine X X Minnesota X X New York X Ohio X X

COMS Framework Components Purpose Data Collection and Transmission Analysis Reporting Using Data Cross-system Coordination Evaluation Early Childhood Outcomes Center 32

COMS Framework Components Purpose Data Collection and Transmission The state has effective Analysis Using Data Reporting procedures for collecting, storing, and transmitting data to the state. Cross-system Coordination Evaluation Early Childhood Outcomes Center 33

COMS Framework Components Purpose State coordinates child outcomes measurement and data use across EC Data Collection systems. Analysis Using Data Reporting and Transmission Cross-system Coordination Evaluation Early Childhood Outcomes Center 34

Quality Indicator • Provides additional detail as to what constitutes quality implementation of the component. • 18 quality indicators across the 7 components Early Childhood Outcomes Center 35

Quality Indicators for Data Collection and Transmission 2. Data collection procedures are carried out efficiently and effectively. 3. Providers, supervisors, and others involved in data collection have the required knowledge, skills, and commitment. 4. State's method for entering, transmitting, and storing data is effective and efficient. Early Childhood Outcomes Center 36

Structure of Self Assessment • Components (7) = Major areas of framework – Quality Indicators (18 total) = Statements of basic requirements of a quality COMS • Elements (number varies with each indicator) = Define what constitutes high quality on the Quality Indicator. Early Childhood Outcomes Center 37

Early Childhood Outcomes Center 38

Early Childhood Outcomes Center 39

40

Early Childhood Outcomes Center 41

42

The Scale for the Quality Indicators Implementation of Elements All elements are fully implemented Nearly all elements are fully implemented and the rest are in process Quality Indicator Score 7 6 Most of the elements are fully implemented and the rest are in process. 5 At least one element is fully implemented and the rest are in process 4 3 2 1 All of the elements are in process Some of the elements are in process None of the elements are yet in process Early Childhood Outcomes Center 43

Early Childhood Outcomes Center 44

Recommended State Approach for Using the Self Assessment 1. Complete the entire self assessment. 2. Identify the component(s) and quality indicators to address first. 3. Develop action plan to improve the related elements. 4. Implement improvement activities. 5. Re-assess status and identify “next step” priorities at regular intervals Early Childhood Outcomes Center 45

Is and Isn’t • Is a comprehensive resource to alert states to all the pieces that need to be in place to have a well functioning COMS • Is not a cookbook or roadmap with each step in the process spelled out. – Way too many decisions! Early Childhood Outcomes Center 46

Where would a state start? Purpose Questions (Analysis) Early Childhood Outcomes Center 47

Packaging • Entire tool will exist online – Live link from each element to a “back up” section – Profile will be filled automatically based on the QI pages • Will develop a version with live links to the back up and profile that will operate off line. • Version that can be printed off as a manual. Early Childhood Outcomes Center 48

Next steps for ECO • Populate the COMS framework with resources related to the components • Revise the self assessment based on feedback • Develop a framework for a Family Outcomes Measurement System Early Childhood Outcomes Center 49

System for Producing Good Child and Family Outcomes Prof’l Development Preservice Evidence • Based • Inservice Practice Good Federal policies and programs Good State policies and programs Good Local policies and programs Strong Leadership Adequate funding Information infrastructure High quality services and supports for children 0 -5 and their families Good outcomes for children and families

Early Childhood Information Infrastructure: Data Needed for Program Improvement WHO SERVICES OUTCOMES PERSONNEL Early Childhood Outcomes Center COST 51

For more information • For updates to the framework and the selfassessment and resources to support the quality indicators: www. the-eco-center. org Early Childhood Outcomes Center 52