Using data for program improvement Early Childhood Outcomes

Using data for program improvement Early Childhood Outcomes Center 1

Using data for program improvement Evidence Inference Action Early Childhood 2 Outcomes Center

Evidence • Evidence refers to the numbers, such as “ 45% of children in category b” • The numbers are not debatable Early Childhood 3 Outcomes Center

Inference • How do you interpret the #s? • What can you conclude from the #s? • Does evidence mean good news? Bad news? News we can’t interpret? • To reach an inference, sometimes we analyze data in other ways (ask for more evidence) Early Childhood 4 Outcomes Center

Inference • Inference is debatable -- even reasonable people can reach different conclusions from the same set of numbers • Stakeholder involvement can be helpful in making sense of the evidence Early Childhood 5 Outcomes Center

Action • Given the inference from the numbers, what should be done? • Recommendations or action steps • Action can be debatable – and often is • Another role for stakeholders Early Childhood 6 Outcomes Center

What can we infer? • Poll results A: – Candidate I. M. Good 51%, Candidate R. U. Kidding 49% (+ or – 3%) • Poll results B: – Candidate I. M. Good 56%, Candidate R. U. Kidding 44% (+ or – 3%) Early Childhood 7 Outcomes Center

Program improvement: Where and how – At the state level – TA, policy – At the regional or local level – supervision, guidance – Classroom level -- spend more time on certain aspects of the curriculum – Child level -- modify intervention Early Childhood 8 Outcomes Center

Key points • Evidence refers to the numbers and the numbers by themselves are meaningless • Inference is attached by those who read (interpret) the numbers • You have the opportunity and obligation to attach meaning Early Childhood 9 Outcomes Center

E – I – A Jeopardy 90% of exit COSFs COSF users in Program B unaware of the need missing a response to answer the yes/no to the yes/no progress question $100 Revise COSF procedures to emphasize completion of yes/no progress question Conduct staff development on using the 7 -point rating scale $200 75% of children in Program A received entry ratings of 2 $200 COSF users misunderstand the definition of points on the 7 -point scale Currently used tools are not accurately assessing children’s social emotional skills Invest resources in materials for assessing socialemotional skills 45% of children reported in category ‘e’ for statewide progress data, Outcome 1 10 $300 $100 $200 $300

Continuous Program Improvement Reflect Are we where we want to be? Check Plan (vision) (Collect and analyze data) Program characteristics Child and family outcomes Implemen t Early Childhood 11 Outcomes Center

Tweaking the System Reflect Is it working? Check (Collect and analyze data) Is there a problem? Why is it happening? Are we where we want to be? What should be done? Plan (vision) Program characteristics Child and family outcomes Implemen t. Is it being done? Early Childhood 12 Outcomes Center

Continuous means… …. the cycle never ends. Early Childhood 13 Outcomes Center

Outcome questions for program improvement, e. g. Who has good outcomes = • Do outcomes vary by… • Region of the state? • Eligibility category? • Services received? • Age at entry to service? • Type of services received? • Family outcomes? Early Childhood 14 Outcomes Center

Examples of process questions • Are ALL services high quality? • Are ALL children and families receiving ALL the services they should in a timely manner? • Are ALL families being supported in being involved in their child’s program? • What are the barriers to high quality services? Early Childhood 15 Outcomes Center

Working Assumptions • There are some high quality services and programs being provided across the state. • There are some children who are not getting the highest quality services. • If we can find ways to improve those services/programs, these children will experience better outcomes. Early Childhood 16 Outcomes Center

Examining COSF data at one time point • One group - Frequency Distribution – Tables – Graphs • Comparing Groups – Graphs – Averages Early Childhood 17 Outcomes Center