Idahos Early Childhood Outcomes System Idaho ECOS Updated

Idaho’s Early Childhood Outcomes System (Idaho ECOS) Updated Measuring Early Childhood Outcomes Idaho Early Childhood Special Education Fall 2009 1

Agenda ¡ Measuring Early Childhood Outcomes: l Why, What, When, Who & How? ¡ Review Child Outcome Summary Form, (COSF-R) Tools & Resources ¡ Reporting the data on the Idaho early Childhood Outcome Collection System ¡ Practice Exercises ¡ 2

Training Objectives Participants will : ¡ Learn how to accurately identify, record, and report a child’s outcome data using the Idaho Child Outcome Summary Form-Revised (Idaho COSF-R) ¡ Increase valid and reliable data reporting on the ECO’s – timely and accurate 3

Goal of Early Childhood Special Education “…To enable young children to be active and successful participants and learners during the early childhood years in a variety of settings – in their homes with their families, in child care, in preschool or school programs, and in the community. ” Early Childhood Outcome Center 4

Why Collect Outcome Data? ¡ Federal government is the driving force behind the move to collect outcome data, however it is not the only reason… ¡Data on outcomes are also valuable l. To document program effectiveness l. To improve district programs l. To ultimately better serve children 5

WHY collecting outcome data makes sense for Idaho ü ü ü Improve current assessment practices and program development Promote progress toward age-expected skills and behavior throughout the child’s day Demonstrate value of Early Childhood Special Ed (Part B) programs Support DEC/NAEYC best practice guidelines Meet Federal Reporting Requirements -Office of Special Education Programs (OSEP) 6

OSEP’s Solution Document the number of children for whom the program has changed their developmental trajectories 7

Thinking about how children are doing with regard to each outcome. Movement away from age-expected Age-expected skills & behavior Movement toward age-expected 8

OSEP Indicators – categories of progress ¡ Percentage of children who: a. Did not improve functioning b. Improved functioning, but not sufficient to move nearer to functioning comparable to same age peers c. Improved functioning to a level nearer to same-age peers but did not reach it d. Improved functioning to reach a level comparable to same aged-peers e. Maintained functioning at a level comparable to same-aged peers 9

OSEP Indicators & Measurement Categories maintained or reached typical 5 year old level made progress but. . did not make progress 2 year old level Enry Exit 10

How do we document change in developmental trajectories while in services ? Obtain and compare a child’s assessment data at program entry and exit. 11

What happens with this data and why is accuracy and timeliness is so important? ¡ ¡ ¡ Individual student data is aggregated at state level States are required to report outcome data to Feds according to % of children exiting ECSE services – State must set targets and report progress toward meeting targets: l l % who entered program below age expectations in each outcome and substantially increased their rate of growth % who were functioning within age expectations in each outcome by time they exited

So, who do we collect outcome data on and when… ¡ All children enrolled in ECSE must have EC Outcome Entry data collected within 45 calendar days of initial consent for placement/IEP start date (if they will be in services for at least 6 months). ¡ For children transitioning from Part C – Part C outcome exit data may be used for Part B entry data, if an approved anchor tool was administered. 13

Who and when …. • EC Outcome Exit data will be collected within 30 days of the end of the school year (including ESY) for all children who: • Have been in Idaho receiving services for at least 6 months, and • Are 5 years old on or before Sept 1 st of the next school year (cut-off date for kindergarten entry), no matter if they are staying in a preschool setting or going into kindergarten. 14

EC Outcome Exit data must also be collected… l Within 30 days of determining a child no longer meets eligibility criteria (prior to age 5 and been in program at least 6 months) l 30 days prior to a child moving out of state ¡ If child leaves before 5 without notice and prior to administering the anchor assessment, complete the COSF-R with the information available. 15

Who is responsible to collect outcome data? ¡ ¡ Preschool teachers/SLP’s should take a lead role in gathering the necessary data/information from multiple people and measures through a team process Related service providers also provide assessment data, observation, etc. 16

What are the OSEP’s Early Childhood Outcomes? 17

OSEP’s 3 Child Outcomes 1. Children have positive socialemotional skills (including social relationships) 2. Children acquire and use knowledge and skills (including communication and early literacy) 3. Children use appropriate behaviors to meet their needs 18

Outcome 1 1. Children have positive social-emotional skills (including positive social relationships) • Relating with adults, and other children • For older children: following rules related to groups or interacting with others • Attachment/separation/autonomy • Responding to others, expressing emotions, turn-taking, using appropriate social greetings, play skills or having friendships with same-age peers, etc. 19

Outcome 2 2. Children acquire and use knowledge and skills (including communication and early literacy) • Using symbols and language • Thinking, reasoning, remembering, problemsolving • Understanding physical and social worlds • Early concepts – symbols, pictures, numbers, classification, spatial relationships • Imitation, object permanence • Expressing thoughts and ideas, listening to and enjoying stories and books, or learning new ways to do things, etc. 20

Outcome 3 3. Children use appropriate behaviors to meet their needs • Taking care of basic needs • Getting from place to place • Using tools like fork, toothbrush, crayon • In older children, contributing to their own health and safety • Self help skills (dressing, feeding, toileting, grooming, household responsibility) • Acting on world to get what he needs 21

The Child Outcome Areas… ¡Represent critical functional outcomes necessary and meaningful in every day activities and routines Not domains based – not trying to separate child development into discrete areas (communication, gross motor, etc. ) and single behaviors ¡ Emphasize how the child is able to integrate series of behaviors or skills across domains to achieve outcomes ¡ 22

Functional Outcomes ¡ What does a child typically do at this age level? ¡ Actual performance across settings and situations ¡ How child uses his/her skills to accomplish tasks ¡ Not the child’s capacity to function under ideal circumstances ¡ Not necessarily the child’s performance in a structured testing situation: " noncompliant” 23

So…how do we demonstrate that our programs produce good outcomes for children? … 24

Data will demonstrate that… ¡ ¡ Children have positive outcomes regardless of, their delays, disabilities, and functioning when they entered ECSE services. Individual child outcomes are better than they would have been without ECSE services. 25

Complicating Issues ¡ ¡ ¡ Many children progress with no intervention (maturing with age). Children with disabilities’ show diverse progress and under the best of programs, will experience different outcomes. Children receive different amounts, types and quality of services. 26

Quality Assessment Quality Service Better Outcomes 27

“Much of developmental psychology (early childhood testing) as it now exists is the science of the strange behavior of children with strange adults in strange settings for the briefest possible periods of time. ” (Bronfenbrenner, 1979) 28

What is assessment? “Early childhood assessment is flexible, collaborative decision-making process in which teams of parents and professionals repeatedly revise their judgments and reach consensus about the changing developmental, educational, medical, and mental health services needs of young children and their families. ” Bagnato and Neisworth, 1991 Quoted in DEC Recommended Practices, 2005 29

DEC Recommended Practices for Assessment ¡ ¡ Involves multiple sources (e. g. , families, professional team members, service providers, caregivers) Involves multiple measures (e. g. , observations, criterion-curriculum-based instruments, interviews, curriculumcompatible norm-referenced scales, informed clinical opinion, work samples) 30

Uses and limitations of assessment tools for measuring child outcomes… • • • Used to compare a child’s skills and behaviors to those of his same-age peers. Most are organized around domains. Not designed for direct measurement of the three outcomes. Some call for standardized assessment items to be administered in a setting other than the child’s natural environment, making it difficult to use the information to determine whether a child uses this skill in everyday life. Teams may have different information that differs, thus “overriding” assessment tool results. 31

Critical Assumptions Related to the Three Child Outcomes ¡ ¡ ¡ Measuring achievement of outcomes is based on comparison with age expectations Children of different ages will demonstrate achievement in different ways, so expectations change with age There are many pathways to functioning for children with atypical development (using sign language, wheelchair, etc. )

What information is gathered? Anchor Assessment. Norm-referenced, criterion-referenced or curriculum-based assessments Informed Professional Judgment & Observation Multiple sources of data are used to rate a child’s functioning Parent Input 33

Which anchor assessment tools will we use in Idaho? The following tools have been adopted for completing the outcome rating scale. ¡ ¡ ¡ BDI – II (Battelle) HELP (Hawaii) Creative Curriculum AEPS Carolina OUNCE Work Sampling High Scope (COR) Brigance Developmental Assessment of Young Children (DAYC) Bayley III (ITP program only) 34

Information to gather… Informed professional judgment l l Teachers, paraprofessionals, related service providers Anecdotal records, documented observations and data, progress reports, work samples, portfolios Parental/Caregiver input l l Best practice Provides critical information about how a child functions across the day and in a variety of settings 35

So, our challenge is… How do we take what we know about assessment and apply it to measuring the 3 outcomes… …when there is no single assessment tool that assesses all 3 outcomes directly. 36

“Crosswalks” were developed by National ECO Center to… ¡ ¡ Show the skills assessed by most commonly used published instruments map to the three outcomes. Provide comparisons across instruments to see their various strengths and weaknesses with regard to the three outcomes. 37

Early Childhood Outcomes Center 38

. So, how do we put it all together… 39

Child Outcomes Summary Form (COSF-R) ¡ ¡ ¡ Produces and compares data from programs across the state that are using different assessment tools A 7 -point scale for summarizing information related to a child’s progress on each of the three child outcome areas Rating based on child’s functioning: l Across settings and situations l Compared with what skills are expected given the child’s age 40

Child Outcome Summary Form Key Features ¡ ¡ ¡ Is NOT an Assessment Is NOT about eligibility determination Does NOT rate or summarize l l l ¡ Info about services provided to child Family’s satisfaction with services Information for planning services for child Blends diverse data from multiple sources into a consistent, reliable unit of measurement that can be aggregated and tracked over time 41

Child Outcome Summary Form Key Features ¡ ¡ ¡ Uses diverse information for global view of a child Compares a child’s functioning to sameaged peers Documents progress and movement toward typical development Considers functional abilities across diverse settings (not under ideal circumstances) Provides information about child’s overall sense of functioning in the three outcome areas 42

COSF Key Points ¡ Assumption: Children can be described with regard to how close they are to age-expected functioning for each of the three outcomes ¡ By definition, most children in the general population demonstrate the outcome in an ageexpected way ¡ Over time, some children will move farther away from age-expected functioning (skills at older ages are more demanding) ¡ By providing services and supports, programs are trying to move children closer to age-expected functioning ¡ Some children will never achieve this

Essential Knowledge for team members completing the COSF Know about the child’s functioning across settings and situations Understand age-expected child development Understand the content of the three child outcomes Know how to use the rating scale Understand age expectations for child functioning within the child’s culture Early Childhood Outcomes Center 44

Child Outcomes Summary Form Early Childhood Outcomes Center 45

The Form ¡ Cover page + three outcome pages ¡ On each outcome page: l l Two questions per outcome – one at entry and two at exit Space to document the summary of relevant results (and examples of ageappropriate, immediate foundational and/or foundational skills), which will be the basis for the rating Early Childhood Outcomes Center 46

The Two COSF Questions for each outcome a. To what extent does this child show behaviors and skills related to this outcome appropriate for his/her age across a variety of settings and situations? (Rating: 1 -7) b. Has the child shown any new skills or behaviors related to [this outcome] since the last outcomes summary? (Yes-No) Early Childhood Outcomes Center 47

Summary Ratings Reflect Global Functioning ¡ ¡ Ratings on each outcome are a snapshot of: l The whole child l Status of the child’s current functioning based on the various types of information from a variety of people who are familiar with the child across a variety of typical settings and situations l Team consensus to ensure validity and reliability Rather than reflective of : l Skill by skill, in one standardized way, or split by domains Early Childhood Outcomes Center 48

The Basis for the Ratings ¡ Scale runs from 1 -7 with a 6 or 7 indicating age appropriate functioning for the outcome ¡ Lower numbers indicate distance from age appropriate functioning for the outcome ¡ Resources – “Definitions for Outcome Ratings” and “Decision Tree for Summary Rating Discussions” Early Childhood Outcomes Center 49

Important Terms… ¡ ¡ Foundational skills are…. Skills and behaviors that occur earlier in development and serve as the foundation for later skill development Teachers and interventionists often use foundational skills to help children move to the next level developmentally Immediate foundational skills are… Skills that are conceptually linked to later skills and immediately precede the later skills developmentally Example: Children play alongside one another before they interact in play

7 – Completely ¡ The child shows behaviors and skills expected in all or almost all everyday situations that are part of the child’s life l Home, store, park, child care, with strangers, etc. ¡ The child’s functioning is considered appropriate for his/her age ¡ No one has significant concerns about the child’s functioning in this outcome area Early Childhood Outcomes Center 51

6 – Between Completely and Somewhat • The child’s functioning generally is considered appropriate for his or her age, but there are some significant concerns about the child’s functioning in this outcome area • May border on not keeping pace with age expectations so monitoring and extra support may be needed Early Childhood Outcomes Center 52

5 – Somewhat ¡ The child shows functioning expected for his/her age some of the time and/or in some situations ¡ The child’s functioning is a mix of ageappropriate and not appropriate functioning ¡ The child’s functioning might be described as like that of a slightly younger child Early Childhood Outcomes Center 53

4 – Between a 5 and a 3 ¡ Child shows some age appropriate functioning some of the time or in some situations or settings but most of the child’s functioning would be described as not yet age appropriate ¡ The child’s functioning might be described as like that of a younger child Early Childhood Outcomes Center 54

3 – Emerging ¡ The child does not yet show functioning expected of a child his/her age in any situation ¡ The child’s behaviors and skills include immediate foundational skills most or all the time, on which to build ageappropriate functioning ¡ The child’s functioning might be described as like that of a younger child Early Childhood Outcomes Center 55

2 – Between 3 and 1 ¡ The child does not yet show functioning expected of a child his/her age in any situation ¡ The child’s behaviors and skills do have some immediate foundational skills on which to build age-appropriate functioning but these are not displayed very often ¡ More functioning reflects skills that are not immediate foundational Early Childhood Outcomes Center 56

1 – Not Yet ¡ The child does not yet show functioning expected of a child his/her age in any situation ¡ The child’s skills and behaviors also do not yet include any immediate foundational skills on which to build age-appropriate functioning ¡ The child’s functioning might be described as like that of a much younger child ¡ Children with 1 ratings still have skills, just not yet at an immediate foundational level Early Childhood Outcomes Center 57

“Concerns” That Distinguish Ratings of 7 vs. 6 ¡ All children have strengths and weaknesses. Families and providers identify areas to work on to support ongoing growth (some call these “concerns”) ¡ What types of concerns would result in a rating of 6? Early Childhood Outcomes Center 58

“Concerns” That Distinguish Ratings of 7 vs. 6 ¡ ¡ Concerns raised for which families and providers may want to offer extra support and strategies to promote development, but the area of concern is not a possible indicator or precursor of a significant developmental problem (7) vs. Developmental concerns—weaknesses significant enough to watch closely and definitely support. Although age expected now, the child’s development borders on not keeping pace with age-expected levels or shows early signs of possible developmental problems (6) Early Childhood Outcomes Center 59

Where to Focus in Deciding the Rating ¡ Focus on the child’s overall functioning across settings and situations ¡ Functioning that is displayed rarely and/or when the child is provided with a lot of unusual support or prompts is of little significance for the rating Early Childhood Outcomes Center 60

The Process for Answering Questions 1 a, 2 a, 3 a For each outcome… 1. Discuss the child’s current functioning in this outcome area across settings and situations 2. Identify areas where the child’s functioning is age appropriate 3. If not all functioning is age appropriate, identify areas where the child’s functioning reflects immediate foundational skills 4. Decide which rating best describes the child’s current functioning Early Childhood Outcomes Center 61

Answering the Questions at Exit: 1 b, 2 b, & 3 b ¡ ¡ At Exit only, answer question “Has child shown new skills or made progress in outcome area? ” (Yes or No) Small steps of progress count! Child CAN make progress without “changing numbers” on the scale! Most will select “Yes”

Identifying who made progress ¡ ¡ Progress = moving up a point on the scale in a subsequent rating, e. g. , 3 to 4 Progress= staying at the same rating but having a “yes” for the progress question 63

Important considerations… • • • How to get team input Role of assistive technology and accommodations (i. e. sign language, wheel chair) Child’s culture and language Special considerations that impacted child’s development Reflect the child’s actual functioning – not what the child is capable of under the ideal or highly unusual circumstances 64

Parental Involvement ¡ ¡ ¡ Parental consent for evaluations is required, as always. Completion of the COSF is NOT an assessment and does not require parental consent. Parents give input to the COSF through assessment process and IFSP/IEP development. Parents need not be present when COSF is completed. Inform parents about process and purpose. 65

What will the process look like in Idaho? . . ¡ ¡ ¡ Team members, including the parents will be well informed and contribute information to the process An anchor assessment will be completed to assess the child in the three outcome areas Information (anchor assessments, parent input, and informed professional observation and judgment) be collected, compiled and documented using the COSF-R Team will make a rating decision (1 -7 and Y or N for exit) regarding a child’s level of functioning Data will be entered on the web-based data system Completed COSF-R from entry or exit will be placed in child’s file 66

Early Childhood Outcome Data Collection System ¡ https: //www. sde. idaho. gov/ECOutcome/default. aspx ¡ http: //itcnew. idahotc. com/dnn/Early. Childhood/tabid/10 92/Default. aspx ¡ Web-based Data Collection System Instructions and Technical Information https: //www. sde. idaho. gov/ECOutcome/docs/ ECOutcome. Instructions. pdf l ¡ Districts are required to enter student data on the website in a timely manner! 67

Practice activities…

Good outcome data is a tool to produce good outcomes for children and families. 70