Progress Monitoring Oregon Reading First Cohort B Regional

Cohort B, Regional Meeting Content Development Content developed by: Carrie Thomas Beck Wayne Callender Rachell Katz Jeanie Mercier Smith Additional support: Katie Tate 2

Acknowledgments • Oregon Department of Education • Center for Teaching and Learning, College of Education, University of Oregon • Early Childhood Research Institute on Measuring Growth and Development – Select slides from this presentation were modified from the presentation entitled: Using DIBELS in an Outcomes Driven Model (Good, Kaminski, Re. Millard, Ebmeyer, Van. Loo, 2003) • U. S. Department of Education, Office of Special Education Programs 3

Copyright • All materials are protected by copyright and should not be reproduced or used without expressed permission of Dr. Carrie Thomas Beck, Oregon Reading First Center. Selected slides were reproduced from other sources and original references cited. 4

Reading First Initiative has Outlined Four Purposes for Assessment: • Outcome - Assessments that provide a bottom-line evaluation of the effectiveness of the reading program. • Screening - Assessments that are administered to determine which children are at risk for reading difficulty and who will need additional intervention. • Diagnosis - Assessments that help teachers plan instruction by providing in-depth information about students’ skills and instructional needs. • Progress Monitoring - Assessments that determine if students are making adequate progress or need more intervention to achieve grade level reading outcomes. 5

DIBELS and the Purposes of Assessment • Outcome – DIBELS benchmark goals provide overall outcome criteria • Screening – Able to assess all children three times a year to identify at-risk students • Diagnosis – Not an objective of DIBELS – May give direction for further assessment • Progress Monitoring – Sensitivity to growth and alternate forms allow frequent, ongoing monitoring 6

Progress Monitoring • Repeated, formative assessment to evaluate progress toward important goals for the purpose of modifying instruction or intervention. • Frequency of Progress Monitoring – 3 times per year for students at low risk (All Students) • Benchmark – 1 per month for students with some risk • Strategic – 1 per every two weeks for students at risk • Intensive 7

Effects of Progress Monitoring: Fuchs and Fuchs (1986) Reading achievement was improved by… – monitoring progress – graphing progress – adding decision rules – setting ambitious goals 8

DIBELS Progress Monitoring Data Base Logistics 9

DIBELS Progress Monitoring Logistics • • Progress Monitoring Materials Progress Monitoring Data System 1. Selecting Students 2. Selecting Measures (e. g. , 1 st Grade ORF for 3 rd grade Student) 3. Entering Scores • Progress Monitoring Graphs/ Reports 10

Materials • Materials for: – – – ISF PSF NWF ORF (1 st, 2 nd, 3 rd) (not LNF) • 20 Alternate Forms per measure • One booklet per measure • One Student Materials per measure 11

Progress Monitoring Data Base 12

Progress Monitoring Menu 13

Selecting Students 14

Selecting Students 15

Selecting Students 16

Entering Scores 17

Entering Scores 18

Selecting Out-of-Grade Measures • Two Methods of Entering “Out-of Grade” Measures 1. When Selecting Class/Grade 2. When Entering Data 19

Selecting Out-of-Grade Measures 1. Select Measure 2. For ORF only: Choice of Grade Level 20

Viewing Graphs/ Reports 21

Viewing Graphs/ Reports • Select School, Year, Grade, and Type of Graph (Color or Black & White) 22

Progress Monitoring Graphs 23

DIBELS Progress Monitoring: Guidelines • Administer 2 measures at each time: 1. The measure for the goal that the student is currently working on, and 2. The measure for the next most difficult goal. 24

Guidelines: • It is not necessary to progress monitor students on the Letter Naming Fluency (LNF) measure. LNF is primarily a measure of risk. • Do not progress monitor first, second, or third graders on the Initial Sounds Fluency (ISF) measure unless the student bottoms out on the Phoneme Segmentation Fluency measure (e. g. , scores less than 5). 25

Guidelines: • Continue monitoring on ISF, PSF, and NWF until the student has met the benchmark goal for two consecutive data points. • Once a student is progress monitored on ORF at his or her grade level, continue to monitor on ORF even after the student reaches the benchmark range. 26

Guidelines: • When progress monitoring on ORF, select the grade level passages that will be sensitive to the instruction provided. • It is only necessary to administer one passage each time a student is progress monitored on ORF. • Once a student meets the benchmark range on a particular grade level passage (e. g. , 40 -60 cwpm on first grade passages), then move the student up to the next grade level passages (e. g. , second grade passages) and work on the corresponding benchmark goal (90 cwpm). 27

Guidelines: Caution: If a student is being progress monitored using ORF passages below grade level, it is critical to plan ahead to provide a schedule that will allow the student to close the gap as quickly as possible. 28

Guidelines: First Grade Example • • • First grade benchmark assessment in Fall measures LNF, PSF, and NWF (ORF will start in the Winter) On the Fall data collection, Fred scored 18 on PSF and 7 on NWF. Monitor on: 1. PSF (goal he is currently working on) 2. NWF (the next most difficult goal) As soon as Fred meets the goal for PSF (35 or more) on two consecutive data points, then monitor on NWF and ORF. Once Fred meets the NWF goal, then continue monitoring on ORF for the remainder of the year. 29

Guidelines: Second Grade Example • In the Fall, Sarah scored 20 on NWF and 10 on ORF. • Monitor on: 1. NWF (goal she is currently working on) 2. ORF* (next most difficult goal) *Monitor with first grade ORF passages so as to be sensitive to intervention. • Note: It is probably a good idea to go back and check to see if your second graders made the benchmark for PSF. Look at Spring of first grade data. If student is not fluent on segmentation, you’ll want to provide some instruction on segmentation and thus monitor on PSF and NWF. 30

Guidelines: Third Grade Example • In the Fall DIBELS data collection, Kelsey scored 12 on ORF. • Administer PSF and NWF to determine if Kelsey has met the benchmarks. • Based on performance on PSF and NWF, select the two measures to monitor Kelsey on. Remember to begin with the first grade ORF passages if that is one of the goals you are monitoring her on. Place Kelsey in an intervention program that allows for acceleration and provide Kelsey with enough instructional time to make good lesson progress. 31

Guidelines: Third Grade Example • In the Fall DIBELS data collection, Les scored 50 on ORF. • Administer NWF to determine if Les has met the benchmark. • Progress monitor on NWF if necessary. If Les is placed in a second grade level of a decoding program, begin progress monitoring using second grade ORF passages. 32

Guidelines: Kindergarten Example • • • Kindergarten benchmark assessment in Fall measures LNF and ISF (PSF and NWF will start in the Winter) On the Fall data collection, Ginger scored 5 on ISF and has an intensive instructional recommendation. Monitor on: 1. ISF (goal she is currently working on) 2. PSF (the next most difficult goal) As soon as Ginger meets the goal for ISF (25 or more) on two consecutive data points, then monitor on PSF and NWF. Once Ginger meets the PSF goal, then continue monitoring on NWF for the remainder of the year. 33

Let’s Try Some. . . • Using the student data and grade level information on the DIBELS Progress Monitoring handout, determine which measures you would monitor each student on. 34

DIBELS Progress Monitoring Activity 35

“Easing In” to Progress Monitoring • With your ERT, develop a plan to begin progress monitoring students in October. • It is not necessary to have a “full blown” progress monitoring system in place these first few months. • Depending on the number of intensive students in your school, you may decide to monitor intensive students once a month to begin with. Similarly, you may decide to begin by monitoring strategic students once every 6 weeks. • Depending on the numbers of intensive and strategic students in your school, you may decide to progress monitor on just one measure to begin with. • Goal is to have a full progress monitoring system in place after the January DIBELS data collection. 36

Bringing Classroom Teachers and Specialists On Board • ORFC will be offering DIBELS administration training in the upcoming months (October, November, or December) to train classroom teachers and specialists to progress monitor. • All teachers and specialists should be trained on DIBELS administration by January of 2006. • Each classroom teacher should be progress monitoring at least 2 -3 of the students in their reading group(s) to gain experience with the measures. 37

How to Use Graphs to Summarize and Interpret Progress Monitoring Data Why not use a Graph? ? ! Wayne

Progress Monitoring Booklets 39

Benefits of Using a Graph • Creates a learning picture. • Allows for decision making. • Helps predict learning. • Provides documentation. • Makes data easier to interpret. 40

“What are major graphing conventions? ” • • • vertical lines (dots) are week lines (Week 1, Week 2, Week 3, Week 4) thick vertical lines indicate a change in month horizontal lines (dots) are counting/data lines you must plot the junction of the week and the data on one dot shaded area indicates benchmark range baseline depicts performance before an intervention aimline tells you the expected rate of learning use a phase line when you make a change in the intervention of a goal The Y (vertical) axis depicts performance The X (horizontal) axis depicts time/categories/nominal data

Data Should Be: • Stable – Three data points or more – Collected by a reliable assessment procedure – Collected in a relatively short period of time • Representative – Teacher says it is “typical” – Accurately reflects the student’s performance 42

Graphing the Baseline: Before Intervention Baseline Intervention 43

Validating Need for Support Phoneme Segmentation Fluency • Verify need for instructional support by retesting with alternate forms until we are reasonably confident. January cutoff 44 Good, Kaminski, Re. Millard, Ebmeyer & Van Loo, 2003

Exploring Support - What is an Aimline? Phoneme Segmentation Fluency The aimline connects where you are to where you want to get to, and shows the course to follow to get there. Aimline 45 Good, Kaminski, Re. Millard, Ebmeyer & Van Loo, 2003

“What is a data decision rule? ” • A decision rule is the systematic procedure by which patterns of data are analyzed. This data analysis assists in making a decision about the effectiveness of an intervention.

Why Decision Rules? • How do you know when to continue or modify an intervention? • Do you have unlimited time to continue with interventions that are not working? • Are students growing at the necessary rate required to reach the DIBELS benchmark? • Would you like to know which things work and which things don’t work for your students?

Applying the Decision Rule Three data points are plotted and the number of data points above or below the aimline are used to determine if the individual is achieving as predicted. • Administer 1 probe each week for 3 weeks or every other week until three data points are obtained. • Each data collection period, plot your student’s newest score. • This is a quick and easy method which reduces variability and eliminates the need for a trendline.

Data Decision Rules 1. If three (3) consecutive data points are above the aimline, raise the criteria

Data Decision Rules 2. If three (3) consecutive data points are below the aimline, change the intervention

Data Decision Rules 3. If neither of the above rules apply, make no change

Things to consider in decisionmaking: • Focus on the question: “Will the individual reach his/her goal by the end of the goal period? ” • Decide to change an intervention whenever the rate of progress falls below the expectation. • Think of changes to the intervention as fine tuning (alterable variables) rather than major reconstruction of interventions.

Changes in the intervention Draw a phase line A phase line is drawn vertically on the graph to identify the beginning point where the intervention change occurs.

Evaluating Support – Modify Instruction? Phoneme Segmentation Fluency Whoops! Time to make a change! Aimline 54 Good, Kaminski, Re. Millard, Ebmeyer & Van Loo, 2003

Phoneme Segmentation Fluency Evaluating Support - Is Instructional Support Sufficient Now? Aimline 55 Good, Kaminski, Re. Millard, Ebmeyer & Van Loo, 2003

Phoneme Segmentation Fluency DIBELS are the GPS for educators! Aimline 56 Good, Kaminski, Re. Millard, Ebmeyer & Van Loo, 2003

GPS On What course should I follow? Where How am are is I the doing? we? port? Port Desired Course You are Here Actual Course 57 Good, Kaminski, Re. Millard, Ebmeyer & Van Loo, 2003

Let’s Try One: Using the following information, set up a progress monitoring graph for Tony (a second grader) on NWF: • • • Sept. Week 3 (DIBELS Baseline) = 12 Draw in aimline. Oct. Week 1 = 14 Oct. Week 3 = 16 Nov. Week 1 = 16 Change Intervention? Add a phase line. Nov. Week 3 = 20 Dec. Week 1 = 28 Dec. Week 3 = 40 Is Tony on track now? 58

What are some ways I can organize the progress monitoring data for my small groups? Examples from Cohort A Coaches 59

Third grade intensive progress monitoring 60

61

62

63

Oral Reading Fluency Progress Monitoring (with Retell Fluency) – Grades 1 -3 Progress Monitoring - ORF and RTF Teacher: ORF RTF ORF RTF Name Who Monitor M 1 M 2 A 1 A 2 M 1 64

Kindergarten Progress Monitoring Example Progress Monitoring - ISF and PSF Teacher: ISF Name Who Monitor S 1 PSF O 1 O 2 N 1 N 2 D 1 J 1 F 2 M 1 M 2 A 1 A 2 M 1 65