Response to Intervention RTI Student Literacy Assessment CurriculumBased

Response to Intervention RTI, Student Literacy Assessment, & Curriculum-Based Measurement: A Skill. Building Lab Jim Wright www. interventioncentral. org

Response to Intervention Power. Points & Handouts from Workshop Available at: http: //www. interventioncentral. org / GSTBOCES. php www. interventioncentral. org 2

Response to Intervention Workshop Topics 1. RTI Assessment Requirements & Curriculum-Based Measurement 2. Evaluating the ‘RTI Readiness’ of Literacy Measures 3. Review of Common CBM Literacy Assessments 4. CBM Oral Reading Fluency: Case Study 5. Collecting School-Wide CBM Norms: Procedures 6. Application of Local Norming Data to Evaluation of Core Literacy Programs, Development of CSE Eligibility Rules www. interventioncentral. org

Response to Intervention Five Core Components of RTI Service Delivery 1. Student services are arranged in a multi-tier model 2. Data are collected to assess student baseline levels and to make decisions about student progress 3. Interventions are ‘evidence-based’ 4. The ‘procedural integrity’ of interventions is measured 5. RTI is implemented and developed at the school- and district-level to be scalable and sustainable over time Source: Glover, T. A. , & Di. Perna, J. C. (2007). Service delivery for response to intervention: Core components and directions for future research. School Psychology Review, 36, 526 -540. www. interventioncentral. org 4

Response to Intervention RTI ‘Pyramid of Interventions’ Tier 3 Tier 2 Tier 1 Tier 3: Intensive interventions. Students who are ‘non-responders’ to Tiers I & II may be eligible for special education services, Tier 2 Individualized intensive interventions. Subset of students receive interventions targeting specific needs. Tier 1: Universal interventions. Available to all students in a classroom or school. Can consist of whole-group or individual www. interventioncentral. org 5

Response to Intervention Avg Classroom Academic Performance Level Discrepancy 1: Skill Gap (Current Performance Level) Target Student Discrepancy 2: Gap in Rate of Learning (‘Slope of Improvement’) ‘Dual-Discrepancy’: RTI Model of Learning Disability (Fuchs 2003) www. interventioncentral. org 6

Response to Intervention NYSED RTI Guidance Memo: April 2008 www. interventioncentral. org

Response to Intervention www. interventioncentral. org 8

Response to Intervention “The Regents policy framework for Rt. I: Defines Rt. I to minimally include: Appropriate instruction delivered to all students in the general education class by qualified personnel. Appropriate instruction in reading means scientific research-based reading programs that include explicit and systematic instruction in phonemic awareness, phonics, vocabulary development, reading fluency (including oral reading skills) and reading comprehension strategies. Screenings applied to all students in the class to identify those students who are not making www. interventioncentral. org academic progress at expected rates. ” 9

“Instruction matched totostudent need with Response Intervention increasingly intensive levels of targeted intervention and instruction for students who do not make satisfactory progress in their levels of performance and/or in their rate of learning to meet age or grade level standards. Repeated assessments of student achievement which should include curriculum based measures to determine if interventions are resulting in student progress toward age or grade level standards. The application of information about the student’s response to intervention to make educational decisions about changes in goals, instruction and/or services and the decision to make a referral for www. interventioncentral. org 10

Response to Intervention “Written notification to the parents when the student requires an intervention beyond that provided to all students in the general education classroom that provides information about the: -amount and nature of student performance data that will be collected and the general education services that will be provided; -strategies for increasing the student’s rate of learning; and -parents’ right to request an evaluation for special education programs and/or services. ” www. interventioncentral. org 11

“The Regents policy framework for Rt. I: Response to Intervention Defines Rt. I to minimally include: Requires each school district to establish a plan and policies for implementing school-wide approaches and prereferral interventions in order to remediate a student’s performance prior to referral for special education, which may include the Rt. I process as part of a district’s school-wide approach. The school district must select and define the specific structure and components of its Rt. I program, including, but not limited to the: -criteria for determining the levels of intervention to be provided to students, -types of interventions, amount and nature of student performance data to be collected, and -manner and frequency for progress monitoring. ” www. interventioncentral. org 12

Response to Intervention Effective Formative Evaluation: The Underlying Logic… 1. What is the relevant academic or behavioral outcome measure to be tracked? 2. Is the focus the core curriculum or system, subgroups of underperforming learners, or individual struggling students? 3. What method(s) should be used to measure the target academic skill or behavior? 4. What goal(s) are set for improvement? 5. How does the school check up on progress www. interventioncentral. org toward the goal(s)?

Response to Intervention Summative data is static information that provides a fixed ‘snapshot’ of the student’s academic performance or behaviors at a particular point in time. School records are one source of data that is often summative in nature— frequently referred to as archival data. Attendance data and office disciplinary referrals are two examples of archival records, data that is routinely collected on all students. In contrast to archival data, background information is collected specifically on the target student. Examples of background information are teacher interviews and student interest surveys, each of which can shed light on a student’s academic or behavioral strengths and weaknesses. Like archival data, background information is usually summative, providing a measurement of the www. interventioncentral. org 14

Response to Intervention Formative assessment measures are those that can be administered or collected frequently—for example, on a weekly or even daily basis. These measures provide a flow of regularly updated information (progress monitoring) about the student’s progress in the identified area(s) of academic or behavioral concern. Formative data provide a ‘moving picture’ of the student; the data unfold through time to tell the story of that student’s response to various classroom instructional and behavior management strategies. Examples of measures that provide formative data are Curriculum-Based Measurement probes in oral reading fluency and Daily Behavior Report Cards. www. interventioncentral. org 15

Response to Intervention Formal Assessment Defined “Formative assessment [in academics] refers to the gathering and use of information about students’ ongoing learning by both teachers and students to modify teaching and learning activities. …. Today…there are compelling research results indicating that the practice of formative assessment may be the most significant single factor in raising the academic achievement of all students—and especially that of lower-achieving students. ” p. 7 Source: Harlen, W. (2003). Enhancing inquiry through formative assessment. San Francisco, CA: Exploratorium. Retrieved on September 17, 2008, from http: //www. exploratorium. edu/ifi/resources/harlen_monograph. pdf www. interventioncentral. org 16

Response to Intervention Academic or Behavioral Targets Are Stated as ‘Replacement Behaviors’ “A problem solution is defined as one or more changes to the instruction, curriculum, or environment that function(s) to reduce or eliminate a problem. ” p. 159 Source: Christ, T. (2008). Best practices in problem analysis. In A. Thomas & J. Grimes (Eds. ), Best practices in school psychology V (pp. 159 -176). www. interventioncentral. org 17

Response to Intervention School Instructional Time: The Irreplaceable Resource “In the average school system, there are 330 minutes in the instructional day, 1, 650 minutes in the instructional week, and 56, 700 minutes in the instructional year. Except in unusual circumstances, these are the only minutes we have to provide effective services for students. The number of years we have to apply these minutes is fixed. Therefore, each minute counts and schools cannot afford to support inefficient models of service Source: Batsche, G. M. , Castillo, J. M. , Dixon, D. N. , & Forde, S. (2008). Best practices in problem analysis. In A. Thomas & J. Grimes (Eds. ), Best practices in school psychology V (pp. 177 -193). delivery. ” www. interventioncentral. org 18

Response to Intervention Formative Assessment: Essential Questions… 1. What is the relevant academic or behavioral outcome measure to be tracked? Problems identified formative assessment should be: 1. Important to school stakeholders. 2. Measureable & observable. 3. Stated positively as ‘replacement behaviors’ or goal statements rather than as general negative concerns (Bastche et al. , 2008). 4. Based on a minimum of inference (T. Christ, 2008). Source: Batsche, G. M. , Castillo, J. M. , Dixon, D. N. , & Forde, S. (2008). Best practices in problem analysis. In A. Thomas & J. Grimes (Eds. ), Best practices in school psychology V (pp. 177 -193). Christ, T. (2008). Best practices in problem analysis. In A. Thomas & J. Grimes (Eds. ), Best practices in school psychology V (pp. 159 -176). www. interventioncentral. org 19

Response to Intervention Academic or Behavioral Targets Are Stated as ‘Replacement Behaviors’ “The implementation of successful interventions begins with accurate problem identification. Traditionally, the student problem was stated as a broad, general concern (e. g. , impulsive, aggressive, reading below grade level) that a teacher identified. In a competency-based approach, however, the problem identification is stated in terms of the desired replacement behaviors that will increase the student’s probability of successful adaptation to the task demands of Source: Batsche, G. M. , Castillo, J. setting. ” M. , Dixon, D. N. , p. & Forde, S. (2008). Best practices in problem the academic 178 analysis. In A. Thomas & J. Grimes (Eds. ), Best practices in school psychology V (pp. 177 -193). www. interventioncentral. org 20

Response to Intervention Inference: Moving Beyond the Margins of the ‘Known’ “An inference is a tentative conclusion without direct or conclusive support from available data. All hypotheses are, by definition, inferences. It is critical that problem analysts make distinctions between what is known and what is inferred or hypothesized…. Low-level inferences should be exhausted prior to the use of high-level inferences. ” p. 161 Source: Christ, T. (2008). Best practices in problem analysis. In A. Thomas & J. Grimes (Eds. ), Best practices in school psychology V (pp. 159 -176). www. interventioncentral. org 21

Response to Intervention Examples of High vs. Low Inference Hypotheses The results of grade-wide benchmarking in reading show that a target 2 nd-grade student can read aloud at approximately half the rate of the median child in the grade. High-Inference Hypothesis. The student Unknow has an auditory processing issue that n prevents success in reading. The student requires a multisensory approach to Known reading instruction to address reading deficits. Unknow Low-Inference Hypothesis. The n student needs to build reading fluency Known skills to become more proficient in decoding. www. interventioncentral. org 22

Response to Intervention Adopting a Low-Inference Model of Reading Skills 5 Big Ideas in Beginning Reading 1. Phonemic Awareness 2. Alphabetic Principle 3. Fluency with Text 4. Vocabulary 5. Comprehension Source: Big ideas in beginning reading. University of Oregon. Retrieved September 23, 2007, from http: //reading. uoregon. edu/index. php www. interventioncentral. org 23

Response to Intervention Formative Assessment: Essential Questions… 2. Is the focus the core curriculum or system, subgroups of underperforming learners, or individual struggling students? Apply the ‘ 80 -15 -5 ‘Rule (T. Christ, 2008) : – If less than 80% of students are successfully meeting academic or behavioral goals, the formative assessment focus is on the core curriculum and general student population. – If no more than 15% of students are not successful in meeting academic or behavioral goals, the formative assessment focus is on small-group ‘treatments’ or interventions. – If no more than 5% of students are not successful in meeting Source: Christ, T. (2008). or Bestbehavioral practices in problem analysis. In A. Thomasassessment & J. Grimes (Eds. ), Best is academic goals, the formative focus practices in school psychology V (pp. 159 -176). on the individual student. www. interventioncentral. org 24

Response to Intervention Using Local Norms in Coordination with Benchmark Data www. interventioncentral. org

Response to Intervention Baylor Elementary School : Grade Norms: Correctly Read Words Per Min : Sample Size: 23 Students Group Norms: Correctly Read Words Per Min: Book 4 -1: Raw Data 31 34 34 39 41 43 52 55 59 61 68 71 74 75 85 89 102 108 112 115 118 131 LOCAL NORMS EXAMPLE: Twenty-three 4 th-grade students were administered oral reading fluency Curriculum-Based Measurement passages at the 4 th-grade level in their school. ØIn their current number form, these data are not easy to interpret. ØSo the school converts them into a visual display—a boxplot —to show the distribution of scores and to convert the scores to percentile form. ØWhen Billy, a struggling reader, is screened in CBM reading fluency, he shows a SIGNIFICANT skill gap when compared to his grade peers. www. interventioncentral. org 26

Response to Intervention Baylor Elementary School : Grade Norms: Correctly Read Words Per Min : Sample Size: 23 Students January Benchmarking Group Norms: Correctly Read Words Per Min: Book 4 -1: Raw Data 31 34 34 39 41 43 52 55 59 61 68 71 74 75 85 89 102 108 112 115 118 131 National Reading Norms: 112 CRW Per Min Group Norms: Converted to Box-Plot Median (2 nd Quartile)=71 1 st Quartile=43 3 rd Quartile=108 Billy=19 Hi Value=131 Low Value=31 0 20 40 Source: Tindal, G. , Hansbrouck, J. , & Jones, C. (2005). Oral reading fluency: 90 years of measurement [Technical report #33]. Eugene, OR: University of Oregon. 60 80 100 120 140 160 Correctly Read Words-Book 4 -1 www. interventioncentral. org 27

Response to Intervention Formative Assessment: Essential Questions… 3. What method(s) should be used to measure the target academic skill or behavior? Formative assessment methods should be as direct a measure as possible of the problem or issue being evaluated. These assessment methods can: – Consist of General Outcome Measures or Specific Sub. Skill Mastery Measures – Include existing (‘extant’) data from the school system Curriculum-Based Measurement (CBM) is widely used to track basic student academic skills. Daily Behavior Report Source: Burns, M. (DBRCs) K. , & Gibbons, K. A. (2008). Implementing response-to-intervention in elementary and Cards are increasingly used as one source of secondary schools: Procedures to assure scientific-based practices. New York: Routledge. formative behavioral data. www. interventioncentral. org 28

Response to Intervention Curriculum-Based Measurement: Assessing Basic Academic Skills www. interventioncentral. org

Response to Intervention Curriculum-Based Evaluation: Definition • • “Whereas standardized commercial achievement tests measure broad curriculum areas and/or skills, CBE measures specific skills that are presently being taught in the classroom, usually in basic skills. Several approaches to CBE have been developed. Four common characteristics exist across these models: The measurement procedures assess students directly using the materials in which they are being instructed. This involves sampling items from the curriculum. Administration of each measure is generally brief in duration (typically 1 -5 mins. ) The design is structured such that frequent and repeated measurement is possible and measures are sensitive to change. Data are usually displayed graphically to allow monitoring of student performance. ” SOURCE: CAST Website: http: //www. cast. org/publications/ncac_curriculumbe. html www. interventioncentral. org 30

Response to Intervention RTI Literacy: Assessment & Progress-Monitoring (Cont. ) To measure student ‘response to instruction/intervention’ effectively, the RTI Literacy model measures students’ reading performance and progress on schedules matched to each student’s risk profile and intervention Tier membership. • Benchmarking/Universal Screening. All children in a grade level are assessed at least 3 times per year on a common collection of literacy assessments. • Strategic Monitoring. Students placed in Tier 2 (supplemental) reading groups are assessed 1 -2 times per month to gauge their progress with this intervention. • Intensive Monitoring. Students who participate in an intensive, individualized Tier 3 reading intervention are assessed at least once. Implementing per week. Source: Burns, M. K. , & Gibbons, K. A. (2008). response-to-intervention in elementary and secondary schools: Procedures to assure scientific-based practices. New York: Routledge. www. interventioncentral. org 31

Response to Intervention Curriculum-Based Measurement: Advantages as a Set of Tools to Monitor RTI/Academic Cases • Aligns with curriculum-goals and materials • Is reliable and valid (has ‘technical adequacy’) • Is criterion-referenced: sets specific performance levels for specific tasks • Uses standard procedures to prepare materials, administer, and score • Samples student performance to give objective, observable ‘low-inference’ information about student performance • Has decision rules to help educators to interpret student data and make appropriate instructional decisions • Is efficient to implement in schools (e. g. , training can be done quickly; the measures are brief and feasible for classrooms, etc. ) Source: M. K. , Hosp, data J. L. , & that Howell, can K. W. (2007). The ABCs of CBM. Newvisual York: Guilford. • Hosp, Provides be converted into displays www. interventioncentral. org 32

Response to Intervention Formative Assessment: Essential Questions… 4. What goal(s) are set for improvement? Goals are defined at the system, group, or individual student level. Goal statements: – – – Are worded in measureable, observable terms, Include a timeline for achieving those goals. Are tied to the formative assessment methods used to monitor progress toward the goal(s). www. interventioncentral. org 33

Response to Intervention Interpreting Data: The Power of Visual Display www. interventioncentral. org

Response to Intervention Sample Peer Tutoring Chart www. interventioncentral. org 35

Response to Intervention Sample Peer Tutoring Chart www. interventioncentral. org 36

Response to Intervention Single-Subject (Applied) Research Designs “Single-case designs evolved because of the need to understand patterns of individual behavior in response to independent variables, and more practically, to examine intervention effectiveness. Design use can be flexible, described as a process of responseguided experimentation…, providing a mechanism for documenting attempts to live up to legal mandates for students who are not responding to routine instructional methods. ” p. 71 Source: Barnett, D. W. , Daly, E. J. , Jones, K. M. , & Lentz, F. E. (2004). Response to intervention: Empirically based special service decisions from single-case designs of increasing and decreasing intensity. Journal of Special Education, 38, 66 -79. www. interventioncentral. org 37

Response to Intervention Single-Subject (Applied) Research Designs: Steps “The basic methods [of single-case designs] are • selecting socially important variables as dependent measures or target behaviors • taking repeated measures until stable patterns emerge so that participants may serve as their own controls (i. e. , baseline) • implementing a well-described intervention or discrete intervention trials • continuing measurement of both the dependent and independent variables within an acceptable pattern of intervention application and/or withdrawal to detect changes in behavior and make efficacy attributions • graphically analyzing the results to enable ongoing comparisons of the student’s performance under baseline Source: Barnett, D. W. , Daly, E. J. , Jones, K. M. , & Lentz, and intervention conditions, and. F. E. (2004). Response to intervention: Empirically based special service decisions from single-case designs of increasing and decreasing intensity. Journal of Specialthe Education, 38, 66 -79. • replicating results to reach the ultimate goal of the www. interventioncentral. org 38

Response to Intervention Formative Assessment: Donald: Grade 3 www. interventioncentral. org 39

Response to Intervention Formative Assessment: Donald: Grade 3 www. interventioncentral. org 40

Response to Intervention Formative Assessment: Essential Questions… 5. How does the school check up on progress toward the goal(s)? The school periodically checks the formative assessment data to determine whether the goal is being attained. Examples of this progress evaluation process include the following: – System-Wide: A school-wide team meets on a monthly basis to review the frequency and type of office disciplinary referrals to judge whether those referrals have dropped below the acceptable threshold for student behavior. – Group Level: Teachers at a grade level assembles every six weeks to review CBM data on students receiving small-group Sources: Burns, M. K. , & Gibbons, K. A. (2008). Implementing response-to-intervention in elementary supplemental instruction to determine whether students are ready and secondary schools: Procedures to assure scientific-based practices. New York: Routledge. to(1989). exit (Burns & Gibbons, 2008). Shinn, M. R. Curriculum-based measurement: Assessing special children. New York: Guilford. www. interventioncentral. org 41

Response to Intervention Team Activity: Formative Assessment and Your Schools At your tables, discuss: • What kinds of formative measures in literacy that your schools tend to collect most often. • How ‘ready’ your schools are to collect, interpret, and act on formative assessment data. . www. interventioncentral. org 42

Response to Intervention Team Activity: CBM Literacy Assessments: First Steps? At your tables: • Discuss what process your school might follow in selecting or updating it’s academic screening tools for RTI. • Think about the ‘first steps’ that your school should adopt in the fall to move forward with selecting or updating RTI academic screening tools. www. interventioncentral. org 43

Response to Intervention END www. interventioncentral. org 44