RTI Implementation Assessment Components Rt I STUDENT SUCCESS

RTI Implementation: Assessment Components Rt. I STUDENT SUCCESS PST Dr. Denise P. Gibbs, Director Alabama Scottish Rite Foundation Learning Centers gibbsdenise@aol. com Gibbs 2011 1

This presentation is provided at no cost to Alabama schools by the Alabama Scottish Rite Foundation. The philanthropy of the Alabama Scottish Rite Foundation began in the 1950’s in Alabama and continues today. The mission of the Alabama Scottish Rite Foundation is to provide help to Alabama Schools as they work with students who struggle in reading -particularly those students with dyslexia. Gibbs 2011 2

Bell ringer activity Anticipation Guide then…Turn and Talk As soon as you are seated, please complete the Anticipation Guide included in your handout. Then…turn to your neighbor and talk about your initial answers Gibbs 2011 3

Gains From High Impact Instructional Strategies: Research Findings (Marzano, 2001) Type of Strategy Effect %ile size gain # of 1. Identifying similarities and differences 1. 61 45 31 2. Summarizing and note taking 1. 00 34 179 3. Reinforcing effort and providing recognition . 80 29 21 4. Homework and practice . 77 28 134 5. Nonlinguistic representations . 75 27 246 Gibbs 2011 studies 4

Gains From High Impact Instructional Strategies: Research Findings (Marzano, 2001) Type of Strategy Effect size %ile gain # of studies 6. Cooperative learning . 73 27 122 7. Setting objectives and providing feedback . 61 23 408 8. Generating and testing hypotheses . 61 23 63 9. Questions, cues, advance organizers . 59 22 1251 Gibbs 2011 5

Today’s High Impact Strategies n n n 9. Questions, cues, and advance organizers – Anticipation Guide, session outcomes, RTI Vocabulary, Double Bubble 6. Cooperative learning – Turn and Talks 2. Summarizing and note taking – Power Point slides, your notes, 3 -2 -1, 1. Similarities and differences – CBM vs CAT Double Bubble 5. Nonlinguistic representations - VVWAs 4. Homework and practice – utilize at least one of these strategies in your work Gibbs 2011 6

Session outcomes…. n Today, you will: ¨ Practice high impact strategies ¨Identify uses of high stakes tests such as the ARMT or AHSGE in the RTI screening process. ¨Analyze essential criteria to guide selection or development of RTI screening and progress monitoring tools. ¨Use the National Center for Response to Intervention's (NCRTI) tools chart to compare some commercially available screening and progress monitoring tools. Gibbs 2011 7

RTI Essential Vocabulary: Assessment n n n Screening process vs tool Progress monitoring process vs tool Classification accuracy ¨ False n n positives? False negatives? Reliability and predictive validity of the performance level and slope of improvement Reliability and predictive validity of the slope of improvement Sensitivity to student improvement Efficiency!!!!!! Gibbs 2011 8

Screening Process and Examples of Some Tools Gibbs 2011 9

Designing your screening process n Commercially available screening tools ¨ Curriculum-Based Measures (CBM) ¨ Computer Adapted Testing (CAT) ¨ Standards-Connected Assessments n High stakes tests ¨ ARMT ¨ Alabama High School Graduation Exam ¨ End of course tests n Other variables ¨ Grades ¨ Behavior ¨ Attendance Gibbs 2011 10

Using High Stakes Tests in the Screening Process n A starting point in screening older students could be to use the results of the ARMT or the AHSGE. ¨ Students who score a I or II on the ARMT in reading or in math could be considered as at risk and could proceed to the next step in the district’s screening process ¨ AHSGE results could be analyzed to determine the relative severity of the student’s deficiencies n Determine if the student would benefit from remediation classes or is in need of a more basic skill -level intervention class Gibbs 2011 11

Double Bubble Time! As an advanced organizer, compare and contrast CBM and CAT Gibbs 2011 12

Three types of screening measures n n n Curriculum Based Measures ¨ Probes which look at skills which predict that a student will not be successful in reading or math. Computer Adapted Testing ¨ Answers determine subsequent questions. Can learn more about skill levels with fewer test items. Standards Connected Assessments ¨ Usually a fixed set of questions that mirror standards-based state assessments and provide practice and performance prediction. Gibbs 2011 13

Independent Review of Screening Tools National Center for Response to Intervention has charts summarizing their review of various screening tools. n Some of the tools reviewed as of Spring 2011 are: n ¨ AIMSweb - CBM ¨ DIBELS - CBM ¨ STEEP - CBM ¨ STAR - CAT ¨ Discovery Education (Think Link) – Standardsconnected Gibbs 2011 14

Curriculum-Based Measures (CBM) Was initially synonymous with RTI. n If concerned about reading, have student complete various reading screening probes n If concerned about math, have student complete math computation and math reasoning/problem solving screening probes. n AIMSweb, DIBELS, and STEEP are all based on CBM n Gibbs 2011 15

A look at some reading CBM probes n Early Literacy Measures ¨ Letter Naming Fluency ¨ Letter Sound Fluency ¨ Phonemic Segmentation Fluency ¨ Nonsense Word Fluency n R-CBM (8 th grade example) ¨ Grades n Mazes 1 -12 (7 th grade example) ¨ Grades 2 -12 Gibbs 2011 16

A look at some math CBM probes n Early Numeracy Measures ¨ Quantity discrimination ¨ Missing Numbers ¨ Oral Counting ¨ Number Identification n Computation (M-CBM) (1 st – 6 th grade example) ¨ Grades n 1 -12 Concepts and Applications (M-CAP) (7 th grade example) ¨ Grades n 2 -8 Focal points (STEEP) Gibbs 2011 17

Computer Adapted Testing (CAT) The best computer adapted testing is based on Item Response Theory (IRT) IRT is a statistical framework in which examinees can be described by a set of one or more ability scores that are predictive, through mathematical models, linking actual performance on test items, item statistics, and examinee abilities. Test items are determined by responses to questions Score is reported as a scaled score. STAR is an example of CAT based on IRT Gibbs 2011 18

Computer Adapted Testing (CAT) STAR Early Literacy Pre. K-Grade 3 41 skills in 7 domains 10 minutes STAR Reading Grades 1 -12 10 minutes STAR Math Grades 1 -12 10 -15 minutes Gibbs 2011 19

Standards Connected Assessments Test includes a fixed number of gradespecific questions reflecting common core or state standards. Can serve as a screening tool for students who might not be likely to perform satisfactorily on state assessments. Can serve as a practice test for state assessments. Generally, will not be able to show growth over time and monitor progress. Gibbs 2011 20

Reports from Screening Tools n Some of the most helpful screening tools provide reports which reflect ¨ Student’s percentile scores ¨ Student’s expected performance on high stakes tests (ARMT!) ¨ Longitudinal data for a group of students ¨ Level of proficiency on key common core standards ¨ Instructional grouping/planning suggestions Gibbs 2011 21

Turn and Talk: Where are we in terms of screening process? In the bags, quick fixes, long -term growth opportunities! Gibbs 2011 22

Progress Monitoring Process and Examples of Some Tools Gibbs 2011 23

What progress to monitor? n Growth of skills in areas recognized as deficient ¨ Foundational academic skills in reading or math Growth of appropriate social and behavior skills n Growth of content knowledge and information n Gibbs 2011 24

Progress Monitoring Process Use of a formal commercially available progress monitoring tool or n Use of a formal teacher/school developed progress monitoring tool n Should also consider additional information which can reflect progress n ¨ Use of work samples reflecting baseline and current work ¨ Observation of classroom behavior and participation Gibbs 2011 25

NCRTI Progress Monitoring Tools Chart Provides administration information n Compares tools according to key reliability, validity, slope, and improvement criteria. n Many of the screening tools have also been favorably reviewed as progress monitoring tools. n But some screening tools were NOT found to have favorably reviewed progress monitoring probes. n Gibbs 2011 26

Key progress monitoring criteria n Reliability of the performance level score ¨ the extent to which the score (or average/median of 2 -3 scores) is accurate and consistent. n Validity of the performance level score ¨ the extent to which the score (or average/median of 2 -3 scores) represents the underlying construct Gibbs 2011 27

Key progress monitoring criteria n Reliability of the slope of improvement ¨ an indicator of how well individual differences in growth trajectories can be detected using a particular measure n Predictive validity of the slope of improvement ¨ the extent to which the slope of improvement corresponds to end-level performance on highly valued outcomes. n Sensitivity to student improvement ¨ the extent to which a measure reveals improvement over time, when improvement actually occurs. Gibbs 2011 28

Progress Monitoring Tools Need to be efficient. n Need to be able to provide a reliable baseline. n Need to be able to reflect growth over time. n Need multiple, equivalent probes. n Results need to be easily graphed. n Gibbs 2011 29

Progress Monitoring Decisions Skills to be progress monitored? n Do you have the probes you need? n Frequency of progress monitoring n ¨ Weekly? Bi-weekly? Monthly? Who will conduct progress monitoring n Who will score? Who will enter data in computer? Print reports? n Gibbs 2011 30

Some reading skills to be targeted… n Word-level needs ¨ General benchmark should be 95% accuracy for grade-level texts n To reach desired accuracy, may need to target both regular (decodable) words and irregular (learned) words ¨ Rate needs to be that designated for student’s grade or 90 -110 words correct per minute (whichever is less) ¨ Prosody/inflection patterns!!!! Gibbs 2011 31

Some reading skills to be targeted… n Comprehension needs Vocabulary ¨ Active reading strategies ¨ n n Before, During, After ¨ Magnificent Seven! Metacognitive strategies ¨ Fix-up strategies Gibbs 2011 32

Math skills to be targeted… n Computation automaticity I am learning the facts ¨ I can think of the answer ¨ I can not avoid seeing the answer when I see the problem. ¨ n Grade-specific problem solving and applications ¨ n Concrete-representational-abstract continuum Subject specific skills Gibbs 2011 33

Selection of Progress Monitoring Tools n Tools should be able to be administered weekly, should be efficient and should be sensitive to growth ¨ If improved reading comprehension is the goal, then you may elect to use mazes or computer adapted testing ¨ If improved reading accuracy is the goal, then you may elect to use oral reading fluency passages and graph percentage of accuracy ¨ If improved math problem solving skills is the goal, then you may elect to use timed problem solving probes or computer adapted testing ¨ If improved math calculation skills is the goal, then you may elect to use timed calculation probes ¨ If improved behavior is the goal, then you may elect to use behavior report card points earned. Gibbs 2011 34

Verbal and Visual Word Association Sensitivity Efficiency Gibbs 2011 35

Developing Progress Monitoring Probes n Interventioncentral. com equips you to generate ¨ Maze passages – from your own texts ¨ R-CBM passages – from your own texts ¨ Wordlist fluency probes ¨ Math fact fluency probes ¨ Behavior report cards! n Establish baseline and set goal Gibbs 2011 36

Developing Progress Monitoring Probes n To monitor progress in learning content, you can develop a set of vocabulary matching probes (Espin, et al. , 2005). ¨ Comprehensive list of terms and definitions for the course/subject ¨ Randomly generated probes with 25 or more terms given as frequently as desired. ¨ Graph number or percentage correct Gibbs 2011 37

About norms for locally developed PM measures You could initially use norms included in ABCs of CBM and other resources to give you a “ballpark idea” of student’s performance level n But remember, since you are measuring the student’s progress from baseline, you can determine growth without much attention to norms! n Goal setting will require some creativity! n Gibbs 2011 38

Goals setting with locally developed PM measures n n n You could determine the performance levels on your probes of your most proficient students and use those levels in developing goals for students in interventions. You could use “norms” included in ABCs of CBM and other sources to set goals Over time, you could determine scores which correspond to successful performance on ARMT or AHSGE and set those as goals. Gibbs 2011 39

Graphing progress monitoring data Commercially available progress monitoring tools are equipped with graphing features. n Results of teacher or school developed progress monitoring probes can be graphed with a number of graphing tools including the Data Management Tool available on the ALSDE website and at this Dropbox URL http: //dl. dropbox. com/u/24788238/ Graphingtool. Gibbs. xls n Gibbs 2011 40

Some valuable resources Gibbs 2011 41

The ABCs of CBM: A practical guide to curriculum-based measurement (Hosp, et al. , 2007) Instructions regarding creating progress monitoring probes n R-CBM, Mazes, Math Computation, Math Problem Solving, Written Expression, Spelling, Early Literacy, Early Numeracy n Expected levels of performance for grades 1 -8 n Gibbs 2011 42

Assessing Reading Multiple Measures for All Educators Working to Improve Reading Achievement (2 nd ed, 2008) n n n n Phonics Screener Phoneme Deletion Test Phonological Segmentation Test Phoneme Segmentation Test Graded High Frequency Word Survey San Diego Quick Assessment of Reading Ability Maze passages Oral reading fluency passages Gibbs 2011 43

3 -Minute Reading Assessments: Word Recognition, Fluency, and Comprehension (Rasinski and Padak, 2005) n Includes a 16 point multidimensional fluency scale!! ¨ Expression and volume ¨ Phrasing and intonation ¨ Smoothness ¨ Pace Gibbs 2011 44

Mathematics RTI: A Problem-Solving Approach to Creating An Effective Model (Allsopp, et al. , 2010) Includes specific recommendations for analyzing mathematics curriculum as it relates to struggling students. n Includes discussion of responsive mathematics teaching practices. n Gibbs 2011 45

RTI in Mathematics (Riccomini and Witzell, 2010) n Excellent tools to facilitate assessment, instruction, and intervention for ¨ Number sense ¨ Whole numbers ¨ Fractions and decimals ¨ Problem solving ¨ Mathematical vocabulary! Gibbs 2011 46

Anticipation Guide Revisited Gibbs 2011 47

THANK YOU! RTI for Early Readers: Implementing Common Core Standards in Your K-5 RTI Model (LRP, 2011) RTI for Middle and High Schools: Strategies and Structures for Literacy Success (LRP, 2008) Leading the Dyslexia Challenge: An Action Plan for Schoolwide Identification and Intervention (LRP, 2004) gibbsdenise@aol. com Gibbs 2011 48