Graphing Calculating and Interpreting Rate of Improvement Caitlin
- Slides: 115
Graphing, Calculating, and Interpreting Rate of Improvement Caitlin S. Flinn, Ed. S. , N. C. S. P. Andrew E. Mc. Crea, M. S. , N. C. S. P. NASP Convention March 3, 2010
Objectives There needs to be a standardized procedure for calculating Ro. I l We’re proposing a method using Simple Linear Regression l
Overview l l Importance of Ro. I Research A Need for Consistency Calculating Ro. I l l Individual Student Graphs Programming Excel l Decision Making Grounding the Data Interpreting Growth l l Individual Student Groups Considerations Resources
Importance of Graphs l Vogel, Dickson, & Lehman, 1990 l Speeches that included visuals, especially in color, improved: l Immediate recall by 8. 5% l Delayed recall (3 days) by 10. 1%
Importance of Graphs “Seeing is believing. ” l Useful for communicating large amounts of information quickly l “A picture is worth a thousand words. ” l Transcends language barriers (Karwowski, 2006) l Responsibility for accurate graphical representations of data l
Skills Typically Graphed l Reading l l l Oral Reading Fluency (ORF) Word Use Fluency (WUF) Reading Comprehension l l l l Math Computation Math Facts Early Numeracy Early Literacy Skills l l l Math MAZE Retell Fluency l l l Initial Sound Fluency (ISF) Letter Naming Fluency (LNF) Letter Sound Fluency (LSF) Phoneme Segmentation Fluency (PSF) Nonsense Word Fluency (NWF) Spelling Written Expression Behavior l l Oral Counting Missing Number Identification Quantity Discrimination
Importance of Ro. I Multi-tiered model l Progress monitoring l Data for decision-making l Goal setting (Shapiro, 2008) l
Importance of Ro. I Visual inspection of slope l Multiple interpretations l Instructional services l Need for explicit guidelines l
Ro. I Research l Deno, 1985 l Curriculum-based l General measurement outcome measures l Short l Standardized l Repeatable l Sensitive to change
Ro. I Research l Fuchs & Fuchs, 1998 l Hallmark components of Response to Intervention l Ongoing formative assessment l Identifying non-responding students l Treatment fidelity of instruction l Dual discrepancy model l One standard deviation from typically performing peers in level and rate
Ro. I Research l Ardoin & Christ, 2008 l Slope for benchmarks (3 x per year) l More growth from fall to winter than winter to spring l Might be helpful to use Ro. I for fall to winter l And a separate Ro. I for winter to spring
Ro. I Research l Fuchs, Walz, & Germann, 1993 l Typical weekly growth rates l Needed growth l 1. 5 l to 2. 0 times typical slope to close gap Example l Bob is below benchmark on ORF l Typical slope is 1 wcpm per week growth l Bob would need slope of 1. 5 to 2 to close gap in a reasonable amount of time
Ro. I Research l Deno, Fuchs, Marston, & Shin, 2001 l Slope of frequently non-responsive children approximated slope of children already identified as having a specific learning disability
Ro. I Research l Algebraic term: Slope of a line l Vertical change over the horizontal change l Rise over run l m = (y 2 - y 1) / (x 2 - x 1) l Describes the steepness of a line (Gall & Gall, 2007)
Ro. I Research l Finding a student’s Ro. I = finding the slope of a line l Using l two data points on that line Finding the line itself l Linear regression l Ordinary Least Squares
Ro. I Research l Gall & Gall, 2007 l 10 data points are a minimum requirement for a reliable trendline l How does that affect the frequency of administering progress monitoring probes?
Ro. I Research Using Ro. I for instructional decisions is not a perfect process l Research is currently looking to address sources of error: l l Christ, 2006 – standard error of measurement for slope l Ardoin & Christ, 2009 – passage difficulty and variability l Jenkin, Graff, & Miglioretti, 2009 – frequency of progress monitoring
Ro. I Research l Questions yet to be empirically answered l What parameters of Ro. I indicate a lack of Rt. I? l How does standard error of measurement play into using Ro. I for instructional decision making? l How does Ro. I vary between standard protocol interventions? l How does this apply to non-English speaking populations?
How is Ro. I Calculated? Which way is best?
Multiple Methods for Calculating Growth “Eye ball” Approach l Last point minus First point Approach l Split Middle Approach l Linear Regression Approach l
1. 1 Words Per Week
Ro. I Consistency? Eye Ball ? ? ? Last minus First 0. 75 Split Middle* 0. 50 Linear Regression 1. 10
Ro. I Consistency? l l If we are not all using the same model to compute Ro. I, we continue to have the same problems as past models, where under one approach a student meets SLD criteria, but under a different approach, the student does not. Hypothetically, if the Ro. I cut-off was 0. 65 or 0. 95, different approaches would come to different conclusions on the same student.
Technical Adequacy l Without a consensus on how to compute Ro. I, we risk falling short of having technical adequacy within our model.
So, Which Ro. I Method is Best?
Literature shows that Linear Regression is Best Practice l l Student’s daily test scores…were entered into a computer program…The data analysis program generated slopes of improvement for each level using and Ordinary-Least Squares procedure (Hayes, 1973) and the line of best fit. This procedure has been demonstrated to represent CBM achievement data validly within individual treatment phases (Marston, 1988; Shinn, Good, & Stein, in press; Stein, 1987). Shinn, Gleason, & Tindal, 1989
Growth (Ro. I) Research using Linear Regression l l Christ, T. J. (2006). Short-term estimates of growth using curriculum based measurement of oral reading fluency: Estimating standard error of the slope to construct confidence intervals. School Psychology Review, 35, 128 -133. Deno, S. L. , Fuchs, L. S. , Marston, D. , & Shin, J. (2001). Using curriculum based measurement to establish growth standards for students with learning disabilities. School Psychology Review, 30, 507 -524. Good, R. H. (1990). Forecasting accuracy of slope estimates for reading curriculum based measurement: Empirical evidence. Behavioral Assessment, 12, 179 -193. Fuchs, L. S. , Fuchs, D. , Hamlett, C. L. , Walz, L. & Germann, G. (1993). Formative evaluation of academic progress: How much growth can we expect? School Psychology Review, 22, 27 -48.
Growth (Ro. I) Research using Linear Regression l l l Jenkins, J. R. , Graff, J. J. , & Miglioretti, D. L. (2009). Estimating reading growth using intermittent CBM progress monitoring. Exceptional Children, 75, 151 -163. Shinn, M. R. , Gleason, M. M. , & Tindal, G. (1989). Varying the difficulty of testing materials: Implications for curriculum-based measurement. The Journal of Special Education, 23, 223 -233. Shinn, M. R. , Good, R. H. , & Stein, S. (1989). Summarizing trend in student achievement: A comparison of methods. School Psychology Review, 18, 356 -370.
So, Why Are There So Many Other Ro. I Models? Ease of application l How many of us want to calculate OLS Linear Regression formulas (or even remember how)? l
An Easy and Applicable Solution
Get Out Your Laptops! Or Kindly Look Over Your Neighbor’s Shoulder! I love ROI
Open Microsoft Excel Microsoft Office 2003 for PCs l Microsoft Office 2007 for PCs l Microsoft Office for Macs l
Graphing Ro. I For Individual Students
Setting Up Your Spreadsheet In cell B 2, type School Week l In cell C 2, type Benchmark l In cell D 2, type WPM (or Student Scores) l
Labeling School Weeks In cell B 3, type 1 l Continue entering numbers through 36 in column B l Week 36 will be in cell B 38 l
Entering Benchmarks In cell C 3, type the fall benchmark 77 l In cell C 20, type the winter benchmark 92 l In cell C 38, type the spring benchmark 110 l
Entering Student Data Points Student data points are entered between cells D 3 and D 38. l Type the student’s score next to the corresponding week that it was administered. l
Entering Student Data Points Week 1 – 41 l Week 8 – 62 l Week 9 – 63 l Week 10 – 75 l Week 11 – 64 l Week 12 – 80 l Week 13 – 83 l Week 14 - 83 l
Entering Student Data Points Week 15 – 56 l Week 17 – 104 l Week 18 – 74 l Week 20 – 85 l Week 21 – 89 l Week 22 – 69 l Week 23 – 85 l
Entering Student Data Points Week 24 – 96 l Week 25 – 90 l Week 26 – 84 l Week 27 – 106 l Week 28 – 94 l Week 32 – 100 l
*CAUTION* If a student was not assessed during a certain week, leave that cell blank l Do not enter a score of Zero (0) it will be calculated into the trendline and interpreted as the student having read zero words correct per minute during that week. l
Creating a Graph Highlight the data in Columns C and D l Include cells C 2 and D 2 through cells C 38 and D 38 l Include any blank cells l
Creating a Graph l Excel 2003/Macs l l Click Insert Click Chart l Excel 2007 l l l Click Insert Find the icon for Line Click the arrow below Line
Creating a Graph l Excel 2003/Macs l A Chart Wizard window will appear l Excel 2007 l 6 graphics appear
Creating a Graph l Excel 2003/Macs l l Choose Line with markers l Excel 2007 l Choose Line with markers
Creating a Graph l Excel 2003/Macs l l Data Range tab Columns l Excel 2007 l Your graph appears
Creating a Graph l Excel 2003/Macs l l l Chart Title School Week (X Axis) WPM (Y Axis) l Excel 2007 l Change your labels by clicking on the graph
Creating a Graph l Excel 2003/Macs l Choose where you want your graph l Excel 2007 l Your graph was automatically put into your data spreadsheet
Creating a Graph l Excel 2003/Macs l Excel 2007
Adding a Trendline l Excel 2003/Macs l Right l Excel 2007 click on any of the student data points
Adding a Trendline l Excel 2003/Macs l Choose Linear l Excel 2007
Adding a Trendline l Excel 2003/Macs l Choose l Excel 2007 Custom and check box next to Display equation on chart
Adding a Trendline Clicking on the equation highlights a box around it l Clicking on the box allows you to move it to a place where you can see it better l
Adding a Trendline You can repeat the same procedure to have a trendline for the benchmark data points l Suggestion: label the trendline Expected ROI l Move this equation under the first l
Individual Student Graph
Individual Student Graph The equation indicates the slope, or rate of improvement. l The number, or coefficient, before "x" is the average improvement, which in this case is the average number of words per minute per week gained by the student. l
Individual Student Graph The rate of improvement, or trendline, is calculated using a linear regression, a simple equation of least squares. l To additional progress monitoring/benchmark scores once you’ve already created a graph, enter additional scores in Column D in the corresponding school week. l
Individual Student Graph Remember to leave cells blank for the weeks that no score was obtained. l The graph will incorporate that score into the set of data points and into the trendline. l
Individual Student Graph The slope can change depending on which week (where) you put the benchmark scores on your chart. l Enter benchmark scores based on when your school administers their benchmark assessments for the most accurate depiction of expected student progress. l
Options for the Graph Resizing l Coloring l Data Labels l
Programming Excel To Calculate Ro. I A Formula
Ro. I Formula l Type Ro. I in cell B 39 below the last week of school
Calculate Expected Slope Click on cell C 39 l Put your cursor at the top next to the fx l Type =SLOPE(C 3: C 38, B 3: B 38) l Hit Enter/Return l
Calculate Actual Slope Click on cell D 39 l Put your cursor at the top next to the fx l Type =SLOPE(D 3: D 38, B 3: B 38) l Hit Enter/Return l
ROI as a Decision Tool within a Problem-Solving Model
Steps 1. 2. 3. 4. Gather the data Ground the data Interpret the data Figure out how to fit Best Practice into Public Education
Step 1: Gather Data Universal Screening Progress Monitoring
Common Screenings in PA DIBELS l AIMSweb l MBSP l 4 Sight l PSSA l
Validated Progress Monitoring Tools DIBELS l AIMSweb l MBSP l www. studentprogress. org l
Step 2: Ground the Data To what will we compare our student growth data?
Multiple Ways to Look at Growth Needed Growth l Expected Growth & Percent of Expected Growth l Fuchs et. al. (1993) Table of Realistic and Ambitious Growth l Growth Toward Individual Goal* l *Best Practices in Setting Progress Monitoring Goals for Academic Skill Improvement (Shapiro, 2008)
Needed Growth Difference between student’s BOY (or MOY) score and benchmark score at MOY (or EOY). l Example: MOY ORF = 10, EOY benchmark is 40, 18 weeks of instruction (40 -10/18=1. 67). Student must gain 1. 67 wcpm per week to make EOY benchmark. l
Expected Growth Difference between two benchmarks. l Example: MOY benchmark is 20, EOY benchmark is 40, expected growth (4020)/18 weeks of instruction = 1. 11 wcpm per week. l
Looking at Percent of Expected Growth Tier III Greater than 150% Between 110% & 150% Possible LD Between 95% & 110% Likely LD Between 80% & 95% May Need More Likely LD Below 80% Needs More Likely LD Tigard-Tualatin School District (www. ttsd. k 12. or. us)
Oral Reading Fluency Adequate Response Table Realistic Growth Ambitious Growth 1 st 2. 0 3. 0 2 nd 1. 5 2. 0 3 rd 1. 0 1. 5 4 th 0. 9 1. 1 5 th 0. 5 0. 8 Fuchs, Hamlett, Walz, & Germann (1993)
Digit Fluency Adequate Response Table 1 st Realistic Growth 0. 3 Ambitious Growth 0. 5 2 nd 0. 3 0. 5 3 rd 0. 3 0. 5 4 th 0. 75 1. 2 5 th 0. 75 1. 2 Fuchs, Hamlett, Walz, & Germann (1993)
Making Decisions: Best Practice Research has yet to establish a blue print for ‘grounding’ student Ro. I data. l At this point, teams should consider multiple comparisons when planning and making decisions. l
Making Decisions: Lessons From the Field When tracking on grade level, consider an Ro. I that is 100% of expected growth as a minimum requirement, consider an Ro. I that is at or above the needed as optimal. l So, 100% of expected and on par with needed become the limits of the range within a student should be achieving. l
Step 3: Interpreting Growth
What do we do when we do not get the growth we want? When to make a change in instruction and intervention? l When to consider SLD? l
When to make a change in instruction and intervention? Enough data points (6 to 10)? l Less than 100% of expected growth. l Not on track to make benchmark (needed growth). l Not on track to reach individual goal. l
When to consider SLD? Continued inadequate response despite: l Fidelity with Tier I instruction and Tier II/III intervention. l Multiple attempts at intervention. l Individualized Problem-Solving approach.
Three Levels of Examples Whole Class l Small Group l Individual Student - Academic Data - Behavior Data l
Whole Class Example
3 rd Grade Math Whole Class Who’s responding? l Effective math instruction? l Who needs more? l N=19 l 4 > 100% growth l 15 < 100% growth l 9 w/ negative growth l
Small Group Example
Intervention Group Intervention working for how many? l Can we assume fidelity of intervention based on results? l Who needs more? l
Individual Kid Example
Individual Kid Making growth? l How much (65% of expected growth). l Atypical growth across the year (last 3 data points). l Continue? Make a change? Need more data? l
Ro. I and Behavior?
Step 4: Figure out how to fit Best Practice into Public Education
Things to Consider Who is At-Risk and needs progress monitoring? l Who will collect, score, enter the data? l Who will monitor student growth, when, and how often? l What changes should be made to instruction & intervention? l What about monitoring off of grade level? l
Who is At-Risk and needs progress monitoring? l Below level on universal screening Entering 4 th Grade Example DORF (110) Student A 115 ISIP TRWM (55) 58 4 Sight (1235) PSSA (1235) 1255 1232 Student B 85 48 1216 1126 Student C 72 35 1056 1048
Who will collect, score, and enter the data? Using MBSP for math, teachers can administer probes to whole class. l DORF probes must be administered oneon-one, and creativity pays off (train and use art, music, library, etc. specialists). l Schedule for progress monitoring math and reading every-other week. l
Week 1 Reading 1 st Reading X X X Math X X 4 th 5 th Math X 2 nd 3 rd Week 2 X X
Who will monitor student growth, when, and how often? l l l Best Practices in Data-Analysis Teaming (Kovaleski & Pedersen, 2008) Chambersburg Area School District Elementary Response to Intervention Manual (Mc. Crea et. al. , 2008) Derry Township School District Response to Intervention Model (http: //www. hershey. k 12. pa. us/56039310111408/lib/56039310111408/_files/Microsoft_Word__Response_to_Intervention_Overview_of_Hershey_Elementary_Model. pdf)
What changes should be made to instruction & intervention? Ensure treatment fidelity!!!! l Increase instructional time (active and engaged) l Decrease group size l Gather additional, diagnostic, information l Change the intervention l
When Instructional Level is Not the Same as Grade Level l Understand needed and expected Ro. I within broader context: l l Needed growth will only get student to next level by next benchmark (as opposed to on level). 100% of expected growth may not be an acceptable minimum (not enough growth b/c level is so low).
Grounding Ro. I When Monitoring Off of Grade Level: Two Options Best Practices in Setting Progress Monitoring Goals for Academic Skill Improvement (Shapiro, 2008). l Tigard-Tualatin SD Chart: 150% instead of 100% as minimum Ro. I requirement? ? ? l
Questions? & Comments!
Resources l www. interventioncentral. com l www. aimsweb. com l http: //dibels. uoregon. edu l www. nasponline. org
Resources www. fcrr. org Florida Center for Reading Research l http: //ies. ed. gov/ncee/wwc// What Works Clearinghouse l http: //www. rti 4 success. org National Center on Rt. I l
Flinn & Mc. Crea’s Ro. I Web Site l http: //sites. google. com/site/rateofimprove ment/ l Download powerpoints, handouts, Excel graphs, charts, articles, etc. l Caitlin Flinn l c. s. flinn@iup. edu l Andrew Mc. Crea l mccreand@chambersburg. k 12. pa. us
References Ardoin, S. P. , & Christ, T. J. (2009). Curriculumbased measurement of oral reading: Standard errors associated with progress monitoring outcomes from DIBELS, AIMSweb, and an experimental passage set. School Psychology Review, 38(2), 266 -283. Ardoin, S. P. & Christ, T. J. (2008). Evaluating curriculum-based measurement slope estimates using triannual universal screenings. School Psychology Review, 37(1), 109 -125.
References Christ, T. J. (2006). Short-term estimates of growth using curriculum-based measurement of oral reading fluency: Estimating standard error of the slope to construct confidence intervals. School Psychology Review, 35(1), 128 -133. Deno, S. L. (1985). Curriculum-based measurement: The emerging alternative. Exceptional Children, 52, 219 -232.
References Deno, S. L. , Fuchs, L. S. , Marston, D. , & Shin, J. (2001). Using curriculum-based measurement to establish growth standards for students with learning disabilities. School Psychology Review, 30, 507 -524. Flinn, C. S. (2008). Graphing rate of improvement for individual students. In. Sight, 28(3), 10 -12.
References Fuchs, L. S. , & Fuchs, D. (1998). Treatment validity: A unifying concept for reconceptualizing the identification of learning disabilities. Learning Disabilities Research and Practice, 13, 204 -219. Fuchs, L. S. , Fuchs, D. , Hamlett, C. L. , Walz, L. , & Germann, G. (1993). Formative evaluation of academic progress: How much growth can we expect? School Psychology Review, 22, 27 -48.
References Gall, M. D. , & Gall, J. P. (2007). Educational research: An introduction (8 th ed. ). New York: Pearson. Jenkins, J. R. , Graff, J. J. , & Miglioretti, D. L. (2009). Estimating reading growth using intermittent CBM progress monitoring. Exceptional Children, 75, 151 -163.
References Karwowski, W. (2006). International encyclopedia of ergonomics and human factors. Boca Raton, FL: Taylor & Francis Group, LLC. Shapiro, E. S. (2008). Best practices in setting progress monitoring goals for academic skill improvement. In A. Thomas and J. Grimes (Eds. ), Best practices in school psychology V (Vol. 2, pp. 141 -157). Bethesda, MD: National Association of School Psychologists.
References Vogel, D. R. , Dickson, G. W. , & Lehman, J. A. (1990). Persuasion and the role of visual presentation support. The UM/3 M study. In M. Antonoff (Ed. ), Presentations that persuade. Personal Computing, 14.
- 5-1 rate of change and slope
- I will always write back book summary
- Bronwyn and caitlin
- 3-3 interpreting the unit rate as slope
- Calculate unemployment rate
- How to calculate gtts/min
- Drug calculations formula
- Calculating rate of infusion
- How to calculate absorption cost per unit
- Calculating iv infusion rate
- Gtt/min formula
- Drip factor formula
- Overhead applied formula
- Machine hour rate meaning
- Calculating average rate of change
- Caitlin driscoll
- Caitlin meehan
- Caitlin d. davidson
- Caitlin coyle
- Ciasta mondrian caitlin freeman
- Caitlin bevers workshop
- Ataf ligament
- Dr elise meoli
- Esme caitlin
- Caitlin venus
- Caitlin grady
- Caitlin lindsay
- Caitlin crowley
- Caitlin benson
- Caitlin puccio
- Stanford children's administrative fellowship
- Aimsweb plus reports
- Nominal v. real interest rates
- Plant growth definition
- Forward rate example
- 1 year forward rate formula
- Unit rate proportions
- Misleading graph
- Cap rate interest rate relationship
- Transaksi forward
- Real exchange rate formula
- Interpreting graphs science
- Drawing and interpreting graphs
- Vertex form meaning
- Selecting organizing and interpreting information
- Analyzing and interpreting quantitative data
- Planning recording analyzing and interpreting
- Analyzing and interpreting data in research
- Analyzing and interpreting data in research
- Analyzing and interpreting data in research
- Analyzing and interpreting literature
- Analyzing and interpreting data in research
- Analyzing and interpreting data in research
- Analyzing and interpreting literature
- Interpreting distance time graphs
- Creating and interpreting distance time graph
- Representing and interpreting data
- Ngss analyzing and interpreting data
- Interpreting test scores and norms
- Find speed
- Speed time distance formula
- Speed distance time calculations
- Www.google.com
- Calculate kb
- Speed distance time questions worksheet
- Distance formula time speed
- Types of interpreting
- National network for interpreting
- Interpreting p value
- What is a significance test
- Interpreting circle graphs
- Interpreting relative risk
- Relative risk interpretation
- Interpreting relative risk
- Listening symbol
- Computer regression output
- What is a semantic network in hci
- Types of interpreting
- Interpreting psat scores
- Interpreting political cartoons
- Identify the 7 principles in interpreting parables
- Secondary consumer
- Absolute risk reduction
- Plant cladogram
- Interpreting relative risk
- Interpreting distance time graphs
- Interprétation de coefficient de corrélation
- Compiling vs interpreting
- Chapter 6:2 interpreting word parts
- Chapter 16 solutions answer key
- Interpreting proportional graphs worksheet
- Interpreting topographic maps worksheet
- Interpreting phase diagrams
- 1-8 interpreting graphs of functions
- Interpreting linear graphs
- Phase diagram ap chemistry
- Language line.com
- Boston court interpreting service
- Interpreting phase diagrams
- What does stephanie mean
- Interpret motion graphs
- Interpreting a cladogram
- Rf topographic map
- Enzyme activity graph
- K5 learning interpreting figurative language
- Brigance 111 scoring
- Interpreting political cartoons 2 answer key
- What is the average velocity of the object
- Numerical
- Charter myth definition
- Mmpi-2-rf interpretation worksheet
- Three criteria for interpreting scripture
- Interpreting tongues lesson
- Vertical interval of a map
- Interpreting circle graphs
- Interpreting line graphs year 5