Vision Every child in every district receives the

Vision: Every child in every district receives the instruction that they need and deserve…every day. Intervention Review Meetings Is what WE are doing working? Cadre 11 Data Day March 2019

RTI Essential Components Clear Vision of Learning for ALL SLD Decision Making Decision Rules Progress Monitoring Intervention Placement Initial and Ongoing PD Ongoing Coaching BIT Meets Regularly Interventions Intervention Principal Attends RTI Meetings Universal Screening Review Core Review Observe & Actionable Feedback Teaming & DBDM Professional Learning Culture of Collaboration to Improve Outcomes Core Materials and Instruction Leadership Growth Mindset & High Expectations For ALL Students

Installation Matrix

RTI Handbook

RTI Team Meetings Tier 1 Core Meetings Tier 2 & 3 Placement Meetings • Reading • Behavior • Attendance Intervention Review Meeting • Intervention Review Meetings • Reading • Behavior • Attendance Tier 3 Individual Problem Solving Meetings

“intervention failure should be an exceedingly rare event…” “Overemphasis of intervention selection and under emphasis of intervention management is probably the most common error in RTI that we see. Again, we remind that intervention failure should be an exceedingly rare event. Where it is not a rare event implementation errors are at work. ” Vander. Heyden and Tilly III 2010

An Important Caveat We often focus on the Data-Based Decision Making when it’s the Data-Based Decision Implementing that really matters most A decision might be “good” or “bad” depending on the quality with which it is implemented

What are Intervention Review Meetings? Team meetings to determine if students are making progress in interventions – What was the goal? – Look at group and individual growth. – Apply Decision rules. – Make a plan! Are we closing the gap?

Intervention Review Meetings • Prerequisites • Logistics • Process

Intervention Review Meetings: Prerequisites • Goal setting – How do we set goals for progress monitoring? • Determine decision rules for adequate progress – How do we know if students are making progress?

Intervention Review Meetings: Prerequisites • Goal setting – How do we set goals for progress monitoring? • Determine decision rules for adequate progress – How do we know if students are making progress?

Why is setting an appropriate goal important? • To know if we are on the right path • To evaluate performance against a standard • Gives the students something to reach for • Effective size: . 56

How do we set an Ambitious, Attainable, and Appropriate goal? • Where we set the goal will help us to determine if we are making adequate progress • Better to set an ambitious goal and not obtain than to set a lower goal and reach it – Goal is to close the gap

Setting Ambitious, Attainable, and Appropriate Goals

Goal Setting • CTL (UO DIBELS) and Acadience/Amplify have different ways to set goals

Acadience/Amplify: Goal Setting • Student growth percentiles provides a measure of "how (ab)normal a student's growth is by examining their current achievement relative to their academic peers ‐‐ those students beginning at the same place" (Betebenner, 2011, p. 3).

Establishing Goals

Goal Setting

m. Class Amplify

m. Class Amplify You can adjust this goal up or down

Video to Help with Amplify • Log into m. Class/Amplify • Go to class data • Click on this to watch a good video about goal setting

CTL: Goal Setting • Option 1: At the end of year Benchmark • Option 2: Individually determined based on ambitious, attainable rates of improvement (ROI)

Option 1: At the end of year Benchmark

Option 1: At the end of year Benchmark PROS • Consistent and easy to determine • Sets high expectations for ALL students CONS • One size fits all, (Not individualized) • May not be attainable for your most at-risk students

3 rd Grade DIBELS ORF Option 2: Individually determined based on Rates of Improvement (ROIs) 100 - 70 = 30 WCPM / 36 Weeks = . 83 WCPM per week 100 86 70 100 – 30 = 70 WCPM 30 70 WCPM / 36 Weeks = 1. 9 WCPM per week Sep Oct Nov Dec Jan Feb Mar Apr May

Option 2: Individually determined based on ROIs 1. Determine the average/typical ROI for students at the grade level 2. Multiply that ROI by 1. 5 or 2 to get your desired ROI 3. Multiply the desired ROI by the # of instructional weeks until the end-of-year goal to get the desired yearly growth 4. Add the desired yearly growth to the student’s current level to get the end-of-year goal

3 rd Grade DIBELS ORF New Goal Based on ROI 100 - 70 = 30 WCPM 100 – 20 = 80 WCPM 30 WCPM / 36 Weeks = 80 WCPM / 36 Weeks = . 83 WCPM per week 2. 2 WCPM per week 100 82 86 70. 83 X 1. 75 =1. 45 30 1. 45 X 36 = 52 52 + 30 = 82 Sep Oct Nov Dec Jan Feb Mar Apr May

Considerations • Will the goal get the student to the next level of risk? • What is the long term goal for the student? – When can they get to benchmark? • Make the right decisions about progress • Student motivation Ambitious but Attainable!

Option 2: Individually determined based on ambitious, attainable ROIs CONS PROS • Need to be • Ensures all individually students receive calculated (more goals that are complicated) ambitious yet • Some students attainable may have goals • May be more that still leave appropriate for your them at-risk at the most at-risk end of the year If they will not meet Benchmark this year, when are students you anticipating them reaching that goal?

When to progress monitor below grade level • Progress monitor at the highest level that you can see growth • Below grade level progress monitoring should be rare • ALL students screened AT grade level 3 x year – May also monitor at grade level 1 x month • Goals for students monitored below grade level should be set to reach proficiency at that grade level in less than a year’s time

Practice • With your own data…. . practice setting goals for a few students

Intervention Review Meetings: Prerequisites • Goal setting – How do we set goals for progress monitoring? • Determine decision rules for adequate progress – How do we know if students are making progress?

What are Decision Rules for Intervention Review Meetings? • When to exit a student from an intervention… • When to keep going with an intervention… • When to change an intervention… • When to modify an intervention…. • When to individualize an intervention (IPS)…

Start with the WHY Why How What Simon Sinek

WHY do we rely on data to determine if the interventions are working?

Which line is longer? Intuitions can be deceiving Which line looks longer?

We are not impartial decision makers We have inherent biases…. . • We tend to believe information that confirms our beliefs and disregard information that disagrees with our views • When we have experience with something it biases what we think about it

Teacher judgment of student reading abilities (Begeney, Krouse, Brown & Mann, 2011) Compared teacher judgment of student reading levels to student DIBELS scores (grades 1 -6) • “…teachers often made inaccurate judgment about students’ actual reading performance, tended to overestimate students’ abilities, and were better judges of high-performing versus low- or average-performing readers. ” • When asked to identify a student’s level of reading risk level, teachers were correct for roughly 58% of their students – There’s a 33% chance of being correct by randomly guessing (Similar findings in Begeny, Eckert, Montarello, & Storie, 2008; Feinberg & Shapiro,

WHY do we use Data-Based Decision Rules? They help us decide if what WE are doing is working They create consistency in how we respond across grade levels and schools They help us determine how to intensify interventions They standardize the process for eligibility decision making

Decision Rules are used to guide our decision making This means that… …thinking is still required …decision rules don’t make the decision for us …decision rules trigger us to stop and think Pause Analyze Respon d

What is adequate progress? • Each DIBELS system CTL & Acadience/Amplify has it’s own way of depicting adequate progress

CTL: What is adequate progress? • Compare the Aimline (expected growth) to the progress monitoring data points (actual growth) “Are we on track to meet our goal? ”

CTL: Compare the Aimline to progress monitoring data points Look at the last 3 data points If the median of the last 3 data points is above the aimline, the student is making adequate progress If the median of the last 3 data points is below the aimline, the student is not making adequate progress

This student is making adequate progress What is the median data point of the last 3? Is it above or below the aimline?

This student is not making adequate progress What is the median data point of the last 3? Is it above or below the aimline?

CTL: Decision Rule When to exit, keep going, or change an intervention • If the median of the last 3 data points is above the aimline, the student is making adequate progress • If the median of the last 3 data points is below the aimline, the student is not making adequate progress

Acadience/Amplify: What is adequate progress? • Students should be in the above average or well above average growth zone

Acadience/Amplify: What is adequate progress?

m. Class Amplify Most data points are in the Well Above Typical Progress Zone

Acadience: Pathways of Progress Most data points are in the typical or above typical progress zone Well Above Typical Below Typical Well Below

Acadience: Decision Rule When to exit, keep going, or change an intervention? • Students are in the above or well above typical zones are making adequate progress • Students in the typical zone are questionable • Students in the below or well below typical zones are not making adequate progress

Integrating other data sources Progress Monitoring Data (DIBELS) Pause Analyze Respon d Intervention assessments Core assessments Observations of intervention Fidelity data

Considerations for ELs

Considerations for ELs

Intervention Review Meetings • Prerequisites • Logistics • Process Bird Walk is Over

Logistics: What do you need to do this work? • Standard Reading Protocol with intervention menu • Data-Based Decision rules • Graphed progress monitoring data • Scheduled time, space and team • In-curriculum assessments • A Growth mindset!

WHO sits at the table? Literacy Specialist/ Title I Grade Level Teacher SPED Teacher Principal May also include: ELD Teacher School Psych/ Counselor Others?

HOW OFTEN, HOW LONG and WHEN do we meet? • How often: – Every 8 -10 weeks, depending on… • Your decision rules • Your weekly schedule • How long: – Approximately 1 hour (per grade level) • When: – After school? – Before school? – During school?

WHY do we meet every 8 to 10 weeks? • Evidence-based interventions do not always work immediately. • In 8 -10 weeks, we do not necessarily expect that the intervention has worked, but we expect that the intervention is “working”.

WHY can’t we just make a change earlier than 8 -10 weeks? • Modifications in instructional delivery should be ongoing • Changes to a different intervention should only be made at the group intervention review meeting because: – Intuitions can be deceiving – We can see the flaws in other people’s thinking much easier than we see the flaws in our own – Data-Based Decision Making is a team sport

Sample Meeting Schedule: 8 Week Cycle Sunday Monday Tuesday Wednesday Thursday Friday Week 1 Kinder 1 st Grade Data due Week 2 1 st Grade 2 nd Grade Data due Week 3 2 nd Grade 3 rd Grade Data due Week 4 3 rd Grade 4 th Grade Data due Week 5 4 th Grade 5 th Grade Data due Week 6 5 th Grade Week 7 RTI Leadership Week 8 Staff PD Kinder Data due

Use an Agenda, Guiding Questions Document The agenda will: • Guide your team’s decision making • Ensure data-based, equitable decisions are made • Keep the team focused on decision rules • Keep the team focused what we can control • Help to avoid storytelling

Talk Time • How will you explain to your staff the importance of the following things: – Having AND Using decision rules for when to change interventions – Meeting every 8 -10 weeks to determine if interventions are working

Intervention Review Meetings • Prerequisites • Logistics • Process

Intervention Review Meeting Process • Brief review of Core Review Meeting • Determine if groups and/or individual students are making adequate progress • Exit, keep going, or change intervention • Make a plan to address changes needed • Document these changes

Core Review…. Review A brief update of core review agreements: • What were the common instructional agreements at the grade level? • How is implementation going? – Do you have any student data or walkthrough data? • What additional support do you need for implementation of your plan?

Intervention Review Meeting Process • Brief review of Core Review Meeting • Determine if groups and/or individual students are making adequate progress • Exit, keep going, or change intervention • Make a plan to address changes needed • Document these changes

WHY do we start by looking at intervention groups rather than talking about every individual student? • Increases your efficiency – Can discuss more kids in less time • Identify a group vs. individual problem – Requires different solutions • Keep the focus on what we are doing

Determine if there are group problems and/or individual student problems Group Problem Individual Problem • The majority of the students in the intervention group are not making adequate progress • The majority of the students in the intervention group are making adequate progress, but one or two are not • (likely an implementation issue) What is adequate progress?

Group or Individual Problem?

Talk Time • What did you notice about how they used data to determine a group vs. individual problem?

Intervention Review Meeting Process • Brief review of Core Review Meeting • Determine if groups and/or individual students are making adequate progress • Exit, keep going, or change intervention • Make a plan to address changes needed • Document these changes

If we have a group problem… “Again, we remind that intervention failure should be an exceedingly rare event. Where it is not a rare event implementation errors are at work. ” Vander. Heyden and Tilly III 2010

Agenda: Group Problems

Make a Plan Group Problem: size reduced, pre-teach added, feedback examined, core instruction aligned, behavior procedures added

Talk Time • What did you notice about how the team focused on what they could control versus things that they could not control?

Intervention Review Meeting Process • Brief review of Core Review Meeting • Determine if groups and/or individual students are making adequate progress • Exit, keep going, or change intervention • Make a plan to address changes needed • Document these changes

Individual Problems

Individual Problem Fidelity

Additional Diagnostic Assessment d e d e l i n h t c o Y N R E V E r o f • Look to your Assessment Protocol • Assessments to consider – Intervention Placement Test – Phonics Screener – CORE Assessments – DRA – QRI – Informal observations of the intervention

Changes to Interventions • Standardized document for the district! • Typically – Reduce group size – Adding time to the intervention/target skill – Add behavior plan – Change or add curriculum

A change of intervention does not necessarily mean a change of intervention curriculum. It means a change in intensity matched to need.

Changing the curriculum If you have placed a student in a researchbased intervention that is appropriately matched to their instructional need, delivered with fidelity AND They are not making adequate progress… …, then changing to a different intervention curriculum has a very low likelihood of increasing student success.

Example from Bend-La. Pine: Intervention Change vs. Modification INSTRUCTION CURRICULUM ENVIRONMENT

Let’s Examine Some Data!

Exit? Decision Rule: Median of Last 3 Data Points

Decision Rules for Exiting Students (Tigard-Tualatin SD Example) Exit students from interventions when: • The student has three progress monitoring data points at or above the next DIBELS Next benchmark, and core reading unit assessments are above the lowest 20% of their grade level peers, and intervention in-program assessments indicate adequate acquisition of skills. • Continue progress monitoring exited students for an agreed-upon timeframe and provide necessary support to classroom teacher to ensure successful transition out of intervention.

Keep Going? Decision Rule: Above or Well Above Zones

Change Intervention? Decision Rule: Median of Last 3 Data Points

Change Intervention? Decision Rule: Above or Well Above Zones

Document Your Change Add 15 more minutes of Reading Mastery

Tracking Attendance

Notify Parents • Inform parents of student’s placement & progress in intervention – Call parent – Progress reports

Some Final Thoughts • Focus less on making the “right” decision, and more on making the “best” decision you can – We can’t be perfect, but we can be better • The best decision you can make for a student is one that… …is based on all of the best evidence available (e. g. , is deliberate & has the best odds of success) …everyone on the team is committed to implementing well

Talk Time • How will you talk to your staff about the following…. . – Starting by examining intervention groups instead of going student by student – Using DIBELS for progress monitoring and not just relying on in-curriculum assessments

Practice Time If you have the following things… • Standard Reading Protocol with intervention menu • Data-Based Decision rules • Graphed progress monitoring data • Scheduled time, space and team • In curriculum assessments …Take a look at one of your groups and practice going through the steps. If you don’t have all of these in place, what are your next steps to prepare for this process?