So Much Data Where Do I Start Assessment

So Much Data – Where Do I Start? Assessment & Accountability Conference 2008 Session #18

Purpose of the Presentation § Review what MI-Access reports are provided § Review what information can be found on the MI-Access reports § Talk about how the results might be used 9/6/2021

Three MI-Access Assessments § Participation: For students who have, or function as if they have, severe cognitive impairment § Supported Independence: For students who have, or function as if they have, moderate cognitive impairment § Functional Independence: For students who have, or function as if they have, mild cognitive impairment 9/6/2021

Selecting the Right Assessment § The IEP Team determines which assessment is appropriate § MI-Access Web page: www. mi. gov/mi-access. Look under “IEP Team Information” section. § The data will only be helpful IF the appropriate assessment is selected 9/6/2021

Content Areas Assessed All three populations are assessed in three content areas in the fall § 1) 2) 3) § 9/6/2021 English language arts Mathematics Science § ELA and mathematics are assessed in grades 3 -8 § Science is assessed in grades 5 and 8 Grade 11 MI-Access students are assessed in the spring (ELA, mathematics, and science)

Students are assessed 9/6/2021 Schools use and distribute reports Answer documents are completed and shipped to Questar, Inc. Districts disseminate reports to schools Questar scans and scores documents Reports are produced and shipped to districts

MI-Access Report Titles Online State Results District Results Folder School Results Folder Individual Student Reports Classroom Results Folder Online Only X Rosters X X Summary Reports X X X Demographic Reports X X X Item Analysis Reports X X X Parent Reports X Student Labels X X District Comprehensive X ISD Comprehensive X 9/6/2021

Individual Student Reports: Design Student demographic information Earned/points possible by component or strand 9/6/2021 Student performance summary Individual item analysis for released items

Individual Student Report: Participation ELA 9/6/2021

How Participation Scores are Derived MI-Access Participation Scoring Rubric • Two types of items • activity-based observation • selected-response • PAA and SAA observe and score student • PAA and SAA scores are added together to determine total item points Score Point/ Condition Code Description 3 Responds correctly with no assessment administrator assistance 2 Responds correctly after assessment administrator provides verbal/physical cues 1 Responds correctly after assessment administrator provides modeling, short of handover-hand assistance A Incorrect response B Resists/Refuses C Assessment administrator provides hand-overhand assistance and/or step-by-step directions

Individual Student Report: Supported Independence Mathematics 9/6/2021

How SI Scores are Derived • Two types of items • activity-based observation (except science) • selected-response MI-Access Supported Independence Scoring Rubric Score Point/ Condition Code 2 • PAA and SAA observe and score student • PAA and SAA scores are added together to determine total item points Description Responds correctly with no assessment administrator assistance 1 Responds correctly after assessment administrator provides verbal/physical cues A Incorrect response B Resists/Refuses C Assessment administrator provides hand-overhand assistance and/or step-by-step directions

Individual Student Report: Participation Science 9/6/2021

Individual Student Report: Functional Independence Science 9/6/2021

How FI Scores are Derived § No scoring rubric for Functional Independence § Only selected-response items § Student receives 1 point for each correct response § The ONLY exception is ELA Expressing Ideas § Open-ended response to a prompt § Scored using a 4 -point rubric § Student receives may receive up to 4 points per prompt

Other Differences: Condition Codes § P/SI Condition Codes: All Content Areas − A = responds incorrectly − B = resists/refuses − C = responds only after the assessment administrator provides hand -over-hand assistance or step-by-step directions § FI Condition Codes: ELA Expressing Ideas Only − − A = off topic B = illegible C = written in a language other than English D = blank/refused to respond § All condition codes = zero points 9/6/2021

Other Differences: Performance Level Change STUDENT PERFORMANCE SUMMARY Earned Points Possible: 0/45 Scale Score: 2391 Performance Level 2007: Emerging Toward the Performance Standard (High) Performance Level 2006: Emerging Toward the Performance Standard (Low) Performance Level Change: Improvement ONLY for Functional Independence ELA and Mathematics

Performance Level Change Fall 2007 Achievement Fall 2006 Achievement Emerging Attained Surpassed Low Mid High Low N I I SI SI SI Mid D N I I SI SI High D D N I I SI SI SI Low SD D D N I I SI SI High SD SD D D N I I SI Low SD SD SD D D N I I Mid SD SD D D N I High SD SD SD D D N SI = Significant Improvement; I = Improvement; N = No Change; D = Decline; SD = Significant Decline

Using ISRs for Instruction and Curriculum § Compare ISR to other data you have for the student. Is this what you would expect? § If prior year and performance level change data are provided, is the student making progress over time? § Using released item booklets (www. mi. gov/mi-access), match to curriculum and instruction. Has the student been taught this? If yes, are teaching methods effective for this student? § Identify strengths to reinforce § Identify areas where additional instruction is needed § Confirm this is the right assessment for the student to take 9/6/2021

Rosters § List scores by individual student § One report for each grade and content area § Provided at class, school, and district levels (not state) 9/6/2021

Show number assessed and mean scale score or mean earned points at top left Rosters List results by student 9/6/2021

Rosters For FI ELA and mathematics, shows current year scale score, two years of performance level (high, mid, and low), and performance level change 9/6/2021

Rosters Shows earned points by component or strand, EGLCE or EB, and overall 9/6/2021

Using Rosters for Instruction and Curriculum § Best place to gather information about assessment items, because results are provided by EGLCE or EB § www. mi. gov/mi-access Supported Independence Grade 5 ELA: Comprehension Math: Geometry Science: Reflecting R. NT. e 4. SI. EGO 4 a G. LO. e 1. SI. EGO 2 AA R. RO. SI. EB. II. 1. e. 4 a Identify what makes stories fiction vs. fact and why an author makes that choice Identify positions of objects in space using terms describing relative position (first, last, above, below, next to, etc. ) Develop an awareness of natural world. Key concepts: Nature, observation, personal safety. Real-world contexts: Caring for the environment; pollution; recycling; water safety

Summary Reports § Executive summaries of student scores § Provided at school, district, and state levels § One report for each grade and each content area assessed (ELA, mathematics, and/or science) § Report generated ONLY when there are 10 or more students at the same grade level taking the same assessment 9/6/2021

Supported Independence District Summary Report: ELA Summary results shown for current year at top right 9/6/2021

Supported Independence District Summary Report: ELA Shows # and % of students that earned each earned points total 9/6/2021

Functional Independence Summary Reports: ELA and Mathematics Grades 4, 5, 6, 7, and 8 ONLY PERFORMACE LEVEL CHANGE — YEAR-TO-YEAR TRANSITIONS Number and Percent of Students by Performance Level Change (from 2006 to 2007) Fall 2006 Emerging Attained Surpassed 9/6/2021 Fall 2007 Emerging Attained Surpassed 104 (30. 2%) 8 (2. 3%) not gaining 32 (9. 3%) 24 (7. 0%) 8 (2. 3%) declining maintaining gaining 48 (14. 0%) 8 (2. 3%) 104 (30. 2%) declining maintaining

Functional Independence Summary Reports: Performance Level Change § “Gaining” means there was an improvement from 2006 to 2007 § “Maintaining” means scores that were “proficient” stayed the same § “Not gaining” means scores that were “not proficient” stayed the same § “Declining” means there was a decline from 2006 to 2007 9/6/2021

Functional Independence Summary Reports: ELA and Mathematics Grades 4, 5, 6, 7, and 8 ONLY PERFORMANCE LEVEL CHANGE — SUMMARY Years Summarized Students Matched # % Performance Level Change Significant Decline # 2006 & 2007 9/6/2021 344 93. 5 88 % 25. 6 Decline No Change # % # 0 0. 0 232 Improvement % # % 67. 4 0 0. 0 Significant Improvement # % 24 7. 0

Using Summary Reports for Instruction and Curriculum § Look at progress collectively. How did our fifth graders do this year compared to last year? § Look at progress over time. Are there trends to be aware of? § Compare state assessment data to other data. Is this what we would expect? § Within the performance levels, where are clusters of students?

Functional Independence Summary Report: Science 9/6/2021

P/SI Parent Reports Page 1 shows student’s earned points and performance level for current year. 9/6/2021

P/SI Parent Reports Page 2 shows appropriate scoring rubric, and ELA and mathematics scores by component or strand ELA Mathematics

P/SI Parent Reports Science ELA Page 3 shows science scores by strand ELA individual student item analysis

P/SI Parent Reports Mathematics Science Page 4 shows mathematics and science individual student item analysis

Functional Independence Parent Reports (Grades 4, 5, 6, 7, & 8) Page 1 shows the student’s scale score and performance level for 2007 and 2006. Also shows “Performance Level Change” for current and past year. 9/6/2021

Functional Independence Parent Reports Page 2 shows student’s earned points for ELA and mathematics, and his/her scale scores in a range ELA Mathematics 9/6/2021

Functional Independence Parent Reports Page 3 shows student’s earned points and scale score range for science. Also shows the student’s item analysis for ELA Science ELA 9/6/2021

Functional Independence Parent Reports Page 4 shows student’s individual item analysis Mathematics Science 9/6/2021

Item Analyses § Provide detailed, aggregated information on released items § Can be used to identify areas of collective strengths and areas that need improvement § Provided at school, district, and state level § Provided only when 10 or more students in the same grade take the same assessment 9/6/2021

District Item Analysis: ELA Shows # and % of students who selected each answer choice for each released Word Recognition and Text Comprehension item 9/6/2021

District Item Analysis: ELA Shows the # and % of students at (1) each score based on a 4 -point rubric, and (2) each condition code 9/6/2021

District Item Analysis: ELA Also shows the # and % of students that received specific comment codes 9/6/2021

Using Item Analyses § Use the Item Analysis Report along with the Released Item Booklet to identify collective strengths and weaknesses (available at www. mi. gov/miaccess) 9/6/2021

Using Item Analyses: ELA State Fall 2006 § Functional Independence Grade 3 § Informational passage about a chameleon § EGLCE being measured: Make inferences, predictions, and conclusions § Only 62. 1% answered the item correctly (A) § C was the incorrect answer chosen most often (23%) 9/6/2021

Using Item Analyses: ELA State Fall 2006 § Same narrative passage § EGLCE being measured: Identify main ideas and details § 74. 7% of students answered the item correctly (B) § 9/6/2021 A was the incorrect answer chosen most often (13%)

Using Item Analyses: ELA State Fall 2006 Expressing Ideas § EGLCE being measured: Write/draw personal narrative § Only 7% of students received a “ 4, ” and 23% received a “ 3” § Comment code given most often = “Showed limited development with insufficient details and/or examples” 9/6/2021

Using Item Analyses: Math State Fall 2006 § Functional Independence Grade 7 § EGLCE being measured: Solve problems using data § Only 33. 6% answered correctly (A) § C was the incorrect answer chosen most often (54. 6%) 9/6/2021

Using Item Analyses: Mathematics State 2006 § EGLCE being measured: Recognize representations for whole numbers to 10, 000 § 96. 5% answered correctly (C) § B was the incorrect answer chosen most often (2. 2%) 9/6/2021

Using Assessment Results § “Results have the greatest impact when people are given opportunities to discuss them—teachers with teachers, coordinators with coordinators, and teachers with parents. You discover trends, identify anomalies, start asking questions, and together seek answers. ” 9/6/2021

Contact Information Peggy Dutcher, Coordinator Professional Development OEAA Email: dutcherp@mi. gov Phone: 517 -335 -0471