RICAS ELALiteracy Calibration 1 Goals To understand the

RICAS: ELA/Literacy Calibration 1

Goals üTo understand the purpose and benefits of calibration/looking at student work üTo use student work from RICAS Released Items to further understand the RICAS rubric and the implications for instruction 2

Calibration PURPOSE: • Build CONSISTENCY § Interpretation of rubric § Evaluation of qualities found in student work § Application of rubric to score student work across classes, schools, and LEAs • Establish RELIABILITY of scores • ALIGN student work to the standard(s) 3

Calibration can: Align the scorers with the existing rubric & anchor papers OR Identify anchor papers using the rubric 4

Calibration Protocol Directions 1. Independently READ text(s) 2. Complete all of the Selected Response Items 3. Record outline/notes articulating the big ideas you would use to frame your essay response on your scratch paper 4. Identify evidence you would use to answer the essay prompt yourself 5. Predict what you will see in a student’s response 5

Selected Response Discussion • Review and discuss questions, answers, and alignment to the Selected Response Items • Optional: Use information in the Item Analysis Tool to determine how your students scored overall for each item and discuss implications 6 6

Calibration: Student Responses INDEPENDENTLY… § Read Student Responses A, B, and C § Rate the 3 responses: HIGH, MIDDLE, LOW üIs the question asked in the prompt being answered? üWhat is the quality and development of the central idea of the response? üWhat is the quality of the evidence and/or details? üHow well is this response organized? üHow well did the student express his/her ideas? 7

Calibration: Student Responses Small Group Work… § Share your ratings with colleagues § Discuss any differences § Come to consensus on the group rating § Articulate as a group HOW and WHY you all assigned the rating of HIGH, MIDDLE, LOW to each response 8

Calibrating Student Responses INDEPENDENTLY… § Read Student Responses D and E § Rate the two new responses in relation to previous three responses § Rank them from 1 -5, 1 being the highest and 5 being the lowest Keep in mind the qualities of good writing: üAnswering the question asked üDevelopment of central idea üEvidence and/or details selected üOrganization üExpression of ideas 9

Calibrating Student Responses Small Group Work… § Share rating with colleagues § Discuss & come to consensus on the progression of all papers from highest to lowest § Articulate WHY your group ranked them as they did Large Group Discussion to sum up small group discussions 10

Calibration: Rubric Analysis INDEPENDENTLY… § Review RICAS: English Language Arts Essay Rubric § Highlight words that signal change between score points 11

Calibration: Scoring Student Work INDEPENDENTLY… § Assign an Idea Development score to each paper based upon Rubric expectations Small Group Work… § Discuss scores & come to consensus for assigned score 12

Calibration: Annotations INDEPENDENTLY: § Read Annotations for 5 student response papers, if available § Ask yourself: § Did the annotations help your understanding of the score assigned? § Did the annotations highlight a new interpretation for you of the student response? § Did the annotation help clarify language of the rubric? § What might you add to the annotation(s)? 13 13

Calibration: Take Away Discussion Scoring Reflection: üArticulate characteristics found within Score Point 4 response? üIdentify differences between Score Point 3 and 4? Score point 3 and 2? üIf you had discrepant scores among scorers, how were they discrepant? Was it due to interpretation of rubric language? Student expectations differences? Implications for Instruction: üHow do students prepare to understand the expectations of the standard? üHow do we support students in answering the essay questions? üWould a Graphic Organizer be appropriate? üWould additional experiences with multiple texts be helpful? 14

The Calibration Process Ultimate Benefit: Deepens collective understanding of standard & expectations Extensions: § Revise/Review current LEA/school rubrics § Use anchor responses for instruction with students: § Shows students how other peers have responded to Essays § Students can analyze/evaluate sample responses critiquing each writer’s craft & applying to own peer feedback § Review student data to guide current instruction/curriculum 15

Resources RICAS Resource Center • Computer-based RICAS Released Items • Answer key/scoring rubric including Reporting Categories, Standards, and Item Descriptions • Student Work link- This will take users to the Massachusetts Department of Education website. RIDE Item Analysis Resources • Item analysis by subject and grade level 16 16

Literacy Supports on RIDE website RICAS ELA Test Design RICAS: Released Items & Student Samples RICAS: Student Score Report/ Item Analysis & Subscore Additional Calibration Supports: Writing Calibration Tool – Turn Key Protocol Addresses : • BOTH Holistic & Analytic Rubrics • If you have Anchors sets and if you do not have Anchor sets 17 17

Questions Please direct all correspondence to: assessment@ride. ri. gov In the subject line of the inquiry include: RICAS: ELA/Literacy Calibration 18