USING SMARTER BALANCED RESULTS FOCUSING ON CLAIMS AND

USING SMARTER BALANCED RESULTS FOCUSING ON CLAIMS AND TARGETS September 2018 HSAP Training 1

When you think of claim and target reports, do you feel like this? Or maybe like this? Hopefully by the end of this session, you will feel like this! Or we’ll feel like this!! 2

Knowledge Check #1 Poll How familiar are you with the ORS and Smarter Balanced Claims and Targets? 2 minutes Using your phone (or laptop if you have a hot spot), go to: End • www. menti. com • enter code 176772 3

Smarter A Balanced Assessment System Summative Assessments: College and career readiness assessments for accountability Common Core State Standards Digital Library: Teacher resources formative tools practices to improve instruction Teachers and schools have information and tools they need to improve teaching and learning All students leave high school college and career ready Interim Assessments: Flexible, open, used for actionable feedback 4

Hawaiʻi Smarter Balanced Summative Results Sample Graph – Hawaii State Performance – Grade 8 Mathematics Range of student performance Widest part = level where MOST students performed 90 th percentile 50 th percentile mean 10 th percentile 5

Investigating Claim and Target Data: An Overview 1. What are claims and targets? 2. Where can I learn about how my school performed on claims and targets? 3. How do I understand claim and target data? 4. What are my next steps? 6

Investigatin g Claim and Target Data 1. What are claims and targets? 2. Where can I learn about how my school performed on claims and targets? 3. How do I understand claim and target data? 4. What are my next steps? 7

Assessment Claims and Targets Overall Claims ELA/L Math Grade 3 - 8 Students can demonstrate progress toward college and career readiness in English Language Arts and Literacy. Students can demonstrate progress toward college and career readiness in mathematics. Grade 11 Students can demonstrate college and career readiness in English Language Arts and Literacy. Students can demonstrate college and career readiness in mathematics. Content Claims • broad evidence-based statements that describe what students know and can do as demonstrated by their performance on the assessments. • There are 4 claims each for Math and ELA/L § ELA/L v Claim 1: Reading v Claim 2: Writing v Claim 3: Speaking and Listening v Claim 4: Research § Math: v Claim 1: Concepts and Procedures v Claim 2: Problem Solving v Claim 3: Communicating Reasoning v Claim 4: Modeling and Data Analysis Targets • map the standards in the CCSS onto assessment evidence that is required to support the content categories and claims • guides the development of items and tasks that will measure the CCSS • A set of targets is aligned to each content area claim. 8

Relationship between Smarter Balanced Claims/Targets and the Common Core Standards Overall Claims Targets Hawaii Common Core Standards Example: Grade 3 ELA/L Relationship Overall Claim Students can demonstrate progress toward college and career readiness in English language arts and literacy. Claim 1 (Reading) Target 1 (Key Details) Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts. Given an inference or conclusion, use explicit details and implicit information from the text to support the inference or conclusion provided. CCSS 3. RL. 1 (Reading Literature Standard 1) Ask and answer questions to demonstrate understanding of a text, referring explicitly to the text as the basis for the answers. 9

How Will Claim and Target Information Help My School? Claim Report • Provides a snapshot of how students are progressing towards gradelevel expectations in each content area • Provides school administrators with an understanding of learning trends in their schools • Provides teachers with an understanding of their students’ progress and achievement, and allows them to identify areas of strengths and areas needing support Target Report • Provides information about how students (down to the roster level) performed on specific skills in each content area • Can be used as a guide for professional development for content area teachers 10

Knowledge Check #1 Results Let’s look at our poll results! 11

Investigatin g Claim and Target Data 1. What are claims and targets? 2. Where can I learn about how my school performed on claims and targets? 3. How do I understand claim and target data? 4. What are my next steps? 12

Online Reporting System (ORS) ■ Digital format of the Family Report, the Individual Student Report (ISR) ■ Aggregate claim- and target-level reports ■ Summary statistics ■ Export student results ■ Roster management 13

ORS Welcome Page To view the SBA data: 1. Select the desired location (complex area, complex, school, etc. ) that you are interested in viewing depending on your role 2. Click on “Score Reports” 14

ORS Home Page Dashboard 1. Select “Smarter Summative” and the administration date (2017 -2018) 2. Click on the third radio button 3. Click on a grade and subject to view complex area data 15

Student Performance in Each Achievement Level by Complex 16

Viewing Claims and Targets 17

ORS Student Report 18

Individual Student Report - Print Version 19

Investigatin g Claim and Target Data 1. What are claims and targets? 2. Where can I learn about how my school performed on claims and targets? 3. How do I understand claim and target data? 4. What are my next steps? 20

Claims Report (by Student) Claims Achievement Category Above Standard At/Near Standard Below Standard 21

Claim Performance at the Student Level White bar – claim achievement standard Black bar – student’s performance relative to the claim achievement standard Green bar – the score range that the student would likely fall within if he/she took the test multiple times (Standard error of measurement) The icons next to each bar chart shows how the student performed for that claim: • Above the standard • At/near the standard • Below the standard 22

Assessment Targets Ø Assessment Targets connect the CCSS to evidence that will be collected from the assessment. Ø The targets map the standards in the CCSS onto assessment evidence that is required to support the content categories and claims. Ø Assessment targets are used to guide the development of items and tasks that will measure the CCSS. 23

Targets Report - Mathematics Performance Relative to Proficiency: Provides information about a group’s target performance in relation to proficiency levels. This column displays whether the target performance is above, at, or below the Smarter Balanced proficiency levels. Performance Relative to the Test as Whole: Shows how a group of students performed in each target relative to their performance on the entire test. Unlike achievement levels (1 -4) provided for the entire test, the strengths and weaknesses at each target on this report are relative to the test as a whole. 24

Targets Report – ELA/L 25

Smarter Balanced Blueprint: Mathematics Targets Claim 1 26

Smarter Balanced Blueprint: ELA/L Targets Claim 1 27

Diving Deeper Claim and Target Reporting Walkthrough 28

1. Generate a Claims Report (School) Ms. Karen Ms. Kelsie Ms. Dianne 29

2. Generate a Targets Report (Grade 5) 30

3. Determine Targets to Prioritize 31

4. Prioritize Target C Target D Target E 32

5. Identify the standards that align to the identified targets • Locate resources that align Assessments to the CCSS to find standards that align to the identified targets 33

6. Identify sample questions that map to these targets/standards http: //www. smarterbalanced. org/assessments/developme nt/ 34

35

Math Item Specification s Grade 5, Claim 1, Target C 36

37

Sample question stems clarify what that evidence could look like. The Item Specifications show what evidence to collect from students in order to evaluate student understanding. 38

Investigatin g Claim and Target Data 1. What are claims and targets? 2. Where can I learn about how my school performed on claims and targets? 3. How do I understand claim and target data? 4. What are my next steps? 39

Table Talk 5 minutes End How does your school use the Smarter Balanced Summative results? (e. g. , types of analyses, interventions, etc. ) 40

Potential Next Steps ü Identify a Team ü Gather Evidence – What evidence will your school rely upon to determine learning progress? How often will they collect this evidence? • Data ü Analyze and Identify • Areas of Need – What areas of learning require more support? • Areas of Strength – How can these successful strategies be shared around the school? ü Strategize and Plan Using Evidence • What works for your school may not work for all schools • Standards • Instructional Research ü Utilize Assessment-Related Resources • Interim Assessment Comprehensive Assessments • Interim Assessment Blocks • Digital Library and Instructional Supports • Item Writing Project and Assessment Literacy ü Monitoring and Check-ins ü Repeat cycle as necessary – It is very likely that this will not be a static process. 41

A Balanced Assessment System English Language Arts/Literacy and Mathematics, Grades 3 -8 and 11 School Year Last 12 weeks of the year DIGITAL CLEARINGHOUSE OF FORMATIVE TOOLS, PROCESSES AND EXEMPLARS Released items and tasks; Model curriculum units; Educator training; Professional development tools and resources; Scorer training modules; Teacher collaboration tools; Evaluation of publishers’ assessments. Interim Assessments (ICA/IAB) Computer Adaptive Assessments and Performance Tasks PERFORMANCE TASK • ELA/Literacy COMPUTER ADAPTIVE TESTS • ELA/Literacy • Mathematics Scope, sequence, number and timing of interim assessments locally determined 42

Connecting Smarter Balanced Results and Instruction Administer ICAs and/or IABs throughout the year. Retrieve and analyze student results in ORS and AIR Ways reporting. Educators collaborate with each other and plan next steps for instruction and find resources in Digital Library. 43

Interim Assessments § Interim Comprehensive Assessments (ICA) § Measure same content and standards as the Summative Assessments § Can be administered multiple times during the year and at grade levels that are not tested (including grades 9 and 10) § Can be used as a Practice Test § Interim Assessment Blocks (IAB) § Focus on specific topics § Each block has between 4 and 18 items depending on grade and content area § Provide information about student performance in three categories: Above Standard, Near Standard, and Below Standard § Between 5 and 12 blocks per content area per grade 44

Interim Comprehensive Assessment (ICA) Scores ■ The ICAs are reported using the same three types of scores as the Summative: § Overall scale score § Achievement level § Claim achievement category ■ The scores are determined the same way as the Summative Scores. 45

Interim Assessment Block (IAB) Scores ■ The IABs are reported using a “performance category”: – Above – At/Near – Below ■ This is very similar to the “claim achievement category” on the Summative. ■ The IAB performance categories are determined the same way as the summative claim score categories. 46

Digital Library 47

Questions? ■ Claim reporting? ■ Target reporting? ■ Item specifications? ■ Interim Assessments? ■ Digital Library? ■ Other? 48

Resources ● Online Reporting System - allows users to view aggregate and individual summative and interim results ● AVA System - allows users to view the interim assessment test question ● Interim Teacher Hand Scoring System - allows users to score interim written responses ● AIRWays Reporting - allows users to view interim assessment student responses ● Digital Library - over 3000 classroom resources created and used by teachers 49

Additional Resources ■ HSAP portal: http: //alohahsap. org/ ■ Math and ELA/L Blueprints and Reporting information: http: //alohahsap. org/SMARTERBALANCED/resources/ ■ Smarter Balanced Item Specifications: http: //www. smarterbalanced. org/assessments/develop ment/ ■ Common Core Standards: http: //www. corestandards. org/read-the-standards/ ■ Hawaii Smarter Balanced Blueprints: https: //smarterbalanced. alohahsap. org/resources/gener al-information/ ■ Riverside County Office of Education Crosswalk Documents 50

Resources / Support 1. Resources and documentation available at the portal: alohahsap. org 2. HSAP Help Desk: § Hours: 7: 30 am to 4: 00 p. m. HST, Monday-Friday (except holidays) § Phone: 1 -866 -648 -3712 (toll free) § Email: HSAPHelp. Desk@air. org 3. Contact the Assessment Section § Phone: (808) 733 -4100 § Email: HSA/SAS/HIDOE@notes. k 12. hi. us 51