Reviewing SBAC Summative Assessment Results to Guide Instructional
Reviewing SBAC Summative Assessment Results to Guide Instructional Practice Shannon Wells Ekaterina Forrester Printed by the Riverside County Office of Education
From Scores to Instruction Overall Scores: ELA Math Claim Scores: ELA: Reading, Writing, Listening/Speaking, Research/Inquiry Math: Concepts and Procedures, Communicating Reasoning, Data Guiding Teaching and Learning: Use of guiding questions to consider implications for practice
SBAC Results Reporting: What is Available? • Public reporting site: research files, aggregate reports (by subgroup) http: //caaspp. cde. ca. gov • Electronic data file (final individual student results in TOMS) • Student score reports
Site Review of SBAC Results • How will your district or school review the SBAC results? What are you going to be looking as a team? o Site-wide data teams, o Grade-level/Department teams, o Individual teachers • What format will be available for review? o Data management systems (Illuminate, OARS, EADMS) • What will be available? o Electronic files, graphs/charts, reports? • Will data be disaggregated by significant subgroups • Will individual student names and needs (e. g. , SED, SWD, Foster, EL) be available along with their scores? • Disaggregated data can include: Parent Ed level; gender by (data not publicly available); LTELs; Native Indian? (not from America); Migrant, etc
Site- and Grade-Level Teams 5
Guiding Questions for Site- and Grade-Level Teams • What do we notice about % of students in each level? What is surprising? • How does overall % in each AL for our site compares to district, county, state? • Grade level % in each AL: any particular grade stands out? • Performance of subgroups: high-performing vs low-performing • Performance by parent education level • Do the overall scores suggest one content area is stronger than the other? • What areas of need (weaknesses) do claim scores within each subject and grade level suggest? • Look at % of level 1 students • What areas should particular grade levels focus on this year? • What do we notice about average scale scores for each grade level? (please clarify – What g/l are they performing at? ) • How do the average grade level scores compare with the “Meet the Standard” threshold? • How are average grade level scores different by subgroups?
SAMPLE SITE: ACHIEVEMENT LEVELS BY GRADE 100% 9% 8% 10% 9% 17% 10% 6% 9% 80% 21% 15% 26% 15% 17% 19% 28% % in Achievement Level 70% 29% 30% 60% 33% 50% What do you notice about the % in each achievement level? What surprises you? 16% 90% 29% 26% 31% Standard Exceeded 38% Standard Met Standard Nearly Met 25% 40% 30% 20% 45% 36% 39% 42% 46% 50% 41% 33% 31% 10% 0% 3 rd Grade 4 th Grade 5 th Grade 6 th Grade 7 th Grade 8 th Grade 11 th Grade County All 7 State All Standard Not Met Which grades are doing better? What do you think contributed to this?
Sample Site Percent of Students in Achievement Levels by Ethnicity 100% 8% 90% 80% 5% 8% 21% 23% 5% 13% 15% 26% 70% 29% 31% 35% 60% 27% 50% Standard Exceeded 29% Standard Met 32% 40% Standard Nearly Met Standard Not Met 24% 30% 54% 20% What do you notice about the % in each achievement level for different subgroups? 41% 48% 38% 27% 10% 20% 0% African American Hispanic White African American Hispanic ELA Math 8 White
Percent of Students Meeting or Exceeding Standard by Parent Education Levels 70% 60% What do you notice about the % meeting or exceeding standard based on parent education level? % Meeting or Exceeding 50% 40% 60% 30% 46% 44% 20% 28% 10% 17% 0% HS Grad or Less Some College Parent Education Level ELA Math 9 College Grad
Sample Site Percent of Students Meeting/Exceeding versus Not Meeting Standard by Language Fluency and Disability 80 What do you notice about the % of EL and SWD students meeting or exceeding standard? 70 60 What do you notice about the % of EL and SWD students not meeting the standard? 50 40 75 75 68 65 30 46 44 20 37 28 32 28 37 30 10 10 9 9 9 0 Standard Met/Exceeded (3 -4) Standard Not Met (1) Standard Met/Exceeded (3 -4) ELA Standard Not Met (1) Math EO EL Non-SWD 10
Average SBAC ELA Scale Score by Grade and Ethnicity Co ll Re ege a 268 dy 2 2650 2600 Average Scale Score 2550 2500 African American Hispanic White 2450 Meets Standard How do the average grade level scores of different subgroups compare with the “Meet the Standard” threshold? 2400 2350 2300 3 rd Grade 4 th Grade 5 th Grade 6 th Grade 7 th Grade 11 8 th Grade 11 th Grade
Average SBAC Math Scale Score by Grade and Ethnicity Co ll Re ege a 271 dy 8 2650 2600 Average Scale Score 2550 2500 African American Hispanic White 2450 Target How do the average grade level scores of different subgroups compare with the “Meet the Standard” threshold? 2400 2350 2300 3 rd Grade 4 th Grade 5 th Grade 6 th Grade 7 th Grade 12 8 th Grade 11 th Grade
Individual Teacher Rosters and Reports 13
FEED BACK: How Did We Do? Questions for 2014 -15 teachers of record looking at their 2014 -15 students’ scores: • How did my students do compared to other students in the same grade level? • Looking at my 2014 -15 students, who met or exceeded standards, what contributed to their success? • Looking at my 2014 -15 students, who were below standard, what instructional strategies or supplemental resources would I add? (implementation of strategies, practices or resources) • How can instructional time be adjusted to increase the number of students who are meeting or exceeding standard? (usage of time) • Which claims within the content area are higher performing? • Which claims within the content area are lower performing? (What are you going to do to go deeper in the claims? ) • How do these results match the areas where I provided targeted instruction? • How do these results point to gaps in instruction? • What might the implications be for instructional practice? For student learning?
FEED FORWARD: Where Are we Going? Questions to ask about students in your class in 2015 -16: • What are the strengths and needs of my class(es) based on their claim scores in ELA and math? • Which claims within the content area are higher performing? Lower performing? • (What are you going to do to go deeper in the claims? ) • If there are patterns of need (e. g. , more than half of my class scored below standard on “Concepts and Procedures”), what changes can I make to this year’s instructional plan to address this need? • What do you feel worked well or assisted students based on the results? • What is the implication for instruction?
Average Michael Communicating Reasoning Kors Problem Solving/Data Analysis Ralph Concept & Procedures Lauren AL_Math Louis SS_Math Vuitton Research Rene Listening Lacoste Writing Coco Reading Chanel AL_ELA Last. Name First. Name SS_ELA Sample Roster ABC Female middle 08 EO Yes 500 10 2766 4 3 3 2774 4 3 3 3 Male ABC middle 07 EL Yes SLD 500 14 2398 1 1 2 2392 1 1 33333 Male ABC middle 07 EO Yes 700 12 2557 3 1 3 2485 2 1 2 2 44444 Male ABC middle 08 RFEP Yes SLD 400 11 2464 1 1 2 2512 2 1 2 2 Male ABC middle 07 EO No 600 15 2521 2 2 1 3 2 2439 1 1 1 2 2541 2 1. 6 2 1. 8 2. 4 2520 2 1. 4 1. 8 2 SSID 11111 22222 55555 Gender School Grade ELStatus SED Disability Ethnicity Parent. Ed 16
Sample Middle School: ELA Performance by Claim 100% 13% 11% 16% 90% 19% 10% 14% 18% 19% 17% 18% Which ELA claim is highest performing for 8 th grade students? 80% 70% Lowest performing? 47% 45% 60% 48% 69% 47% 64% 48% 67% 52% 54% 57% 60% 50% Above Standard At or Near Standard 40% Below Standard 30% 20% 40% 35% 34% 33% 10% 20% 30% 28% 26% 25% 24% qu iry in g h/ In en st es R R es ea rc rit in g Li rc ea W ng di ea h/ In st Li es R Grade 6 R qu iry in g en g rit in W ng di ea rc ea h/ In st Li R qu iry in g en g rit in W R ea di ng 0% Grade 7 Grade 8 17
Sample Middle School: Math Performance by Claim 100% 90% 15% 10% 12% 15% 11% 14% 11% Which Math claim is highest performing for 6 th grade students? 80% 70% 33% 33% 47% 49% 50% Lowest performing? 53% 60% 52% 67% 50% Above Standard At or Near Standard 40% Below Standard 30% 52% 42% 20% 53% 52% 40% 38% Problem Solving & Modeling/Data Analysis Communicating Reasoning 21% 10% 0% 35% Concepts & Procedures Problem Solving & Modeling/Data Analysis Grade 6 Communicating Reasoning Concepts & Procedures Problem Solving & Modeling/Data Analysis Communicating Reasoning Grade 7 Concepts & Procedures Grade 8 18
Connecting SBAC Claim Scores to Targets and Standards 19
Guiding Questions for Informing Instruction • Do teachers know how to access resources to help them understand targets and standards that make up each claim? • Do teachers have sufficient materials and resources aligned to each claim? • Are learning success criteria articulated for each claim and its targets? • How can teachers use/adapt local formative assessments to gauge overall learning and improvement in the areas of need (e. g. , low performing claims)? • Do teachers have access to sample assessment items aligned to the targets and standards that make up each claim?
Resources for Understanding Targets and Standards for Each Claim 21
KDS’ SBAC Claims, Targets, Standards Alignment http: //www. rcoe. us/educational-services/files/2014/09/SBAC-Claims-Targets-Standard-Alignment-Grade-8 -ELA. pdf 22
KDS’ SBAC Claims, Targets, Standards Alignment http: //www. rcoe. us/educational-services/files/2014/09/SBAC-Claims-Targets-Standard-Alignment-Grade-7 -Math. pdf 23
Claim 1: Reading • Literacy instruction that centers on careful examination of texts • Reading closely and drawing evidence from the text to support inferences and judgments made • Variety of text types (literary and informational) • Students should be able to • Cite the textual evidence that most strongly supports an analysis of what the text says explicitly, as well as inferences drawn from the text. • Determine a theme or central idea of the text and analyze its development over the course of the text. Provide an objective summary of the plot. • Determine the meaning of words and phrases in a text, including figurative and connotative meanings. Analyze the meaning of certain word choices on meaning and tone. • Analyze how differences in the points of view of the characters and the audience create such effects as suspense or humor. • Compare and contrast the structure of two texts and analyze how their differing structures contribute to meaning and style. • Distinguish among fact, opinion and reasoned judgement in a text based on research findings and speculation in the text. • Describe how a text presents information (e. g. , comparatively, sequentially, causally).
Threshold Achievement Level Descriptors –Grade 8 ELA http: //www. smarterbalanced. org/wordpress/wp-content/uploads/2012/11/Smarter-Balanced-ELA-Literacy-ALDs. pdf 25
Sample Items Aligned to Targets 26
Sample Item: MC – Grade 6 ELA “Summer on Wheels” by Gary Soto Claim 1: Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts. Question 1: Bentley feels hurt and upset after falling off his bike. Which of the following sentences from the passage best supports this statement? A. Bentley sat at the kitchen table running an ice cube back and forth across the knot on his forehead. B. The ice cube glided across smooth skin before it jumped up and over the knot. C. Bentley whimpered like the puppy he was. D. And the air left his lungs when he belly flopped Target 1: KEY DETAILS: Given an inference or conclusion, use explicit details and Implicit information from the text to support the inference or conclusion provided. 27
Sample Item: MC – Grade 6 ELA Claim 1: Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts. Target 3: WORD MEANINGS: Determine intended or precise meanings of words, including academic/tier 2 words, domain‐specific (tier 3) words, and words with multiple meanings, based on context, word relationships (e. g. , synonyms), word structure (e. g. , common Greek or Latin roots, affixes), or use of resources (e. g. , dictionary, glossary), with primary focus on the academic vocabulary common to complex texts in all disciplines 28
Sample Item: MS – Grade 8 ELA Claim 1: Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts. Target 7: LANGUAGE USE: Interpret figurative language, literary devices, or connotative meanings of words and phrases used in context and the impact of those word choices on meaning or tone 29
Sample Item: HT – Grade 8 ELA Claim 1: Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts. Target 3: WORD MEANINGS: Determine intended or precise meanings of words, including academic/tier 2 words, domain‐specific (tier 3) words, and words with multiple meanings, based on context, word relationships (e. g. , synonyms), word structure (e. g. , common Greek or Latin roots, affixes), or use of resources (e. g. , dictionary, glossary), with primary focus on the academic vocabulary common to complex texts in all disciplines 30
Sample Item: EBSR – Grade 8 ELA “Ansel Adams, Painting with Light” by Melanie G. Snyder Claim 1: Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts. Target 11. REASONING & EVALUATION: Apply reasoning and a range of textual evidence to justify analyses of author’s presentation of information (author’s line of reasoning, point of view/purpose; relevance of evidence or elaboration to support claims; development or connections among complex concepts, ideas). 31
Sample Item: ST/CR – Grade 6 ELA “Planes on the Brain” Read the article “Planes on the Brain” by Elisabeth Deffner, from Faces Magazine. Claim 1: Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts. Question 1: How does the author emphasize the point that the TAM program was a positive influence on the sisters’ lives? Use details from the text to support your answer. Target 1: KEY DETAILS: Given an inference or conclusion, use explicit details and Implicit information from the text to support the inference or conclusion provided. Target 13: TEXT STRUCTURES & FEATURES: Relate knowledge of text structures or genre-specific features to analyze or integrate information http: //sampleitems. smarterbalanced. org/itempreview/sbac/ELA. htm 32
Sample Item: ST/CR – Grade 6 ELA “Planes on the Brain” Read the article “Planes on the Brain” by Elisabeth Deffner, from Faces Magazine. Claim 1: Students can read closely and analytically to comprehend a range of increasingly complex literary and informational texts. Question 3: What does the author mean by “the sky is no longer the limit”? How does the meaning apply to the Anyadike sisters? Use details from the text to support your response. Target 11. REASONING & EVALUATION: Apply reasoning and a range of textual evidence to justify analyses of author’s presentation of information (author’s line of reasoning, point of view/purpose; relevance of evidence or elaboration to support claims; development or connections among complex concepts, ideas). http: //sampleitems. smarterbalanced. org/itempreview/sbac/ELA. htm 33
Sample Item: WR/CR – Grade 7 ELA Cell Phones in School—Yes or No? Some of the reasons to support cell phones in school are as follows: • Students can take pictures of class projects to e-mail or show to parents. • Students can text-message missed assignments to friends that are absent. • Many cell phones have calculators or Internet access that could be used for assignments. • If students are slow to copy notes from the board, they can take pictures of the missed notes and view them later. • During study halls, students can listen to music through cell phones. • Parents can get in touch with their children and know where they are at all times. • Students can contact parents in case of emergencies. Some of the reasons to forbid cell phones in school are as follows: • Students might send test answers to friends or use the Internet to cheat during an exam. • Students might record teachers or other students without their knowledge. No one wants to be recorded without giving consent. • Cell phones can interrupt classroom activities. • Cell phones can be used to text during class as a way of passing notes and wasting time. Based on what you read in the text, do you think cell phones should be allowed in schools? Using the lists provided in the text, write a paragraph arguing why your position is more reasonable than the opposing position. 34 Claim 2: Students can produce effective and wellgrounded writing for a range of purposes and audiences Target 6. WRITE/REVISE BRIEF TEXTS: Apply a variety of strategies when writing or revising one or more paragraphs of text that express arguments about topics or sources: establishing and supporting a claim, organizing and citing supporting evidence using credible sources, providing appropriate transitional strategies for coherence, appropriate vocabulary, or providing a conclusion appropriate to purpose and audience.
Sample Item: EBSR – Grade 11 ELA Claim 3: Students can employ effective speaking and listening skills for a range of purposes and audiences Target 4: LISTEN/INTERPRET: Analyze, interpret, and use Information delivered orally 35
Sample Item: MC – Grade 6 ELA Claim 4: Students can engage in research/inquiry to investigate topics, and to analyze, integrate, and present information Target 3: EVALUATE INFORMATION/SOURCES: Use reasoning, evaluation, and evidence to assess credibility and accuracy of each source in order to gather and select information to support analysis, reflection, and research. 36
Sample Item: MA – Grade 11 ELA Claim 4: Students can engage in research/inquiry to investigate topics, and to analyze, integrate, and present information. Target 4: USE EVIDENCE: Cite evidence to support analyses, arguments, or critiques. Support claims with logical reasoning and relevant evidence. 37
Sample Item: DD - Grade 7 Math Claim 1: CONCEPTS & PROCEDURES: Students can explain and apply mathematical concepts and carry out mathematical procedures with precision and fluency. Target 1 B: Apply and extend previous understandings of operations with fractions to add, subtract, multiply, and divide rational numbers. . 38
Sample Item: Grade 8 Math (Drag and Drop) Claim 2: Problem Solving: Students can solve a range of well‐posed problems in pure and applied mathematics, making productive use of knowledge and problem‐solving strategies Target 2 A: Apply mathematics to solve well‐posed problems in pure mathematics and arising In everyday life, society, and the workplace. Target 2 B: Select and use appropriate tools strategically. 39
Sample Item: ST Grade 11 Math Claim 3: Communicating Reasoning: Students clearly and precisely construct viable arguments to support their own reasoning and to critique the reasoning of others. Target 3 A: Test propositions or conjectures with specific examples. Target 3 B: Construct, autonomously, chains of reasoning that will justify or refute propositions or conjectures. 40
Sample Item: ST– Grade 11 Math Claim 4: Modeling and Data Analysis: Students can analyze complex, real‐world scenarios and can construct and use mathematical models to interpret and solve problems. Target 4 A: Apply mathematics to solve problems arising in everyday life, society, and the workplace. Target 4 B: Construct, autonomously, chains of reasoning to justify mathematical models used, Interpretations made, and solutions proposed for a complex problem. Target 4 D: Interpret results in the context of a situation. Target 4 F: Identify important quantities in a practical situation and map their relationships (e. g. , using diagrams, two‐way tables, graphs, flowcharts, or formulas). 41
Shannon@Key. Data. Sys. com Ekaterina@Key. Data. Sys. com 42
- Slides: 42