Blueprinting Your Way to HighQuality Assessment Todays Goal

Blueprinting Your Way to High-Quality Assessment

Today’s Goal To create a test blueprint and revise an assessment.

Test Blueprint Define Review Confirm Framework

Using a Blueprint

Building a Test Blueprint Ideally - - - The blueprint is created before the assessment.

Blueprinting an Assessment Realistically- - - We can improve our assessments by creating a blueprint from a current assessment and making adjustments.

1 The Standards Why are you giving this assessment? What are you measuring?

Assessments for Student Growth: Need to assess priority targets (essential skills) Must align all items to standards Need to allow for stretch (low-high ability) Need to be valid & reliable

Identify Learning Priorities Longevity Leverage Levels © 2013. Battelle for Kids. All Rights Reserved. Learning Priorities Ainsworth, L. (2003). Power standards: Identifying the standards that matter the most. Englewood, CO: Lead and Learn Press.

Learning Priorities © 2013. Battelle for Kids. All Rights Reserved. Introduced early Reinforced often Multiple opportunities for learning

Understanding Student Expectations Grades 3 -11 ELA & Math Evidence Tables Performance Level Descriptors PLD Science Grades 5 & 8 HS Physical Science Social Studies Grades 4 & 6 HS American History & Government Other Subjects Test Specifications Content Standards Anchor Standards – College & Career Readiness

Using a Blueprint

Depth of Knowledge How deeply do the students need to engage with the content?

4 Common Misconceptions about DOK

Misconception #1 All kids can’t do this. • Work with others first; DOK 3 -4 are not meant to only be done independently. • Speaking and listening support deeper thinking. • Plan questioning and formative checks from DOK 1 -4 over the course of a lesson/unit. -Plan for differentiation and scaffolding Hess

4. NF. 2 Compare two fractions with different numerators and different denominators, e. g. , by creating common denominators or numerators, or by comparing to a benchmark fraction such as ½. Farmer Fred’s fields are worth twelve hundred dollars total. Each field’s value is based on its size. What fraction of the total value is each field worth? How much is each field worth? Show and explain all of your mathematical thinking. NYC DOE

Misconception #2 Bloom’s = Webb’s DOK It’s what comes after the verb that indicates the complexity of the task. Hess

Description Tasks • Describe the information contained in graphics or data tables in the text; or describe the rule for rounding a number. • Describe how the two characters are alike and different. • Describe the data or text evidence that supports your solution, reasoning, or conclusion. • Describe varying perspectives on global climate change using supporting evidence, and identify the most significant effects it might have on the planet in 100 years. Hess

Analysis Tasks • Analyze Washington’s Farewell Address to explain why it is considered a seminal U. S. document. • Analyze each statement to decide whether it is fact or opinion. • Analyze the data. What do you see as other possible outcomes? • Analyze primary and secondary sources to differentiate between historical fact and historical interpretation.

Misconception #3 Webb’s DOK model is a taxonomy. • Bloom’s is a taxonomy, intended to be a hierarchy. • Webb’s model is not a taxonomy. -It differentiates levels of engagement with content -DOK 3 -4 are not better than DOK 1 -2 Hess

Misconception #4 DOK is about difficulty. • DOK is about complexity, not difficulty • What mental processing must occur? Hess

What it is… Level 2 Level 1 DOK Level 3 Level 4

DOK Level 1 Recall or Reproduction • Recall a fact, term, principle, concept, or perform a routine procedure. • Follow a well-known rule, procedure, or formula. • The answer is either right or wrong. Hess, Karin 2013 CRM

DOK Level 2 Basic Application & Reasoning • Use of information • Select appropriate procedures • Usually multiple steps • Routine problems • Apply 2+ concepts • Make limited decisions • One right answer Hess, Karin 2013 CRM

DOK Level 3 Strategic Thinking • Requires reasoning • Developing a plan to approach a problem • Non-routine • Involves making a claim and providing supporting evidence = “Back it up, Jack!” • Often more than one possible approach or answer Hess, Karin 2013 CRM

DOK Level 4 Extended Thinking – Real World • Requires time to research, problem solve, and process multiple conditions of a task • An original investigation or application to real world • Requires significant conceptual understanding and application of skills across disciplines • Often more than one possible approach or answer • Multiple sources Hess, Karin 2013 CRM

DOK is about complexity— not difficulty! • The intended student learning outcome determines the DOK level. What mental processing must occur? • While verbs may appear to point to a DOK level, it is what comes after the verb that is the best indicator of the rigor/DOK level. • The DOK level represents a “ceiling. ” Karin K. Hess, Ed. D. , Senior Associate Center for Assessment, Dover, NH khess@nciea. org

Using a Blueprint

2 The Assessment Does the assessment provide the BEST evidence of student learning? How do I create an assessment that will allow all students to demonstrate growth?

Assessment Methods Selected Response Constructed Response Performance Assessment

Assessment Methods Selected Response • Multiple Choice • Matching • True/False Constructed Response Performance • Short Answer • Extended Response • Essay • • Product Visual Verbal Physical

Multiple Assessment Methods Students Demonstrate Learning through the Written Word Selected Response Learning is measured through correct or incorrect responses. Constructed Response Students Demonstrate Learning through Performance Product Performance Physical/ Verbal/Visual Performance Learning is measured through rubrics or scoring guides. These methods may involve a variety of responses and/or processes.

Matching Level-Method Selected Response Constructed Response Performance Level 1 – Knowledge/Recall Strong Good Poor Level 2 – Application Good Strong Good Level 3 – Strategic Thinking Poor Strong Level 4 – Extended Thinking Poor Good Strong

Assessment Methods 1. You can use more than one method for each standards. 2. Choose an appropriate method to meet the demands of the standard and are realistic for number of students and time constraints. Learning Expectations Target DOK Level Assessment Method DOK 1 Recall or Reproduce DOK 2 Basic Application of Skills/ Concepts DOK 3 Strategic Thinking DOK 4 Extended Thinking Total Points % points % Describe how characters in a story respond to major events or challenges. 2 SR & CR Ask and answer questions to demonstrate understanding of key details in a text. 1 SR Determine the lesson or moral of a story. 2 CR Identify difference in point of view of different characters. 3 P Compare and contrast two or more versions of the same story. 3 P points % Total Points points 0 points % % % % 0%

Stretch Assessments for the purpose of demonstrating growth need stretch to allow both high and lowachieving students to show evidence of learning.

Two ways to achieve this: Difficulty Who was the 1 st president of the United States? Who was the 32 nd president of the United States? DOK Identify 1 possible source of water contamination? Using data on water quality in two locations, conduct a comparative analysis to quantify the quality of the water.

Hess’ Cognitive Rigor Matrix Bloom’s Taxonomy Webb’s Depth of Knowledge

Karin Hess – Cognitive Matrix

August 2007 Kentucky Department of Education

Other Options to Demonstrate Stretch Include items that assess content from adjacent grade levels (IF you TEACH this content!) common core institute

Creating Stretch Across Grade Levels • 1 st Grade: Describe characters, settings, and major events in a story using key details. • 2 nd Grade: Describe how characters in a story respond to major events and challenges. • 3 rd Grade: Describe characters in a story (e. g. , their traits, motivations, or feelings), and explain how their actions contribute to the sequence of events.

Example of Stretch Writing: Write arguments to support claims in an analysis of substantive topics or texts, using valid reasoning and relevant and sufficient evidence. DOK 1: Given a topic, identify two arguments to support or oppose the topic DOK 2: Identify relevant evidence in a text DOK 3: Analyze two different texts on the same topic to determine the effectiveness of both arguments. Justify your conclusions. DOK 4: Research a topic and write an argumentative essay

The poster seeks to protect America and aid Britain in the struggle against A. Italy B. Germany C. Japan D. China Low complexity – DOK 1

What type of propaganda is used in this poster? A. Name Calling B. Transfer C. Fear D. Testimonial Justify your answer using evidence from the poster and your knowledge of propaganda. Higher complexity – DOK 2

Compare these 3 WWII propaganda posters. Explain the type of propaganda used for each. Use your knowledge of the types of propaganda and specific details from the posters to decide with poster you think would be most effective. High complexity – DOK 3

Other Options to Demonstrate Stretch Include items that assess grade-level content at a greater or lesser difficulty/lexile newsela. com

Levels of Cognitive Complexity 1. Ensure your assessment items meet the depth of knowledge of the standard. 2. Provide an opportunity for stretch in the assessment so all student can show growth. Learning Expectations Target DOK Level Assessment Method DOK 1 Recall or Reproduce DOK 2 Basic Application of Skills/ Concepts DOK 3 Strategic Thinking DOK 4 Extended Thinking Total Points % Describe how characters in a story respond to major events or challenges. 2 SR & CR 2 points 3 points 5 points 20% Ask and answer questions to demonstrate understanding of key details in a text. 1 SR 3 points 12% Determine the lesson or moral of a story. 2 CR 1 point 2 points 3 points 12% Identify difference in point of view of different characters. 3 P 2 points 4 points 6 points 24% Compare and contrast two or more versions of the same story. 3 P 8 points 32% Total Points 6 points 7 points 12 points 0 points % 24% 28% 48% 0%

High Quality Assessment Items Not all test questions are created equal. . . Here a few things to consider. .

3 The Blueprint Use the blueprint to align each assessment item with the standard and Do. K level of the question in order to evaluate the weight and rigor of the assessment.

Reflect on Weight & Rigor Learning Expectations Weight Assessment Target DOK Level Method DOK 1 Recall or Reproduce DOK 2 Basic Application of Skills/ Concepts DOK 3 Strategic Thinking DOK 4 Extended Thinking Total Points % 1. Understand a fraction is part of a whole. 1 sr 4 2 A & 2 B. Represent a fraction on a number line by defining the interval from 0 to 1 at lengths 1/b. 2 sr 1 2 3 points 30 % 3 A. Understand two fractions are equivalent if they are at the same point on a number line. 1 sr 1 1 points 10 % 3 C. Express whole numbers as fractions. 1 sr 1 1 points 10 % 3 D. Compare two fractions and record the results of the comparison as >, =, <. Recognize comparisons are valid only when the two fractions compare to the same whole. 2 sr 1 1 points 10 % Total Points points % % points % 7 points 70 % 4 points 40 % Rigor 3 points 30 %

The Power of the Blueprint

Your Turn • Blueprint your own assessment • Critically analyze the depth of the standards and the depth of the assessment items • Make adjustments for next year Does this assessment provide the BEST evidence of student learning?

education. ohio. gov Shannon Pence NW Assessment Literacy spence@auglaizeesc. org www. livebinders. com Search – Ohio Assessment Literacy What we learn today doesn’t make what we did yesterday wrong, but it can make tomorrow better. – Jan Chappuis