Chapter 5 Summative Assessment Traditional Paperand Pencil Tests

Chapter 5 Summative Assessment: Traditional Paper-and -Pencil Tests Frey, Modern Classroom Assessment © 2014 SAGE Publications, Inc.

Objectives The learner will be able to: • Identify the relative frequency with which teachers use the various approaches to classroom assessment • Describe the various formats of paper-and-pencil questions • Apply recommended guidelines for composing traditional paper-and-pencil test items • Analyze the quality of real-world paper-and-pencil assessments Frey, Modern Classroom Assessment © 2014 SAGE Publications, Inc.

Traditional Paper-and-Pencil Assessment: is any question format where there is one pre- determined correct answer can be scored objectively or subjectively can be selection (answer is provided) or supply items (student must supply answer) Frey, Modern Classroom Assessment © 2014 SAGE Publications, Inc.

Classroom Teachers' Use of Traditional Paper-and. Pencil Assessment Studies show that the traditional approach is still the most common approach used by classroom teachers. Summative assessment is much more common than formative. Frey, Modern Classroom Assessment © 2014 SAGE Publications, Inc.

Paper-and-Pencil Tests and Young Children: NOT a Good Fit Instructions are too complex and abstract for preschoolers/kindergarteners. Young children construct knowledge through hands-on experience and not through paper-and-pencil activities. Traditional tests take time and the attention span of young children is short. Frey, Modern Classroom Assessment © 2014 SAGE Publications, Inc.

Common Traditional Paper-and-Pencil Formats Multiple Choice Matching True/False Short Answer Fill-in-the-Blank Frey, Modern Classroom Assessment © 2014 SAGE Publications, Inc.

Table of Specifications (TOS) TOS is a common method of ensuring valid representation across the many questions on a teachermade test TOS is a matrix, organized list, or "blueprint" of the items that will be on the test Can indicate the level of the question (i. e. Bloom's Taxonomy), and the matching instructional objective Frey, Modern Classroom Assessment © 2014 SAGE Publications, Inc.

Table of Specifications (TOS) Purpose: to identify the achievement domains being measured and to ensure that a fair and representative sample of questions appear on the test Allows teacher to construct a test which focuses on the key areas and weights those different areas based on their importance Frey, Modern Classroom Assessment © 2014 SAGE Publications, Inc.

TOS Benefits Students in 2 Ways: Improves the validity of teacher-made tests Can improve student learning - helps ensure that there is a match between what is taught and what is tested; provides the link between teaching and testing Frey, Modern Classroom Assessment © 2014 SAGE Publications, Inc.

The “Rules” • The following guidelines are based on what is recommended by experts. • Real world teachers violate many of these guidelines all the time. • It is possible that teachers are aware of these guidelines but choose not to apply them. • It is likely that teachers who do not apply most item rules because they have not been taught them. Frey, Modern Classroom Assessment © 2014 SAGE Publications, Inc.

Rules that Make Sure Items are not Measuring the Ability to Figure out the Instructions or Clarity in What is Being Asked: "all of the above" should not be an answer option "none of the above" should not be an answer option Negative wording should not be used Answer options should include only one correct answer Stems must be unambiguous and clearly state the problem Items should use appropriate vocabulary In fill-in-the-blank items, a single blank should be used at the end True-false items should have simple structure True-false items should be entirely true or entirely false Frey, Modern Classroom Assessment © 2014 SAGE Publications, Inc.

(Continued) Matching item directions should include basis for match Directions should be included Vague frequency terms should not be used Multiple-choice stems should be complete sentences Complex item formats should not be used Frey, Modern Classroom Assessment © 2014 SAGE Publications, Inc.

Rules That are Designed to Decrease the Chances of Guessing Correctly: All answer options should be plausible In matching, there should be more answer options than stems There should be 3 -5 answer options In matching, answer options should be available more than once For matching, the number of answer options should be less than 7 for elementary For matching, the number of answer options should be less than 17 for secondary Frey, Modern Classroom Assessment © 2014 SAGE Publications, Inc.

Rules Designed to Make the Test-Taking Process as Focused and Free from Distraction as Possible: Answer options should not be longer than the stem All parts of an item or exercise should appear on the same page Answer options should not have repetitive wording Point value of items should be presented Questions using the same format should be together Individual items should be short All items should be numbered Test copies should be clear, readable and not hand-written Stems should be on the left, and answer options on the right Frey, Modern Classroom Assessment © 2014 SAGE Publications, Inc.

Rules Designed to Control for Being “Good at Taking Tests" Order of answer options should be logical or vary Answer options should all be grammatically consistent with stem Specific determiners should not be used Answer options should be homogeneous Correct answer options should not be the longest answer option Items should be independent of each other Stems and examples should not be directly from textbook Answer options should be logically independent of one another There should be an equal number of true and false statements True-false statements should be of equal length Frey, Modern Classroom Assessment © 2014 SAGE Publications, Inc.

There’s a Stat for That! Item Difficulty Index Shows the proportion of students who got an item right Often used to make decisions on whether to use the same question again in the future or to change it in some way Formula: number who got the correct answer total number of students taking the test Frey, Modern Classroom Assessment © 2014 SAGE Publications, Inc.

There’s a Stat for That! Corrected Score Corrected score - has been adjusted for guessing; gives a purer indication of student knowledge Number of Items Right - Number of Items Wrong Avg. Number of Answer Options - 1 Makes it easier to appreciate really high levels of performance that are far removed from chance and to identify students who did not perform better or much better than chance Frey, Modern Classroom Assessment © 2014 SAGE Publications, Inc.

Things to Think About 1. Many states do not require training in classroom assessment as part of teacher education requirements. Why do you think that is? 2. In your experience, which of the Top 40 “Rules” are most commonly ignored? Why? 3. What test-taking strategies have you used as a student? 4. What are the weaknesses of multiple-choice tests? Frey, Modern Classroom Assessment © 2014 SAGE Publications, Inc.

Key Points • Traditional paper-and-pencil assessment is still the most common approach to classroom assessment. • Popular formats for objectively scorable test questions include multiple choice, true/false, fill in the blank, matching, and short answer. • There are dozens of guidelines for composing traditional paper-and-pencil test items. They are reasonable and make sense, but are mostly not research-based. • The traditional tests that teachers make for their own classrooms can be evaluated for quality by, at a minimum, applying the accepted item-writing guidelines. Frey, Modern Classroom Assessment © 2014 SAGE Publications, Inc.