Multiple Choice Tests Pick Answer C and You

Multiple Choice Tests: Pick Answer C and You Will Be Right About 25% of the Time Warren J. White Thanks to Neil Salkind for the Title.

Agenda • Structured vs. Unstructured Tests • General Quality Standards of Multiple Choice Tests • Bloom and His Taxonomy • The Table of Specifications • Analytics for Multiple Choice Tests

Structured Tests Have limited number of response options. True-False Multiple Choice Matching Fill-in-the-blanks

Unstructured Tests Have a wider variety of response options controlled by test taker. Technical writing Oral presentation Procedural demonstration Case study Portfolio

Pluses for Structured Response Tests Comprehensive knowledge assessed Scoring economical and speedy Moderate to high reliability Amenable to statistical analysis Amenable to collection of comparable and trend data

Minuses for Structured Tests Items laborious to construct Higher order thinking skill items even more difficult to construct Test security a requirement

Pluses for Unstructured Response Tests • Higher order thinking more easily assessed • Moderate to high authenticity for “real” life tasks • Requires greater student activity and engagement • Minimal influence of guessing on performance • Ease of construction

Minuses for Unstructured Response Tests • Need scoring key/rubric • Scoring requires significant time • Pre-calibration of evaluators needed to increase reliability • More difficult to assess broad range of knowledge quickly • Copmparative and trend data harder to collect

Focus on Multiple Choice Tests • Commonly used Lange classes High stakes testing Item analysis highly developed Facilitates systemic item improvement

General Terminology • Stem-sets the premise for the question • Correct alternative • Distractors-incorrect alternatives

VERY BASIC Guidelines • Handout

Characteristics of Multiple Choice Items that Measure Higher Order Thinking • Difficult to construct-must develop context • Require lots of context-reading selections, scenarios, vignettes, tables, charts, graphs • Require more testing time • Should be reviewed by others

Analytics for Multiple Choice Tests • Item difficulty Index • Item discrimination index • Distracter Analysis