Writing effective multiplechoice questions Use a question format

Writing effective multiple-choice questions

Use a question format or a blank at the end of the stem. Poor Item Improved Item It takes approximately ______ for an earthquake P-wave to travel the first 6500 kilometers after the earthquake occurs. How long does it take for an earthquake P-wave to travel the first 6500 kilometers after the earthquake occurs? a. 6. 5 minutes b. 8. 0 minutes c. 10. 0 minutes d. 18. 5 minutes

Avoid absolute descriptors (e. g. , always and never). Poor Item Improved Item If you find someone lying on the ground face-down and unconscious, you should If you find someone lying on the ground face-down and unconscious, what is the first thing you should do? a. only assist them if there’s someone else available to assist you. b. call 911. c. always check the pulse. d. never roll the person over. a. assist them, if there’s someone else available to assist you. b. call 911. c. check the pulse. d. roll the person over.

v. It is a dead giveaway when all the distractors, save the “key”, have absolutes as part of the question.

Present most of the content in the stem rather than in the options; but stem should be clear and focused. Poor Item Improved Item Pennsylvania What is the nickname for the state of Pennsylvania? a. is nicknamed the Empire State. b. borders 6 other states. c. was the birthplace of two U. S. presidents. d. has a mountain that is 3500 feet above sea level. a. the Empire State b. the Keystone State c. the Sunshine State d. the Show Me State

v. Focus the students’ attention on the information you want them to actually know.

Word the stem positively, not negatively. Poor Item Improved Item Which of the following is not a common trait of invasive species: Which of the following is NOT a common trait of invasive species: a. Tolerance for a range of environmental conditions b. Association with humans c. Localized reproduction d. Fast growth

Correct answer: B v We tend to teach what IS, not what isn’t. Students are accustomed to learning what IS. This format is hard on the reader; does not transfer well from the way information was learned/presented. The example presented here is the way to highlight the item if you can’t change it. The better strategy is the make the statement positive, make the key (correct answer) a true response, and create plausible distractors around it.

Write options that are independent, not overlapping. Poor Item Improved Item How long does an annual plant generally live? a. It dies after the first year. b. It lives for many years. c. It lives for more than one year. d. It needs to be replanted each year. Taken from: Burton, Sudweeks, Merrill, & Wood, 1991. a. 1 year b. 2 years c. 3 years d. It blooms without replanting each year for up to 10 years.

Order options logically. Poor Item Improved Item How many different wing types did the Wright Brothers test before they settled on their successful design? a. 2 b. 200 c. 18 d. 150 a. 2 b. 18 c. 150 d. 200

v Order options: Chronologically Numerically Alphabetically Less cognitive load on the test taker and for someone who knows the answer, there’s no reason to complicate the ordering.

Avoid options such as ‘All of the Above, ’ ‘None of the Above, ’ and Type K options. Poor Item Improved Item Julie has 50 jelly beans in a bag and 20% of the jelly beans are red. How many of the jelly beans in Julie’s bag are red? a. 5 b. 10 c. 25 d. None of the above a. 5 b. 10 c. 25 d. 35

v For “All of the above” questions—if test takers know two options are correct, then they know the answer v For “None of the above” questions—If test takers know even one option is correct, they can eliminate “None of the above” as an option. In either case, it’s a partial knowledge issue v Type K (choose several options) questions—Less discriminating, VERY cognitive demanding but hasn’t been substantiated that they are truly “more difficult”. They mainly take a long time to write and take.

Keep options as homogeneous as possible, in parallel form, and consistent with the stem. Poor Item Improved Item Mark Twain was an _____. Mark Twain was a/an _____. a. Navy pilot. b. chemist. c. 19 th century architect. d. American author. a. African author. b. European author. c. Australian author. d. American author.

Present plausible distracters. Poor Item Improved Item Which American city boasts the nation’s largest city park system? a. Pittsburgh b. Missouri c. Dallas d. Denver a. Pittsburgh b. Kansas City c. Dallas d. Denver

Keep distracter (and key) lengths similar. Poor Item In the Mediterranean region, cattle raising is handicapped by: a. Hot climate b. Low beef demand c. A lack of sufficient productive pasturage d. High labor costs Improved Item In the Mediterranean region, cattle raising is handicapped by: a. Challenges of a hot climate b. Low beef demand among consumers c. Insufficient productive pasturage d. High labor costs for ranchers

Provide only 1 ‘correct’ answer. Poor Item Improved Item Consider the following equation: 13 + 9 + 11 + 7 4 = 10 The answer, 10, is referred to as the a. mean b. variance c. average d. range 13 + 9 + 11 + 7 4 = 10 The answer, 10, is referred to as the a. mean b. variance c. standard deviation d. range

References Haladyna, T. M. , Downing, S. M. , & Rodriguez, M. C. (2002). A review of multiple-choice item-writing guidelines for classroom assessment. Applied Measurement in Education, 15(3), 309 -333. Notar, C. E. , Zuelke, D. C. , Wilson, J. D. , & Yunker, B. D. (2004). The table of specifications: Insuring accountability in teacher made tests. Journal of Instructional Psychology, 31(2), 115 -129. Rohrer, D. , Taylor, K. , & Sholar, B. (2010). Tests enhance the transfer of learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36, 233‑ 239.