Writing Meaningful Test Questions Kurt Gilliland Ph D

Writing Meaningful Test Questions Kurt Gilliland, Ph. D. Dept. of Cell Biology and Physiology Ryan Madanick, M. D. Division of Gastroenterology and Hepatology Dept. of Medicine

Writing Meaningful Test Questions • Format of stems, lead-ins, and options » What are common flaws to avoid? • Content of stems, lead-ins, and options » How can your items reflect learning objectives? » How can you develop items that test application instead of recall?

1. The primary purpose of the stam is to remove which of the following? A. Carm B. Denton C. Menice D. Stam bar

1. The primary purpose of the stam is to remove which of the following? ✔ A. Carm B. Denton C. Menice D. Stam bar Item Flaw Word repeat: A word or phrase is included in the stem and in the correct answer.

2. Which of the following pairs has won the greatest number of Abby awards? A. Jones and Smith B. Smith and Taylor C. Smith and White D. White and Allen

2. Which of the following pairs has won the greatest number of Abby awards? A. Jones and Smith B. Smith and Taylor ✔ C. Smith and White D. White and Allen Item Flaw Convergence strategy: The correct answer included the most elements in common with other options.

3. How many pounds of pressure are exerted by a callam? A. 260 B. 150 C. 2. 6 D. 2600

3. How many pounds of pressure are exerted by a callam? ✔ Item Flaws A. 260 B. 150 C. 2. 6 D. 2600 Logical cues: A subset of the options are collectively exhaustible. Often, the correct answer is in the middle of the range. Language in the options is not parallel: Options are in an illogical order, in this case.

4. The stanon is aided by a A. anstel B. immon C. octal D. port

4. The stanon is aided by a A. anstel B. immon C. octal ✔ D. port Item Flaw Grammatical cues: One or more distractors don’t follow grammatically from the stem.

5. Why does the stanon frequently overheat? A. All grestels are belious. B. No immons are directly fectitious. C. Octals are usually casable. D. Ports are always actial.

5. Why does the stanon frequently overheat? Item Flaws A. All grestels are belious. B. No immons are directly fectitious. ✔ C. Octals are usually casable. D. Ports are always actial. Frequency terms in the topics are vague: Terms such as “rarely” or “usually” not consistently defined or interpreted, even by experts. Absolute terms: Terms such as “always” or “never” are used in options.

6. Stamation normally occurs when the… A. anstels rupture. B. immon falls and the denton is in place. C. octal rotates easily. D. ports pass over the carm.

6. Stamation normally occurs when the… Item Flaw A. anstels rupture. ✔ B. immon falls and the denton is in place. C. octal rotates easily. D. ports pass over the carm. Long correct answer: Correct answer is longer, more specific, or more complete than other options.

BONUS. Find 4 flaws. Regarding syphilis, all are true except… A. The skin rash of secondary syphilis may be macular or papular and spirochetes may be visualized by the Warthin-Starry stain. B. Nontreponemal serologic tests for syphilis include the FTA and MHA tests; these are positive early in T. pallidum infections but become negative in tertiary syphilis. C. Gummas are granulomatous lesions of tertiary syphilis and show necrotic centers and a surrounding rim of plasma cells. D. Cardiovascular tertiary syphylis is seen in the ascending and or thoracic aorta with aortitis and aneurysm. E. None of the above

BONUS. Find 4 flaws. Item Flaws Regarding syphilis, all are true except… A. The skin rash of secondary syphilis may be macular or papular and spirochetes may be visualized by the Warthin-Starry stain. B. Nontreponemal serologic tests for syphilis include the FTA and MHA tests; these are positive early in T. pallidum infections but become negative in tertiary syphilis. C. Gummas are granulomatous lesions of tertiary syphilis and show necrotic centers and a surrounding rim of plasma cells. D. Cardiovascular tertiary syphylis is seen in the ascending and or thoracic aorta with aortitis and aneurysm. E. None of the above 1. Short stem but long, heterogeneous options. 2. A negative term in the stem. 3. Unfocused stem. 4. Use of “none of the above” as option.

Frequency Terms What do terms like “usually” or “frequently” mean to you?

Frequency Terms

Name That Exam Flaw

The most important prognostic factor in AIDS is… A. CD 4: CD 8 ratio B. presence of Kaposi’s sarcoma C. absolute leukocyte count D. absolute CD 4 count E. level of GP 120 antibody

The most important prognostic factor in AIDS is… A. CD 4: CD 8 ratio B. presence of Kaposi’s sarcoma C. absolute leukocyte count D. absolute CD 4 count E. level of GP 120 antibody Unfocused stem Use of absolute terms such as “most”

What is the cause of muscle soreness that is present the day or two after exercise? A. Accumulation of lactic acid B. Microscopic muscle damage leading to an inflammatory response including the release of bradykinin, PGE 2, and histamine which sensitive nerve endings C. Muscle stiffness D. Use of anabolic steroids

What is the cause of muscle soreness that is present the day or two after exercise? A. Accumulation of lactic acid B. Microscopic muscle damage leading to an inflammatory response including the release of bradykinin, PGE 2, and histamine which sensitive nerve endings C. Muscle stiffness D. Use of anabolic steroids Long distractor Heterogeneous distractors

The tophi of gouty arthritis: A. are a form of erythema nodosum B. have a fibrinoid center surrounded by peripheral histiocytes C. are composed of prominent osteophytes at the DIP joints of fingers D. none of the above

The tophi of gouty arthritis: A. are a form of erythema nodosum B. have a fibrinoid center surrounded by peripheral histiocytes C. are composed of prominent osteophytes at the DIP joints of fingers D. none of the above Unfocused stem Short stem, long distractors “None of the above”

Sigmund Freud’s Interpretation of Dreams was published in what year? A. 1911 B. 1912 C. 1913 D. 1914 E. 1915

Sigmund Freud’s Interpretation of Dreams was published in what year? A. 1911 B. 1912 C. 1913 D. 1914 E. 1915 Correct answer (1913) in middle of range Question is recall of minutia and not application

Aspirin should NOT be prescribed when the patient has which of the following conditions? A. Peptic ulcers B. Headache C. Fever D. Pain E. Inflammation

Aspirin should NOT be prescribed when the patient has which of the following conditions? A. Peptic ulcers B. Headache C. Fever D. Pain E. Inflammation Use of “not” is acceptable in this case since the teaching point was in fact a contraindication.

Which of the following is true regarding lysosomal storage disease? A. 会意字 / 會意字 B. ﻋﺮﺑﻰ / ﺭﺑﻲ C. 日本語 D. ������� ��

Which of the following is true regarding lysosomal storage disease? A. 会意字 / 會意字 B. ﻋﺮﺑﻰ / ﺭﺑﻲ C. 日本語 D. ������� �� Unfocused stem