The Effective Use of Well Constructed MultipleChoice Items

  • Slides: 15
Download presentation
The Effective Use of Well. Constructed Multiple-Choice Items in Geoscience Assessments Part 2: Developing

The Effective Use of Well. Constructed Multiple-Choice Items in Geoscience Assessments Part 2: Developing wellconstructed items Mimi Fuhrman, American Institutes for Research

What makes a “Good” Multiple-choice item? Ø Alignment: to content, learning outcomes/goals, cognitive demand,

What makes a “Good” Multiple-choice item? Ø Alignment: to content, learning outcomes/goals, cognitive demand, difficulty Ø Importance: testing trivia is a waste of resources; concentrate on fundamental concepts Ø Clarity: the intent of the task and the meaning of the options must speak for themselves and be interpreted in the same way by all examinees Ø Item construction: well-constructed items can be answered successfully by examinees who have the knowledge or skill you are testing, and NOT by examinees who are lacking the skill or knowledge

Outcomes/goals n Clear outcomes/goals must be stated before items can be developed n An

Outcomes/goals n Clear outcomes/goals must be stated before items can be developed n An item or assessment is of little use if it is not “valid”; if it doesn’t measure what you want it to

Two Items: Same “Topic” The proper order of the divisions of the geologic time

Two Items: Same “Topic” The proper order of the divisions of the geologic time scale from largest to smallest is: A) eon, epoch, period, era B) eon, era, period, epoch C) era, eon, epoch, period D) epoch, period, eon, era Key = b The divisions of the geologic time scale were based primarily on: A) differences in the types of rocks B) differences in sedimentary rock structures C) differences in minerals D) differences in fossils Key = d

This one’s easy! Your choice in options will often determine the difficulty of the

This one’s easy! Your choice in options will often determine the difficulty of the item- the finer the distinction between options, the higher the difficulty.

In order to determine the age of formation, potassium-argon dating is most appropriate for

In order to determine the age of formation, potassium-argon dating is most appropriate for which of the following rock types? A) sandstone B) granite C) marble D) coal A) syenite B) diorite C) monzonite D) rhyolite

Anatomy of a multiple-choice question Which of the following cations is most abundant in

Anatomy of a multiple-choice question Which of the following cations is most abundant in ocean water? Stem O P (A) Na+ (B) Mg 2+ (C) Si 4+ (D) K+ Key T I Distractors O N S

Common Item Faults n Confusing to the examinee ¨Ambiguity ¨Unclear directions ¨Complicated visual stimuli

Common Item Faults n Confusing to the examinee ¨Ambiguity ¨Unclear directions ¨Complicated visual stimuli or syntax ¨Difficult vocabulary ¨Nondirected or nonspecific stem ¨Too much reading ¨Apples and Oranges ¨Illogical sequence of options

Common Item Faults n Giveaways to testwise examinees ¨Break in the grammatical structure ¨Use

Common Item Faults n Giveaways to testwise examinees ¨Break in the grammatical structure ¨Use of specific determiners ¨Presence of key words in stem ¨Conspicuous key ¨Pairs of options ¨Blatantly false options ¨Subsuming options

Presence of key words in stem Large-scale, low-angle overthrusts pose a problem for which

Presence of key words in stem Large-scale, low-angle overthrusts pose a problem for which of the following reasons? (A) Such structures are confined to the margins of eugeosynclinal belts. (B) No comparable structures are recognized in shield areas. (C) The heat created by friction exceeds the stability field of carbonates found in thrust sheets. (D) As shown by the Coulomb friction model, internal friction is a variable dependent on rock parameters. (E) The apparent stress differential in overthrust sheets exceeds the compressive strength in the rocks.

Conspicuous key Translation gliding in minerals during deformation most likely occurs by (A) simultaneous

Conspicuous key Translation gliding in minerals during deformation most likely occurs by (A) simultaneous slip along grain boundaries (B) consecutive movement of dislocations along the translation plane (C) propagation of Griffith cracks (D) twinning (E) microfracturing

Pairs of options A typical vertical sequence of sediment through a point bar would

Pairs of options A typical vertical sequence of sediment through a point bar would show (A) interstratified sand shale (B) thick homogeneous sand (C) an upward gradation from sand to shale (D) an upward gradation from shale to sand (E) thick laminated shale

Subsuming options In order to be useful as a groundwater reservoir, a rock must

Subsuming options In order to be useful as a groundwater reservoir, a rock must be (A) Relatively porous (B) Relatively permeable (C) Both porous and permeable (D) Unsaturated (E) Impermeable

Suggestions for Writing Good Items ¨ The stem poses one problem, clearly and concisely

Suggestions for Writing Good Items ¨ The stem poses one problem, clearly and concisely ¨ Only one correct or best answer ¨ Each option fits with the stem and presents a plausible possibility ¨ Test knowledge of important facts and ideas ¨ Be as brief as possible ¨ Each option is independent of the other options

Guidelines for Options ¨ Plausible to the uninformed ¨ A common misconception ¨ A

Guidelines for Options ¨ Plausible to the uninformed ¨ A common misconception ¨ A critical misunderstanding ¨ No specific determiners: e. g. , always, all, never, must ¨ All about same length ¨ No clues in stem

Creating Effective MultipleChoice Items Designing Items to TestCreating Effective MultipleChoice Items Designing Items to Test
Writing effective multiplechoice questions Use a question formatWriting effective multiplechoice questions Use a question format
CONTENTS ITEMS 1 ITEMS 2 3 ITEMS 4CONTENTS ITEMS 1 ITEMS 2 3 ITEMS 4
CONTENTS ITEMS 1 ITEMS 2 3 ITEMS 4CONTENTS ITEMS 1 ITEMS 2 3 ITEMS 4
CONTENTS ITEMS 1 ITEMS 2 3 ITEMS 4CONTENTS ITEMS 1 ITEMS 2 3 ITEMS 4
CONSTRUCTED RESPONSE BiWeekly Constructed Response Activity 16 WeeksCONSTRUCTED RESPONSE BiWeekly Constructed Response Activity 16 Weeks
RACES CONSTRUCTED RESPONSE TYPES Constructed Response is theRACES CONSTRUCTED RESPONSE TYPES Constructed Response is the
Theme Constructed Response Exemplar Theme Constructed Response RequiredTheme Constructed Response Exemplar Theme Constructed Response Required
MATH MILESTONES INFORMATION CONSTRUCTED RESPONSE Understanding Constructed ResponseMATH MILESTONES INFORMATION CONSTRUCTED RESPONSE Understanding Constructed Response
Background Conclusion Constructed action and constructed dialogue areBackground Conclusion Constructed action and constructed dialogue are
Constructed Response Writing a successful constructed response BasicsConstructed Response Writing a successful constructed response Basics
Constructed Response using RACES Moonrise Constructed Response RConstructed Response using RACES Moonrise Constructed Response R
Teacher Practices in Scoring MultipleChoice Items Interpreting TestTeacher Practices in Scoring MultipleChoice Items Interpreting Test
Are MultipleChoice Items about to Go Extinct 10282010Are MultipleChoice Items about to Go Extinct 10282010