1 HISTORY OF MULTIPLE CHOICE ITEMS Multiple choice

1

HISTORY OF MULTIPLE CHOICE ITEMS -Multiple choice items/ questions have attained the status of main testing format at global level -A form of assessment where respondents are asked to select one or more choices -Used in educational testing, market research & in elections - Frederick J. Kelly used it in 1915 for educational assessment in Kansas School -Used to judge IQ of Military recruits during WW-1 2

ANATOMY OF A MULTIPLE-CHOICE ITEM A standard multiple-choice test item consists of two basic parts: A problem (stem) - in the form of either a question or an incomplete statement A list of suggested solutions (alternatives) - contains one correct or best alternative (answer) and a number of incorrect or inferior alternatives (distractors). The distracters must appear as implausible solutions for those students who have achieved the objective The answer should appear plausible to these students 3

STATE THE PROBLEM IN THE STEM THE ITEM Multiple-Choice Item May have several correct answers Consists of a stem and some options Always measure factual details The Components of a Multiple-Choice Item are Stem and several foils Correct answer and several foils Stem , A correct answer 5

STATE THE PROBLEM IN THE STEM THE ITEM The Most Serious Aspect of the Energy Crisis is the Possible lack of fuel for industry Possibility of widespread unemployment Threat to our environment from pollution Possible increase in inflation Cost of developing alternative sources of energy 6

According to Professor Koenig, The Most Serious Aspect of the Energy Crisis is the Possible lack of fuel for industry Possibility of widespread unemployment Threat to our environment from pollution Possible increase in inflation Cost of developing alternative sources of energy 7

MULTIPLE-CHOICE TESTS A multiple-choice test usually has dozens of questions or "items. " For each question, the test- taker is supposed to select the "best" choice among a set of four or five options. (They are sometime called "selected-response tests. ") For example: What causes night and day? The earth spins on its axis The earth moves around the sun Clouds block out the sun's light The earth moves into and out of the sun's shadow. The sun goes around the earth. The "wanted" answer is "A. " The other answer options are called "distractors. " 8

SALIENT OF MULTIPLE-CHOICE TESTS Are multiple-choice tests "objective"? What can multiple-choice items be used for? Multiple-choice and critical thinking Informing instruction 9

DANGERS OF RELYING ON MULTIPLECHOICE TESTS Because of cultural assumptions and biases, the tests may be inaccurate. Assuming the test is accurate because of its supposedly "objective" format may lead to making bad decisions about how best to teach a student Students may recognize or know facts or procedures well enough to score high on the test, but not be able to think about the subject or apply knowledge, even though being able to think and apply is essential to "knowing" any subject 10

DANGERS OF RELYING ON MULTIPLECHOICE TESTS What is easily measurable may not be as important as what is not measurable or is more difficult to measure. A major impact on curriculum and instruction -- is that only things that are easily measured are taught Since the questions usually must be answered quickly and have only one correct answer, students learn that problems for which a single answer cannot be chosen quickly are not important 11

DANGERS OF RELYING ON MULTIPLECHOICE TESTS When institutions view multiple-choice tests as important, they often narrow their curriculum to cover only what is on the exams. For example, to prepare for multiple-choice tests, curriculum may focus on memorizing definitions and recognizing (naming) concepts. This will not lead students to understand important scientific principles, grasp how science is done, and think about how science affects their lives When narrow tests define important learning, instruction often gets reduced to "drill and kill" - - lots of practice on questions that look just like the test. In this case, students often get no chance to read real books, to ask their own questions, to have discussions, to challenge texts, to conduct experiments, to write extended papers, to explore new ideas -- that is, to think about and really learn a subject 12

SHOULD MULTIPLE-CHOICE TESTS BE USED AT ALL? FAVOURABLE FACTORS What the purpose of the test is What uses that will be made of its results To check on factual and procedural knowledge Test will not have a major effect on overall curriculum and instruction Conclusions about what students know in a subject will not be reduced to what the test measures 13

DANGEROUS FACTORS Once it substantially control curriculum or instruction Become as the basis of major conclusions that are reported to the public Once used to make important decisions about students 14

GUIDELINES FOR CONSTRUCTING MULTIPLE-CHOICE ITEMS Construct each item to assess a single written objective Base each item on a specific problem stated clearly in the stem Example Which of the following was the principal keyboard instrument in 16 th century Europe? – Clavichord – Harpsichord – Organ – Pianoforte The principal keyboard instrument in 16 th century Europe was the: – Clavichord – Harpsichord – Organ – Pianoforte 15

GUIDELINES FOR CONSTRUCTING MULTIPLE-CHOICE ITEMS Include as much of the item as possible in the stem, but do not include irrelevant material State the stem in positive form (in general) Word the alternatives clearly and concisely Keep the alternatives mutually exclusive Keep the alternatives homogeneous in content Keep the alternatives free from clues as to which response is correct Keep the grammar of each alternative consistent with the stem Keep the alternatives parallel in form Keep the alternatives similar in length 16

GUIDELINES FOR CONSTRUCTING MULTIPLE-CHOICE ITEMS Avoid the use of specific determiners Avoid including keywords in the alternatives Use plausible distractors Avoid the alternatives “all of the above” and “none of the above” (in general) Use as many functional distractors as are feasible Include one and only one correct or clearly best answer in each item Present the answer in each of the alternative positions approximately an equal number of times, in a random order Lay out the items in a clear and consistent manner Use proper grammar, punctuation, and spelling Avoid using unnecessarily difficult vocabulary Analyze the effectiveness of each item after each administration 17 of the test

WEAKNESSES OF POORLY WRITTEN MULTIPLE-CHOICE TEST QUESTIONS They attempt to measure an objective for which they are not well -suited They contain clues to the correct answer They are worded ambiguously – Distinguish between objectives which can be appropriately assessed by using multiplechoice items and objectives which would be better assessed by some other means – Evaluate existing multiple-choice items by using commonly-accepted criteria to identify specific flaws in the items – Improve poorly-written multiple-choice items by correcting the flaws they contain – Construct well-written multiple-choice items that measure given objectives 18

EXAMPLE What is chiefly responsible for the increase in the average length of life in the USA during the last fifty years? Compulsory health and physical education courses in public schools The reduced death rate among infants and young children The safety movement, which has greatly reduced the number of deaths from accidents The substitution of machines for human labor 19

ADVANTAGES OF MULTIPLE-CHOICE ITEMS Advantages Versatility Analyze phenomena Apply principles to new situations Comprehend concepts and principles Discriminate between fact and opinion Interpret cause-and-effect relationships Interpret charts and graphs Judge the relevance of information Make inferences from given data Solve problems 20

LIMITATIONS OF MULTIPLE-CHOICE ITEMS Limitations Versatility Articulate explanations Display thought processes Furnish information Organize personal thoughts Perform a specific task Produce original ideas Provide examples 21

LIMITATIONS OF MULTIPLE-CHOICE ITEMS Such learning outcomes are better measured by short answer or essay questions, or by. Performance tests. Reliability. Although they are less susceptible to guessing than are true false-test items, multiple-choice items are still affected to a certain extent Difficulty of Construction. Good multiple-choice test items are generally more difficult and time-consuming to write than other types of test items 22

DECIDING WHEN MULTIPLE-CHOICE ITEMS SHOULD BE USED If the attainment of the educational objective can be better measured by having the student supply his response, a short-answer item or essay question may be appropriate If there are several homogeneous test items, it may be possible to combine them into a single matching item for more efficient use of testing time If the attainment of the objective can be better measured by having the student do something, a performance test should be considered 23

MEASURING HIGHER-LEVEL OBJECTIVES WITH MULTIPLE-CHOICE ITEMS One of the reasons why some teachers dislike multiplechoice items is that they believe these items are only good for measuring simple recall of facts. This misconception is understandable, because multiple-choice items are frequently used to measure lower-level objectives, such as those based on knowledge of terms, facts, methods, and principles. The real value of multiple-choice items, however, is their applicability in measuring higher-level objectives, such as those based in comprehension, application, and analysis 24

VARIETIES OF MULTIPLE-CHOICE ITEMS Single Correct Answer Best Answer Negative Multiple Response Combined Response 25

Select Diagnostic Foils or Distracters such as— Clichés Common misinformation Logical Misinterpretation Partial Answers Technical Terms or textbook Jargon A Terminal may be Defined as A final stage in a computer program The place where a computer is kept AN input –out put device used when much interaction is required An auxiliary memory unit A slow but simple operating system 26

What Type of Validity is Determined by Correlating Scores on a test with Scores on a Criterion Measured at a later date? Concurrent Construct Content Predictive A test which can be scored by a Clerk Untrained in the Content Area of the test, Is said to be Diagnostic Criterion-referenced Objective Reliable Subjective 27

A test which can be scored by a Clerk Untrained in the Content Area of the test, Is said to be Diagnostic Criterion-referenced Objective Reliable Subjective A Random Sample is one in which Subjects are selected by levels 28

A Random Sample is one in which Subject are selected by levels in proportion to the number it each level in the population Each subject has an equal probability of being chosen Every nth subject is chosen from a list Groups, rather than individuals, are the unit of analysis The Major Purpose of Item Analysis is to Determine the distribution of test scores Analyze the patterns of examinee responses Determine whether the test content was appropriate Evaluate the overall difficulty of the test 29

THANKS 30