Item Analysis Multiple Choice Questions What makes a

Multiple Choice Questions What makes a good question? ? ?

How can we determine if it is a good question? ? ? Distractor power

Distractor power Strength of each distractor (should be equal) # answering incorrect # of

E. G. Option a is correct n if 9 people choose option b (9/3

Item Difficulty = p (percent of people passing) P = number answering correctly total

Item Difficulty is. . . A behavioral measure n Based on the current group

Item discrimination = d Does the question discriminate between those who do well overall

Distribution To determine level upper (U) = top 25 to 33 % of scores

Determining item discrimination d = U-L n where d = item discrimination U =

Value of d Range for d = -1 to 1 n higher d indicates

Examples of d With 33 in U and L n if all in U

Chapter 7 Test Development process – p 191 test conceptualization test construction test tryout

Answer types (table 7 -1) Likert scale (rating scale) n Forced choice (paired comparisons)

Slides: 15

Download presentation

Item Analysis

Multiple Choice Questions What makes a good question? ? ?

How can we determine if it is a good question? ? ? Distractor power n Item difficulty n Item discrimination n

Distractor power Strength of each distractor (should be equal) # answering incorrect # of distractors

E. G. Option a is correct n if 9 people choose option b (9/3 = 3) n and 12 people choose option c (12/3 = 4) n and 15 people choose option d (15/3 = 5) n then option d is the strongest distractor n

Item Difficulty = p (percent of people passing) P = number answering correctly total number answering n If 100 people take the test and 34 get question 1 correct, what is the difficulty of that question? P 1 = 34/100 =. 34 n For 65 correct p would be 65/100 =. 65 SO… the higher the p, the easier the question (score must be between 0 and 1)

Item Difficulty is. . . A behavioral measure n Based on the current group n Allows external comparison n

Item discrimination = d Does the question discriminate between those who do well overall and those who do poorly?

Distribution To determine level upper (U) = top 25 to 33 % of scores lower (L) = bottom 25 to 33 % of scores

Determining item discrimination d = U-L n where d = item discrimination U = upper group who pass L = lower group who pass n = number in one group (U or L)

Value of d Range for d = -1 to 1 n higher d indicates more discriminating (1 = all in upper group got it correct and none in lower group) n negative d indicates a really bad question (1 indicates all in lower group got it correct and none in upper group) n

Examples of d With 33 in U and L n if all in U get it correct, and none in L 33 - 0 / 33 = 1 n if none in U get it correct, and all in U 0 - 33 / 33 = -1 n if all in U and all in L get it correct 33 - 33 / 33 = 0 n if 30 in U and 10 in L get it correct 30 - 10 / 33 =. 61 n

Chapter 7 Test Development process – p 191 test conceptualization test construction test tryout analysis revision

Answer types (table 7 -1) Likert scale (rating scale) n Forced choice (paired comparisons) n Matching n True/false n Multiple choice n Open ended n