Comparing Decision Rules Decision accuracy of different decision
- Slides: 7
Comparing Decision Rules Decision accuracy of different decision rules combining multiple tests in a higher educational context Iris Yocarini, Samantha Bouwmeester, Guus Smeets, & Lidia Arends 18/03/2016
Decision accuracy of BSA decision • Comparison decision based on true score vs. observed score Decision based on true score Negative BSA Decision based on observed score • Positive BSA Negative BSA Correct classification Misclassification False negative Positive BSA Misclassification False positive Correct classificatie Compensatory vs. conjunctive decision rule • Evaluation argument pro compensation: ‘average is more reliable’ correct decision
Results Compensatory vs. conjunctive rule • Compensatory lower proportion misclassifications Decision Rule 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 GPA Minimum 5. 5 1 5. 5 3 5. 5 4 5. 5 6 1 6 3 6 4 6 5 6 6 6. 5 1 6. 5 3 6. 5 4 6. 5 5 6. 5 Mean Proportion Error 0. 06 0. 10 0. 17 0. 26 0. 24 0. 15 0. 18 0. 25 0. 17 0. 23 0. 22 0. 23 0. 10
Results Compensatory vs. conjunctive rule • Compensatory lower proportion misclassifications Decision Rule 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 GP A Minimum 5. 5 1 5. 5 3 5. 5 4 5. 5 6 1 6 3 6 4 6 5 6 6 6. 5 1 6. 5 3 6. 5 4 6. 5 5 6. 5 Mean Proportion Error 0. 06 0. 10 0. 17 0. 26 0. 24 0. 15 0. 18 0. 25 0. 17 0. 23 0. 22 0. 23 0. 10 Sample Size True Positive BSA 92% 88% 61% 40% 78% 77% 60% 21% 53% 53% 48% 9%
Results Compensatory vs. conjunctive rule • • Compensatory higher sensitivity rate lower false negative rate Compensatory lower specificity rate higher false positive rate Decision Rule 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 GPA Minimum 5. 5 1 5. 5 3 5. 5 4 5. 5 6 1 6 3 6 4 6 5 6 6 6. 5 1 6. 5 3 6. 5 4 6. 5 5 6. 5 Mean Proportion Mean Error Sensitivity Specificity 0. 06 0. 98 0. 38 0. 10 0. 93 0. 51 0. 17 0. 86 0. 59 0. 26 0. 74 0. 71 0. 24 0. 69 0. 79 0. 14 0. 97 0. 45 0. 15 0. 94 0. 52 0. 18 0. 88 0. 59 0. 25 0. 71 0. 17 0. 63 0. 86 0. 23 0. 95 0. 56 0. 22 0. 94 0. 58 0. 22 0. 90 0. 62 0. 23 0. 80 0. 72 0. 10 0. 58 0. 93
Conclusion & Discussion • Decision accuracy important consideration • Compensatory decision rule: relatively fewer false negatives and more false positives • depends on specific setting & tests used • Focus on both specific decision rule as well as selected tests
Thank you for your attention! Questions? Contact: yocarini@fsw. eur. nl
