Michigan Assessment Consortium Common Assessment Development Series Module

Michigan Assessment Consortium Common Assessment Development Series Module 14 Detecting and Eliminating Bias in Assessment Items

Narrated By: Julie Mc. Daniel Oakland Schools 2

Previous Modules § You learned how to: § Design your common assessment § How to develop items for that assessment § How to assure that your items are of the highest quality 3

This Module § § § Begin an initial step in the editing process Understand bias Learn some ways to detect and to eliminate bias 4

Bias Removal § Bias is the extent to which student performance is misrepresented or inaccurately represented due to pollution in the items § By minimizing bias, then you will be able to determine what students really know and are able to do 5

Example of Bias § English language learner and mathematics assessment 6

Bias Removal Process § One effective way for you to do this is for a group of teachers in your school to review the items you developed or selected looking for content that would favor (or disadvantage) any learner 7

Sources of Bias § Stiggins has identified three sources of bias: § Those within the learner § Those within the context of the classroom § Those found within the assessment itself 8

Bias Within the Learner § Examples include: § A student in poor health § A student with a physical challenge § An English language learner § A student with real life issues outside the classroom § You should take into account these things within the learner that may prevent an accurate demonstration of knowledge 9

Bias Within the Classroom § This type of bias is called classroom or bias in the assessment context § Poor lighting and noise interruption § Lack of learning-conducive environments § Low degree of trust 10

Bias Within the Assessment § This bias exists within the assessment itself § This bias prevents students from demonstrating full understanding § Examples include § Vague or missing directions § Poorly worded or confusing questions § Cultural insensitivity 11

Bias by Assessment Method § There are sources of bias unique to different types of items § Multiple-choice items: § Amount of reading § Size and type of font § Reading difficulty of the questions § Use of separate test booklet and answer sheet § Number of test items used 12

Bias by Assessment Method § There are sources of bias unique to different types of items § Constructed-response items: § Students with poor reading and/or writing skills § Teachers who are unfamiliar a common rubric § Teachers who are inadequately trained § Students who do not know how their responses will be evaluated 13

Bias by Assessment Method § There are sources of bias unique to different types of items § Performance assessments: § Bias within the task itself § Inconsistent assessment administration § Inadequately trained scorers 14

Steps to Avoid Bias in Performance Assessment § Familiarize students with the tasks before the assessment § Pilot the assessment § Develop high-quality rubrics and scoring procedures § Provide safe assessment environment 15

Reviewing Items for Bias § All assessments, especially teacher-constructed assessments, should be reviewed § Identify potential biases § Correct the test or the testing situation § Follow standards of assessment quality to identify and correct bias in assessments 16

Group Content Reviews § Groups of teachers are more effective in detecting bias than individuals: § More accurate in identifying distortion § More efficient and effective in groups § More powerful when done as a professional community (can learn from others how to identify and eliminate biased items) 17

Summary § Bias in assessments prevents students from demonstrating what they know and can do § Bias can occur in types of assessments § Group bias reviews are the most effective detection method § Critiquing assessments for bias an important step in editing 18

Series Developers § § § § Kathy Dewsbury White, Ingham ISD Bruce Fay, Wayne RESA Jim Gullen, Oakland Schools Julie Mc. Daniel, Oakland Schools Edward Roeber, MSU Ellen Vorenkamp, Wayne RESA Kim Young, Ionia County ISD/MDE 19

Development Support for the Assessment Series § The MAC Common Assessment Development Series is funded in part by the Michigan Association of Intermediate School Administrators § In cooperation with § Michigan Department of Education § Ingham and Ionia ISDs, Oakland Schools, and Wayne RESA § Michigan State University 20