Chapter 10 Fair or Foul The plays the

Chapter 10: Fair or Foul. . . The play’s the thing! “I'll have grounds more relative than this, the play's the thing wherein I'll catch the conscience of the King. ” – Hamlet, Prince of Denmark Or Catching your own conscience; reflecting on your testing approaches.

Topics on Administering & Analyzing Tests When using them for important summative assessments n Test administration n Item analysis n Cheating

Fairness – Real and Perceived n Important that students perceive value in the assessments n Motivated to do especially well on tests related to n n recorded grades, report cards passing or failing decisions n Three attributes of perceived fairness n Teach what you test. n n Give adequate notice. n n Broadly meant; not just what you said in class. Pop quizzes are okay as long as you let students know they are a possibility. Be fair in grading. n n Consistent and reasonable. Fairness does not mean easy. Many students appreciate high standards and a challenging test.

Suggestions, prior to test day, for. . . Preparing Test Materials 1. Give clear directions – what, if any, supplemental materials are permitted (scratch paper, dictionaries, calculators). 2. 3. Use a simple format – use “white space” and logical page breaks. Allow lead time for copying – don’t race into office and expect your test to be done right away. 4. 5. Spot check – pages complete, in correct order, not inverted. Take one test and answer it – non-fatal mistakes you can correct with oral directions; fatal means recopy (don’t postpone). 6. 7. Worry about security – keep secure; may wish to number copies. Comparable forms for multi-section courses – students in later classes will ask students in earlier sections about the test.

Suggestions, prior to test day, for. . . Preparing Yourself and the Students Assuming you have provided excellent content instruction, recommended actions include: q Give adequate notice to the students. This includes the type(s) of items that will be found on the test. q Schedule the test so that you have time built-in when the test is completed for scoring of the test. q Consider likely circumstances. Are there any special equipment needs? q Is there anything scheduled that could be disruptive (fire drill, pep rally)? q Should you pre-guide students as to what to do if there is unanticipated commotion in the room or building? (What to do with test documents. ) q q Make advance preparation for accommodations. Accommodation is the technical term for alterations in testing conditions related to student disability. Specifically: An accommodation changes a condition (e. g. , same test; more time) q A modification changes the test itself (e. g. , different test) q

Ohio Assessment Accommodations Criteria established in Ohio Administrative Code Rule 3301 -13 -03 To quote from the Accommodations statements found on the Ohio Department of Education (ODE) website: Statewide Assessment Accommodations include a list of common assessment accommodations meant to help IEP teams make appropriate decisions about allowable accommodations for statewide assessment. It is not an exhaustive list. Designations such as "allowable" or "not allowable" for the test subjects/grades and accommodations have been made in light of the four allowable criteria for statewide assessment accommodations explained at the beginning of the list. n To quote from other parts of the ODE statement specific to testing: n “Any accommodation that gives a student with disabilities an unreasonable advantage is not allowable, because it does not allow for valid assumptions to be made from the results. ” n “Accommodations are changes made in how a student has access to the curriculum or demonstrates learning. . In testing situations, accommodations are changes in format, response, environment, timing or scheduling that does not alter in a significant way what the test measures or the comparability of the scores. ” n

Suggestions, on the day of the test, for. . . Immediate Preparation of Students 1. Think about classroom seating arrangement – would rearranging be offensive; how would you rearrange (e. g. every other seat). 2. Determine how materials (e. g. , test, answer sheet) will be passed out (yourself; passed front to back) - check that everyone has all materials. 3. Establish control – students often talk when materials are passed about, decide when to declare conversation time over. 4. Direct students to place names on answer sheet – and on test itself if this is important to you for security reasons. This may suggest to student’s that you look at any marks they make on the test itself. 5. Review test directions – some students skip the written directions while others agonize too long over them (should be no surprises). 6. Provide directions as to what the students do when they finish their test (e. g. , turn in, wait quietly, study). 7. Give quick words of encouragement – humor may work or may backfire.

Suggestions, once the test starts, for. . . Managing the Testing Environment 1. Monitor the testing situation – don’t use this time as personal time to work on other classes; circulate around the room, help students stay on task; see if it appears someone is not correctly following the directions (make sure you tell students you will be doing this). When you do sit, sit behind students out of their vision. 2. Answer questions with fairness in mind – don’t give hints to individual students, clarification only. If it appears what you might be telling them is important, consider telling the entire class. 3. Apprise students of remaining time – if timing is important or if there is no clock visible.

Suggestions, once the test finishes, for. . . Scoring and Returning Test Results 1. Many teachers dislike this part; it is time consuming. The utility of having provided yourself with a previously prepared scoring key and time to score is key for success in this area. I suggest you score exams yourself. 2. Return results promptly; most often the sooner the better. Getting results back quickly is good for the mental health of students, and yourself! 3. Don’t announce individual scores (although you can publicly acknowledge a student’s outstanding performance); don’t post grades (legal issue will be discussed later). You may make general narrative statements (either positive or negative). . . I prefer positive (for example, the class did well). 4. Reiterate the scoring system you used. Your scoring approach should not be a surprise to the students but, by restating it, you demonstrate you tested them exactly as you said you would. 5. Decide how you want to deal with student questions. Will you do over all items, some items? What kinds of questions will you accept, how long will the questions go on? Will you change grades based on the discussions?

Suggestions, once the test review finishes, for. . . Retaining Test Results 1. Recover materials as desired – do this in such a way that students do not see the scores of others. Do you want all parts back? Can students keep old tests for studying? If collected, what will you do with them? 2. Record scores – do this after you have gone over test with students. Use grade book (hardcopy or electronic). 3. Make notes - on changes you would like to make in the assessment system when you teach this in the future. Do this now while it is fresh in your mind.

Classroom Item Analysis. . . Examining the quality of individual items; “p” and “D” statistics n When the test is completed, we have two basic questions: n n Was the difficulty of the item appropriate or was it too hard or too easy? I can compute the item difficulty index by dividing the number of correct answers by total answers: n p = percent correct n The higher the value; the easier the item If the item was difficult, was this difficulty okay? Does it discriminate between those who know and those who don’t? To answer this we compute the item discrimination index. In this case we look at each item in terms of did the students who do the best on this test tend to get it right more so than the students who did the worst. The book divides the class in half (H & L); I tend to like thirds (H, M, L). Compare the percent correct between H & L groups n D = H% - L% n The higher the value, the more the item discriminates. . . notice it go negative or positive. What’s a “+” or “-” mean?

Item Analysis: Example Note: the values in the table are percentages of correct responses, High/Low/Total Group

Classroom Item Analysis, “So What”. . . n Item analysis is important because a poor or bad item may add to the unreliability of the test. Why? Because if your purpose in the test was to distinguish between those students who know the material and those who do not, a bad item fails to work toward this end (e. g. , items answered correctly by all, or items that no one answers correctly. ) n A rough "rule-of-thumb" is that if the item difficulty is more than. 75, it is an easy item; if the difficulty is below. 25, it is a difficult item. n It is my feeling that items with difficulties less than 20% or more than 80% (especially if they fail they discriminate) deserve your attention. Such items should either be revised or replaced. Perhaps you have a non-functioning distracter. Perhaps your question misleads the best students. An exception might be at the beginning of a test where easier items (90% or higher) may be desirable for initial confidence building.

Taking item analysis to the next level. . . Item Analysis Programs If you get into item analysis on a large scale (many students/items) consider: n Independent vender computer software “Test Packages” on test construction which often include item analysis features. These may not be good long term since the companies that produced them come and go (no tech support). n The “Test Development Software Package” provided by the publisher of the textbook you are using. n Test item analysis using the “Microsoft Excel” spreadsheet program. n Your school district may have a centralized test scoring service (usually limited to selected response types of exams). This likely means that you would need to require that your students use a bubble sheet because it would be scanned. These computer programs may use more complex calculations for indices (aka, indexes) related to item difficulty or discrimination but the concept will be the same. For example, item discrimination might be presented in terms of a correlation coefficient (with a Greek letter or an unusual sounding name, of course). Remember how to interpret a correlation coefficient? The scale is “-1 to +1” so you would interpret it much like the “D” statistic.

Cheating Graphic from “A Taxonomy of Text and Exam Fraud” by Dennis Maynes, 2008

Say it ain’t so, Joe. . . Cheating and Academic Integrity Most teachers would rather not discuss this; perhaps a form of denial. n Prevalence n All levels of education n Student View “No big deal – everybody does it. ” n Practices n Plagiarizing, especially Internet-based n Cut and paste without attribution n Other projects completed outside class n Assistance from parents, siblings, family friends, fellow students range from incidental to complete authorship n Cheating on exams, tests (crib notes, look at another’s test, pass notes) n Let’s look at some cheating examples; a major source was § Gary K. Clabaugh & Edward G. Rozycki, Preventing Cheating and Plagiarism, 2 nd Edition (2003) Oreland, PA: New. Foundations Press n

Taking note of “crib” notes. . . So called because of how one “cradles” the small sheets. OLD SCHOOL: n Get into the classroom prior to test and write crib notes on the desktop or the back rest of seat in front. Variation, tape note folder on underside of desktop. n Make sure the crib notes can be easily destroyed. Example, chew gum during exam; if a teacher becomes suspicious eat the piece of paper. Do NOT write on yourself, it is impossible to hide if a teacher questions you. n Print off crib notes in size 6 font about 4” wide by ½” long. Acquire transparent pen (BIC recommended). Roll the paper around the tube of ink and slide it in. n Remove label from a clear beverage, such as Snapple. Put notes on the back of the label, then paste it back in place using transparent glue. During the exam take slow "thoughtful" swigs out of the bottle. n Wear laced up shoes on the day of the exam (with laces remove). Attach your crib notes behind each tongue of the shoe. NEW SCHOOL: q Crib notes on an electronic organizer: a mini tape recorder can be fitted with a tiny earphone that can be hidden by combing hair over the ear; programmable calculators; use of a cell phone in a lavatory.

Other Cheating Gambits. . . n n n n Signal answers among group. Keep an eye out for the rearrangement of items on desktops, coded body movements, and hand signals. A student who has postponed a test or who takes it in a later section, enlists a confederate to smuggle an extra copy of the exam out of the exam room. Fraternity approach. Smuggle a copy of the test from the room at the end of the test, note the answer sequence when the test is reviewed then keep both the test and the answers on file. Student sits for the exam, pretends to be taking the test, but completes nothing. Then they turn in the blank exam and makes sure the instructor notices it has been returned. When the tests are graded and the student gets a zero because they didn't take the test, he or she complains bitterly that their test has been lost. When tests are returned students either alter their answers from wrong to right or fill in answers they deliberately left blank. Then they complained that their test was marked incorrectly. Students fake injury or illness to postpone tests for which they are unprepared. Students show up late, feign surprise and claim that they misunderstood when the test was to be given.

Cheating Suggestions for Prevention 1. Create positive classroom atmosphere - reasonable standards, sensible workloads, keep content interesting, keep upbeat attitude yourself. 2. Talk to students about cheating – define cheating behavior, identify consequences; immediate/long term. Is cheating rewarded in society? 3. Reduce anxiety and uncertainty - be clear and informative about upcoming tests, what will be tested, how. 4. Be vigilant – consider changing seating, walk among the students during testing. 5. When cheating occurs, confront it – teachers’ disregard of cheating is one reason students say they feel so free to do it. 6. Determine if your school has a policy; support it - if the policy is nonfunctional, then help develop new policy.

Practical Advice 1. Use summary checklists regarding each aspect of test administration to remind you of considerations at each phase you value. 2. Learn to do simple item analysis, use occasionally. 3. Take cheating seriously.

Terms/Concepts to Review and Study on Your Own n accommodations n “D” n item analysis n item bank n item difficulty index n item discrimination index n modifications n “p” or “p-value”