Techniques for Preventing Response Bias Write questions that

Techniques for Preventing Response Bias Write questions that are clear, precise, and relatively short. Do not use “loaded” or “leading” questions. Avoid double-barreled questions. Avoid double negatives. Avoid confusing questions.

Techniques for Preventing Response Bias Write questions that are clear, precise, and relatively short Because every question is measuring something, it is important for each to be clear and precise. Your goal is for each respondent to interpret the meaning of each survey question in exactly the same way. If your respondents are not clear on what is being asked in a question, their responses may result in data that cannot or should not be applied to your survey goals. Keep questions short; long questions can be confusing and stressful for respondents

Techniques for Preventing Response Bias Do not use “loaded” or “leading” questions A loaded or leading question biases the response given by the participant. A loaded question is one that contains loaded words. For example, politicians often avoid the loaded word “environmentalist” because it creates a negative reaction in some people regardless of the content of the statement. A leading question is phrased in such a way that suggests to the respondent that the researcher expects a certain answer.

Techniques for Preventing Response Bias Do not use “loaded” or “leading” questions Example Don’t you agree that social workers should earn more money than they currently earn? A. Yes, they should earn more B. No, they should not earn more C. Don’t know/no opinion The phrase “Don’t you agree” leads the respondent. A more neutral wording would be: Do you believe social worker salaries are a little lower than they should be, a little higher than they should be, or about right? A. Social worker salaries are a little lower than they should be B. Social worker salaries are a little higher than they should be C. Social worker salaries are about right D. Don’t know/no opinion

Techniques for Preventing Response Bias Avoid double-barreled questions Do you think professors should have more contact with university staff and university administrators? Clearly, this question asks about two different issues: Do you think professors should have more contact with university staff? AND Do you think that teachers should have more contact with university administrators? Combining the two questions into one question makes it unclear which attitude is being measured, as each question may elicit a different attitude. Tip: If the word “and” appears in a question, check to verify whether it is a double -barreled question.

Techniques for Preventing Response Bias Avoid double negatives When respondents are asked for their agreement with a statement, double negatives can occur. Do you agree or disagree with the following statement? Teachers should not be required to supervise their students during recess. If the respondent disagrees, you are saying you do not think teachers should not supervise students. In other words, you believe that teachers should supervise students. If you do use a negative word like “not”, consider highlighting the word by underlining or bolding it to catch the respondent’s attention.

Techniques for Preventing Response Bias Avoid confusing questions. Any question that causes an analyst or reader to say, "huh? " probably had the same impact on the respondent. Does it seem possible or does it seem impossible to you that the Nazi extermination of the Jews never happened? What is the question? Even if the respondent understood the question, would they understand how to answer? What does a reply of 'yes' mean? Needless to say, this question did not yield reliable data and the survey firm recognizing this, went back into the field with a less confusing question.

Examples of “biased” questions Example 1: Bob asked David “You don’t like this pair of jeans, Do you? ” It’s clear that “NO” is the response expected from David. Instead, Bob could revise his question as follows: “Do you like this pair of jeans? ” Example 2: Sally asked Sarah “Don’t you agree that the new rule is a problem? ” Sally’s question is biased. The question leads Sarah to agree with Sally’s view. Sally has in fact phrased her opinion in the form of a question. Instead Sally could ask the following question: “Do you agree or disagree that the new rule is a problem? ”

Examples of “biased” questions In a survey, the questions should NOT be designed to favor certain outcomes. Example 1: The question “Do you want to eat a hamburger or the usual vegetable sandwich? ” is unfair, because it favors hamburger over vegetable sandwich. Usually companies use biased questions in their advertisements or marketing surveys to make people favor their products over others. Example 2: The following is a biased question posed by XYZ Beauty Company.

Examples of “biased” questions More people in the City are using our beauty products than any other brand. Do you use our beauty products? A. Yes B. No Clearly the question indicates that the respondent should be using XYZ beauty products.

Examples of “biased” questions Example 2 A company manufactures product A. The company conducts a survey about the product. The following is one of the questions in the questionnaire. How would you rate our product? A. Excellent B. Good C. Satisfactory The question is biased, because, NO negative option is provided.

Examples of “biased” questions A biased question makes assumptions that may or may not be true. For example: The question “Is green your favorite color? ” is asked based on an assumption. The person to whom this question is asked may or may not like green color. Is the following question biased? Say yes or no. ‘Do you watch movies directed by Steven Spielberg’? No, the given question is not biased, because neither does it favor one answer over others nor does it make any assumption.

Examples of “biased” questions Categories are mutually exclusive when there is no overlap: Example What is your current age? A. 10 or less B. 10 to 20 C. 20 to 30 D. 30 to 40 E. 40 to 50 F. 50 or greater These categories are not mutually exclusive because there is overlap present. For example, a person who is 20 years old could be placed into two separate categories (same with those respondents aged 30, 40 and 50).

Examples of “biased” questions Categories are exhaustive when there is a category available to all potential responses. Below is an example of a question where categories are not exhaustive: Example What is your current age? A. 1 to 4 B. 5 to 9 C. 10 to 14 The categories are not exhaustive because there is no category available for respondents more than fourteen years old or respondents less than one year old.

Examples of “biased” questions Here is an example of response categories that are both mutually exclusive and exhaustive: What is your current age? (Choose only. ) A. B. C. D. E. Less than 18 18 to 29 30 to 39 40 to 49 50 or older

Examples of “biased” questions One technique used to prevent response sets is to reverse the wording in some of the survey items. Below is an example of this in a rating scale question: Please rate your manager on each of the following descriptive scales. Place a checkmark on the space between each pair of words that best indicates your opinion: Sociable 1 2 3 4 5 Unsociable Kind 1 2 3 4 5 Cruel *Hard 1 2 3 4 5 Soft Successful 1 2 3 4 5 Unsuccessful *Wise 1 2 3 4 5 Foolish Strong 1 2 3 4 5 Weak You can see that items 3 and 5 (with asterisks) are “reversed” when compared to the rest of the items, i. e. , most of the left-hand descriptors are associated with positive attributes while the right-hand descriptors are associated with negative attributes.

Examples of “biased” questions Why is statistics better than calculus? Should smacking a child as part of good parental correction be a criminal offense? Do you favor the United States Army abolishing the affirmative-action program that produced leaders such as former US Secretary of State Colin Powell? Yes or no? " Do you still cheat on your taxes? How satisfied are you with your pay and job conditions?

Examples of “unbiased” questions Which is better, statistics or calculus? Should smacking a child be a criminal offense? Do you favor the United States Army abolishing the affirmative-action program? Yes or no? " Have you ever cheated on your taxes? How satisfied are you with your pay? How satisfied are you with your job conditions?