Survey Error Focus on Systematic Error Dr Michael

Survey Error: Focus on Systematic Error Dr. Michael R. Hyman, NMSU

2

Total Survey Error: Components Random sampling error Total error Systematic error (bias) 3

Random Sampling Error A statistical fluctuation that occurs because of chance variation in the elements selected for the sample. 4

Systematic Error Systematic error results from some imperfect aspect of the research design or from a mistake in the execution of the research 5

6

Respondent Error A classification of sample bias resulting from some respondent action or inaction • Nonresponse bias • Response bias 7

Respondent Error: Components Nonresponse error Respondent error Response bias 8

Non-response Error • Non-respondents – Refuse to cooperate – Not-at-homes • Self-selection bias – Over-represents extreme positions – Under-represents indifference • Varies by type of interview 9

Why People Cooperate • • To be supportive/helpful Have a social interaction Curiosity Remuneration 10

Factors Contributing to Refusal • Fear/anxiety • Invasion of privacy • Hostility toward – Survey sponsor – Topic of interview – Interviewer 11

Response Bias A bias that occurs when respondents tend to answer questions with a certain slant that consciously or unconsciously misrepresents the truth 12

Response Bias: Components Deliberate falsification Response bias Unconscious misrepresentation 13

Unwillingness to Respond Accurately • • Invasion of privacy Time pressure and fatigue Physical/social environment Questionnaire-specific issue – Question/response wording – Layout • Mischievous respondents 14

15

Inability to Respond Accurately • Inarticulate • Ignorant – Asked about other people – Asked to predict own behavior – Asked question meant for other people 16

17

18

Inability to Respond Accurately • Faulty memory – Only ask about recent events – Problem with questions about behavior over time (e. g. , frequency) – Telescoping • Everything happened yesterday 19

Unconscious Misrepresentation: Components Acquiescence bias Extremity bias Interviewer bias Auspices bias Social desirability bias 20

Acquiescence Bias A type of response bias due to some people tending to agree with all questions or to concur with a particular position. 21

Extremity Bias A type of response bias due to response styles that vary from person to person; some people tend to use extremes when responding to questions. 22

Interviewer Bias A response bias that occurs because the presence of the interviewer influences answers. 23

24

Auspices Bias in respondents’ answers due to being influenced by the organization conducting the study. 25

Social Desirability Bias in responses caused by respondents’ desire, either conscious or unconscious, to gain prestige or appear in a different social role. 26

27

28

Systematic Error: Components Administrative error Systematic error (bias) Respondent error 29

Administrative Error • Improper administration of the research task • Blunders • Confusion • Neglect • Omission 30

Administrative Error • Interviewer cheating - filling in fake answers or falsifying interviewers • Data processing error - incorrect data entry, computer programming, or other procedural errors during the analysis stage. • Sample selection error -improper sample design or sampling procedure execution. • Interviewer error - field mistakes 31

Interviewer Error • Subtle source of cues – Can inadvertently reinforce some responses • Appearance of incompetence 32

33

34

Interviewer Error Insufficient or poor probing 35

36

37

38

Interviewer Error Fraud • Why commercial houses validate roughly 10 -15% of completed interviews Selecting unqualified respondents 39

Checklist for Unacceptable Questionnaires • Major portions of questionnaire or key questions left unanswered • Evidence that respondent did not understand instructions or did not take task seriously • Missing pages • Respondent not qualified for target population • Questionnaire returned after cutoff date 40