Nonsampling errors n Total survey error sampling errors

Non-sampling errors n Total survey error, sampling errors, non-sampling errors

Survey errors n n Sampling error is one among other errors that occur in surveys (survey errors). Usually, we distinguish between the sampling errors and the group of other errors. That group we call non-sampling errors

Survey Error Sampling Error Due to selecting a sample instead of the entire pop’n. Nonsampling Errors due to mistakes or system 2 deficiencies.

Non-sampling Errors Much more difficult to deal with than sampling error Many sources of non-sampling error: • · Questionnaire • · Interviewer • · Respondent • · Recall • · Non-response • · Data processing • · Analysis - estimation, weighting

Non-sampling errors Frame Error Nonresponse Error Processing Error Measurement Error 2 -

Frame Error Nonresponse Error • Undercoverage • Overcoverage • Duplications Processing Error Measurement Error 2 -

Frame Error Nonresponse Error • Unit nonresponse • Item nonresponse • Incomplete Info. Processing Error Measurement Error 2 -

Frame Error Nonresponse Error Processing Error Measurement Error • Editing • Data Entry • Coding • Weighting • Tabulation 2 -

Frame Error Nonresponse Error Processing Error Measurement Error • Respondent • Interviewer • Questionnaire 2 -

Non-sampling errors Systematic Error Variable Error 2 -

Non-sampling errors Systematic Errors that tend to go in same direction. Noncompensating errors. Bias Variable Errors that go in both directions. Compensating errors. Variance 2 -

Total Survey Error MSE = Mean square error Variance is a measure of the variation of over imagined repetitions of the survey

Total Survey Error The square root of MSE is: : Standard error = the square root of the variance

Total Survey Error = Total survey error

Total Survey Error Different situations: s. e(Y) Bias An absolutely perfect sample survey! Not an unusual situation for some estimates in poorly managed surveys Census

Total Survey Error 2 6 1. 4 6. 5 The total survey error has increased despite the doubling of the sample size Bias has increased The sample size is doubled (e. g. from 5000 hh to 10000 hh), giving a reduction in standard error from 2 to 1. 4

Effects of Nonsampling Errors of Estimates n Variable errors increase the variances of means, totals, and proportions n n n Confidence levels for interval estimates may be over-stated Systematic errors bias the estimates of means, totals, and proportions Both variable and systematic errors bias estimates of correlation and regression coefficients 217

n n Sampling errors can be assessed from the survey data. Non-sampling errors cannot be assessed from survey data. Special studies are needed for that.

Studies to assess non-sampling errors n n Reinterview studies. To obtain the true value for a subset of items and a subsample of respondents Record check studies. To obtain the true value of characteristics by accessing administrative or official records

Difficult to assess. So, what to do? n Keep the non-sampling errors under control by: n n n Make sure the sampling frames are of good quality Test the questionnaires properly Select good interviewers and train them well Tight supervision of field work Good data editing procedures

A few examples n n Respondent fatigue. 1 st and second week in diary of expenditures. Recall errors. Children ever born Respondent don’t know the answer Response alternatives directs the answers

Ratio between the estimate of average monthly consumption from one-week diary and from two-week diary Commodity group Food and non-alcoholic beverages Alcoholic beverages, tobacco Clothing and footwear Housing, water, electricity, gas and other fuels Furnishing, hh equipment, maintenance Health Transport Communication Recreation and culture Education Restaurants and hotels Miscellaneous goods and services Total consumption Ratio Standard CI error of lower the ratio CI upper 1. 16 1. 21 1. 03 0. 02 0. 07 0. 05 1. 11 1. 08 0. 93 1. 21 1. 34 1. 14 0. 98 0. 03 0. 93 1. 03 1. 39 0. 05 1. 30 1. 48 1. 35 1. 15 0. 99 1. 04 1. 07 1. 26 1. 25 1. 15 0. 09 0. 07 0. 03 0. 08 0. 23 0. 08 0. 05 0. 02 1. 18 1. 02 0. 94 0. 89 0. 61 1. 10 1. 16 1. 10 1. 52 1. 28 1. 05 1. 18 1. 53 1. 42 1. 35 1. 20

Mean number of children ever born by age of mother. Comparison of collection methods. Nepal (Chidambaram, Cleland Verma, 1980) Collection Age of mother method 35 -39 Small scale, intensive, survey Large scale survey or census 40 -44 45 -49 5. 1 5. 5 5. 7 3. 7 4. 0

Example: Misreporting from respondent, ”Age heaping”

Reported Daily TV Viewing as a Function of Response Alternatives Low Frequency Alternatives Up to 1/2 h to 1 h 1 h to 11/2 h to 2 h 2 h to 21/2 h > 2 1 /2 h 7. 4% 17. 7% 26. 5% 14. 7% 17. 7% 16. 2% High Frequency Alternatives Up to 21/2 h 62. 5% 21/2 h to 3 h 23. 4% 3 h to 31/2 h 7. 8% 31/2 h to 4 h 4. 7% 4 h to 41/2 h 1. 6% > 4 1 /2 h 0. 0%

Views TV at least 21/2 hours Low frequency High frequency alternatives --------------------16. 2% 37. 5%

END