Introduction to Sample Surveys Census and Statistics Department

Introduction to Sample Surveys 政府統計處 Census and Statistics Department

Sample Surveys Sample survey has widespread applications, ranging from surveys conducted for business purposes to collection of data for use in public affairs and social studies. In order to ensure the reliability of survey results, it is essential that great care in addition to adequate professional knowledge be applied in the planning and conduct of surveys and in the analysis of data. 政府統計處 Census and Statistics Department

Major Steps in Conducting a Sample Survey Overall planning Design and selection of sample Design of questionnaire Collection of raw data Compilation and analysis of statistics and dissemination of survey results 政府統計處 Census and Statistics Department

Overall Planning Clearly defined survey objectives Clearly defined target population Clearly defined statistical standards (definition of terms, counting rules, classification, statistical methodologies) Respondents should be able to provide the information required. Adequate resources such as manpower and time must be available. 政府統計處 Census and Statistics Department

Design and Selection of Sample Sampling deals with the selection of a number of elements, i. e. a sample. A sample can be drawn using different methods. A probability sample, with each element having a known, non-zero chance of being included, should be used as far as practicable so that scientific inference can be drawn from the survey results. 政府統計處 Census and Statistics Department

Design and Selection of Sample Non-probability sampling methods, such as haphazard sampling, should be avoided. Such methods are not scientific and bias usually exists in survey results. A complete and up-to-date sampling frame should be acquired. 政府統計處 Census and Statistics Department

Design and Selection of Sample In case of unequal probability of selection, it is necessary to ensure that proper weighting methods are applied to survey results. Using established statistical methodology to compute the required sample size. Once the sampling units are selected, alterations are not allowed. 政府統計處 Census and Statistics Department

Common Probability Sampling Methods Simple random sampling Systematic sampling Stratified sampling Clustering sampling Multi-stage sampling 政府統計處 Census and Statistics Department

Non-probability Sampling Methods Selection of elements is based on subjective judgment and experience, but not a random manner Common non-probability sampling methods ØStreet interview/mall intercept ØQuota sampling ØRespondent-initiated telephone polling ØInternet survey 政府統計處 Census and Statistics Department

Non-Probability Sampling Methods Why unreliable? ØScientific inference cannot be drawn from the sample data (results are confined to describing the group of respondents, but cannot be extracted to the entire population) ØLevel of precision of the estimates cannot be scientifically assessed ØBiases will most likely exist in the survey results 政府統計處 Census and Statistics Department

Non-Probability Sampling Methods Why unreliable (cont’d)? ØNo way to know how exactly the group of respondents is formed • In street interviews, the interviewers tend to select those “friendly faces” for interview. • In self-selecting polls, only those who have strong views on the survey may volunteer to participate. ØThe results may be inaccurate and misleading, hence not of much use. 政府統計處 Census and Statistics Department

General Principles in Designing Questionnaires Questions should be relevant to the survey objectives Questions should be arranged in a proper order Use screening question to enhance the flow of the questions. Use an appropriate language Clear instructions 政府統計處 Census and Statistics Department

General Principles in Designing Questionnaires Questions wordings should be appropriate, specific and precise Avoid leading questions or questions being loaded in favour of a particular response Avoid difficult vocabulary Avoid composite and double negative questions Beware of memory error 政府統計處 Census and Statistics Department

General Principles in Designing Questionnaires “Don’t know/No opinion” should be included as appropriate Long questionnaires are undesirable. Questionnaires should be tested on some prospective respondents before finalized. 政府統計處 Census and Statistics Department

Collection of Raw Data Methods of data collection for surveys include: ØSelf-administered questionnaires by mail ØPersonal interviews ØTelephone interviews ØComputer Assisted Telephone Interviewing (CATI) Very often, mixed modes of data collection can be used. 政府統計處 Census and Statistics Department

Collection of Raw Data An appropriate mode should be selected by carefully considering respondents' willingness to co-operate, the degree of complexity of the subject of enquiry and other relevant factors (e. g. practicability of using personal or telephone interviews). 政府統計處 Census and Statistics Department

Comparison of Different Data Collection Methods Characteristics Personal interview High Telephone interview Medium Mail questionnaire Low Low Non-response/ non-contact rate Suitability for scattered population Interviewer bias Relatively lower Costly Medium to high Less costly High Ｍedium Yes No Response quality Length of questionnaire High May be longer May be more complicated Not suitable Medium Preferably short Simple Low Preferably short Simple Easier Cost Manpower Time consuming Design of questionnaire Asking embarrassing questions

Collection of Raw Data Interviewers should be trained before they start working and closely supervised during fieldwork to ensure their quality of work. Identity and information supplied by individual respondents should be kept confidential. The survey results are to be presented in the form of aggregate statistics. 政府統計處 Census and Statistics Department

Collection of Raw Data Every effort should be made to achieve a high response rate (or reducing the number of non-responses). Methods to reduce non-responses: ØKeep the questionnaire brief and concise ØConsider rewards to respondents ØBetter publicity measures (e. g. advertisement, advance letters) ØAssurance of confidentiality of individual data 政府統計處 Census and Statistics Department

Collection of Raw Data Methods to reduce non-responses (Cont’d): ØMore experienced interviewers, especially when handling refusals ØVisit households in evening time ØVisit at different time of different days ØIncrease the number of re-visits/call backs ØUse self-administered questionnaires in case of non-contact 政府統計處 Census and Statistics Department

Compilation and Analysis of Statistics Data should be carefully and thoroughly checked before compilation. Appropriate statistical methodology should be adopted in compiling and analysing data. 政府統計處 Census and Statistics Department

Reporting and Assessing the Reliability of Surveys To enable readers to make judgment on whether the findings are credible, a good survey report should include: ØSponsorship of the survey ØPopulation covered ØSampling method ØMode of data collection ØTime period of data collection ØWording of questions 政府統計處 Census and Statistics Department

Reporting and Assessing the Reliability of Surveys A good survey report should include (cont’d): ØSample size and response rate ØPoint estimates and confidence intervals (if possible) Ø Likely sources of non-sampling errors Information supplied by individual respondents should not be disclosed. 政府統計處 Census and Statistics Department

Reporting and Assessing the Reliability of Surveys One may access the reliability of a survey by asking the following questions : ØHas probability sampling methods been used ? ØIs the sample size reasonably large ? ØIs the questionnaire design proper ? Any leading questions or wordings ? 政府統計處 Census and Statistics Department

Reporting and Assessing the Reliability of Surveys One may access the reliability of a survey by asking the following questions (cont’d): ØWhat is the interviewing method ? Any improper influence by interviewers or third parties during the interview ? ØIs the response rate too low ? ØAre the sampling errors of acceptable magnitude ? How about non-sampling errors and biases ? 政府統計處 Census and Statistics Department