SURVEYS LOUIS COHEN LAWRENCE MANION AND KEITH MORRISON

SURVEYS © LOUIS COHEN, LAWRENCE MANION AND KEITH MORRISON © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

STRUCTURE OF THE CHAPTER • • • What is a survey? Some preliminary considerations Planning a survey Low response and non-response, and how to reduce them Survey sampling Longitudinal, cross-sectional and trend studies Strengths and weaknesses of longitudinal, cohort and cross-sectional studies Postal, interview and telephone surveys Comparing methods of data collection in surveys © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

KEY FEATURES OF SURVEYS • • • • Gather data one-shot basis; Represent a wide target population; Generate numerical, often large-scale data; Provide descriptive, inferential and explanatory data; Manipulate key factors and variables; Gather standardized information; Ascertain correlations; Remove contextual data; Gather data from multiple choice, closed questions, test scores or observation schedules; Support or refute hypotheses about the target population; They are piloted; Make generalizations about, and observe patterns of response in, the targets of focus; Louiswhich Cohen, Lawrence Keith Morrison; individual chapters, the contributors Gather© 2018 data can. Manion be and processed statistically.

TYPES OF SURVEY LONGITUDINAL PANEL STUDY/SAME SAMPLE / COHORT OVER TIME CROSS-SECTIONAL SNAPSHOTS OF DIFFERENT SAMPLES AT ONE OR MORE TIMES TREND/ SELECTED FACTORS PREDICTION CONTINUOUSLY OVER TIME STUDIES © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

PLANNING A SURVEY Specify the exact purpose of the enquiry: • Primary topics • Subsidiary topics • Specific information requirements Specify the population on which the survey is to focus Specify the resources that are available: • • Human (e. g. for administering and processing survey) Material Financial Administrative Temporal Geographical © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors Software

PLANNING A SURVEY Data collection • Researcher-administered • Self-administered • Postal survey • Telephone survey • Internet survey • Face-to-face interviews • Email © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

PLANNING A SURVEY Self-reporting • Problem of bias, under-reporting (e. g. to avoid socially undesirable responses), over-reporting (to give socially desirable answers). • Words are inherently ambiguous Ethics • Researcher must make it clear what the consent is being given for • Confidentiality, anonymity, privacy and nontraceability • Data security and identify protection © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors • Do no harm

PLANNING A SURVEY Problem definition Sample selection Design of measurements Concern for participants © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

STAGES IN SURVEY DESIGN DEFINE OBJECTIVES DECIDE INFORMATION SAMPLING INSTRUMENTATION PILOT TRAINING DATA COLLECTION DATA ANALYSIS REPORTING © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

PLANNING ISSUES • Research questions • The conceptual framework: the concepts that will be used • Operationalizing the research questions (e. g. into hypotheses) • The instruments to be used for data collection • Sampling strategies and sub-groups within the sample • Pre-piloting the survey (to generate items for the survey) • Piloting the survey • Data-collection practicalities and conduct • Data preparation • Data analysis • Reporting the. Cohen, findings (answering research questions) © 2018 Louis Lawrence Manion and Keith Morrison; the individual chapters, the contributors

PLANNING ISSUES • • • The kind of survey to be used Ethical issues Questionnaire and instrument design and appearance Question construction (measures, responses, measurement error) Validity and reliability Sampling Response rates, non-responses and attrition The medium of delivery, completion and return of the survey Data entry and data cleaning Data analysis and reporting Missing data © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors Data archiving

STAGES IN CONDUCTING A SURVEY • Define the objectives 1 • Decide the kind of survey required 2 • Formulate research questions or hypotheses 3 • Decide the issues on which to focus 4 • Decide the information that is needed to address the issues 5 • Decide the sampling required 6 • Decide the instrumentation and metrics required 7 © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

STAGES IN CONDUCTING A SURVEY • Generate the data collection instruments 8 • Decide how the data will be collected 9 • Pilot the instruments and refine them 10 • Train the interviewers (if appropriate) 11 • Collect the data 12 • Analyze the data 13 • Report the results 14 © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

PROBLEMS IN SURVEYS Poor sampling Poor question design and wording (failure to operationalize) Incorrect or biased responses Low response or non-response © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

HOW TO IMPROVE RESPONSE RATES • Identify where the problem lies: – The survey never reached the intended people; – People refuse to answer; – People may not be available; – People may not be able to answer the questions; – People may not actually have the information requested; – People may overlook some items in error; – The survey was completed and posted but failed to return; – The pressure of competing activities on the time of the respondent; – Potential embarrassment at their own ignorance if © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors respondents feel unable to answer a question;

HOW TO IMPROVE RESPONSE RATES – Ignorance of the topic/no background in the topic; – Dislike of the contents or subject matter of the interview; – Fear of possible consequences of the survey to himself/herself or others; – Lack of clarity in the instructions; – Fear or dislike of being interviewed; – Sensitivity of the topic, or potentially insulting or threatening topic; – Betrayal of confidences; – Losing the return envelope or return address; – The wrong person may open the mail, and fail to pass it on to the most appropriate person. © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

HOW TO IMPROVE RESPONSE RATES • • • Ensure that the topic is interesting and motivating; Ensure that it is easy to complete and layout is attractive; Include a covering letter that explains the survey; Choose a good time to conduct the survey; Follow-ups and polite reminders; Pre-paid return stamped addressed envelopes; Institutional affiliation, survey sponsorship or support from a high-status agent; Financial incentives and rewards for return; Make instructions about responses and return very clear; Avoid open-ended questions unless these are really important; Avoid placing open-ended questions at the start of a questionnaire; © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

HOW TO IMPROVE RESPONSE RATES • Have easy-to-follow instructions and spacing of the text; • Flatter the participants without being seen to flatter them; • Provide information about the research through a covering letter and/or advance notification; • Make the survey look very unlike junk mail; • Consider asking the respondents for an interview to complete the survey questionnaire; • Deliver the questionnaire personally rather than through mail; • Ensure that the questions or items are nonjudgemental; • Assure confidentiality and anonymity; © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

HOW TO IMPROVE RESPONSE RATES • Follow a sequence: – Send non-respondents a friendly reminder after ten days, stressing the importance of the research; – Send a further friendly reminder ten days after the initial reminder, stressing the importance of the research; – Make a telephone call to the respondents shortly after the second reminder, indicating the importance of the research. © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

CONSIDERATIONS IN SAMPLING • • A probability and non-probability sample Confidence level and interval The desire to generalize, and to whom The sampling frame The sample size The representativeness of the sample Access to the sample Anticipated response rate © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

LONGITUDINAL DATA ARE AFFECTED BY. . . History • Events occurring may change the observations of a group under study Maturation • Participants mature at different speeds and in different ways Testing • Test sensitization may occur – participants learn from exposure to repeated testing/interviews The timing of cause and effect • Some causes may produce virtually instantaneous effects and © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors others may take a long time for the effects to show

LONGITUDINAL DATA ARE AFFECTED BY. . . The direction of causality • It is not always clear or singular Comparability of data • Over time, over groups Attrition • Dropout Respondents’ memory © 2018 Louisfor Cohen, recall Lawrence Manion and Keith Morrison; individual chapters, the contributors • Capability

MEMORY IS AFFECTED BY. . . • The time that has elapsed since the event took place; • The significance of the event for the participant; • The amount of information required for the study – the greater the amount, the harder it is to provide; • The contamination/interference effect of other memories of a similar event (i. e. the inability to separate similar events); • The emotional content or the social desirability of the content; • The psychological condition of the participant at interview; © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors • Hindsight.

ADVANTAGES OF POSTAL SURVEYS • • • Reach many people Comparatively cheap Can be completed at respondents’ preferred time No risk of interviewer bias Can reach scattered populations Can gather sensitive data (as nobody else present) © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

DISADVANTAGES OF POSTAL SURVEYS • • • Low response rate No check on understanding No check whether all items are completed Need to have a very simple format Presentation matters affect completion © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

ADVANTAGES OF INTERVIEW SURVEYS • • Improves response rates Can clarify queries from respondents Can stimulate the respondent to give full answers Can be flexible (e. g. in item sequence) Benefit from non-verbal communication Can build in trust and rapport Ensures that only the respondent answers the questions © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

DISADVANTAGES OF INTERVIEW SURVEYS Interviewer characteristics affect responses Conduct of interview affects responses Flexibility can reduce standardization Costly in time, travel and training © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

ADVANTAGES OF TELEPHONE SURVEYS Reduce bias in the researcher or the interviewee Reduce costs of time and travel Easy to find more people to contact © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors

DISADVANTAGES OF TELEPHONE SURVEYS • • Not everyone has a telephone Not everyone is available for interview People are ex-directory The person answering the call may not be the most suitable person Multiple choice and rating scale questions are difficult Order effects can be strong People may lie or hang up if questions are unwelcome, sensitive, too long, too many Lack of non-verbal cues © 2018 Louis Cohen, Lawrence Manion and Keith Morrison; individual chapters, the contributors