Launching Surveys and Sample Management using Qualtrics Day

Launching Surveys and Sample Management using Qualtrics Day 3 Nathan R. Jones, Ph. D Senior Project Director University of Wisconsin Survey Center nrjones@ssc. wisc. edu University of Wisconsin Survey Center

Schedule • Day 1 - Overview • Day 2 - Questionnaire development • Day 3 - Data collection University of Wisconsin Survey Center

Principal parts of a survey • • • Study Design Sample Design Mode of administration Instrument development Pretesting, IRB Data collection Data preparation - weighting, processing, delivery Documentation Analysis Budget University of Wisconsin Survey Center

Planning related to data collection Draft project proposal and budget should include • • Study goals Timeline Modes for collecting data Instrument design Sample & sample management Methods to maximize response rate Progress reporting IRB and other permission University of Wisconsin Survey Center

Review - Study goals Two common goals that may call for very different study designs are… • To describe a population • To make comparisons among groups, such as comparisons among men and women or people of different age groups. University of Wisconsin Survey Center

Timeline Determine: • When does the survey ideally need to be started and completed? • What else is going on that could interfere? • How much uncertainty is there about when you can start? • How much flexibility do you have in when you can start? University of Wisconsin Survey Center

Modes for collecting data Ways of interacting with respondents • • • Mail Phone (CATI) Web Face-to-face (CAPI) Focus groups Mixed or multi-mode University of Wisconsin Survey Center

Factors that influence mode choices Survey population • Characteristics, abilities, resources, and interests Sample frame • Information available (area, name, address, telephone number, email address) Study topic • Sensitivity, task difficulty, need to use visual aids Analytic goals • Comparisons needed with studies using other modes University of Wisconsin Survey Center

Factors that influence mode choices Administrative infrastructure • Availability of technology and experience with technology Interviewers • Ability and experience Cost • Budget; existing resources Time • Desired length of field period; timely access to data Response rate • Need for high response rate University of Wisconsin Survey Center

Sample and sample management Some important questions: • What is the desired sample size? • Who will complete the survey? • Do you have a list of the individuals to be surveyed? • What contact information is on the list? • How current? • How accurate? • Will someone need to purchase or otherwise acquire the sample? University of Wisconsin Survey Center

Sample More questions: • Will you need to track/trace sample members? • Will this be done before the project is put into the field, while the project is in the field, or both? • Will certain groups need to be oversampled? • Will households need to be screened to determine if there is anyone eligible to participate in the study? University of Wisconsin Survey Center

Sample Do you need a sampling statistician or other specialist for this project? • • Hard to reach populations Low incidence populations Complex sampling Complex weighting University of Wisconsin Survey Center

Increasing response rates Incentives “Can I just pay people to answer? ” University of Wisconsin Survey Center

Incentives Why do people participate? • Theories • Social exchange (Dillman) • Economic exchange (Biner and Kidd 1994) • Leverage-salience theory (Groves et al. 2000) • Dimensions • • Prepaid vs. promised incentives Monetary vs. nonmonetary Amount of incentive Mode University of Wisconsin Survey Center

Incentives meta-analysis (Church 1993) • Meta-analysis of 38 mail surveys • Prepaid incentives increased response rates more than promised incentives • Prepaid monetary incentives yield higher response rates than gifts • Response rates increase with increasing amounts of money, although not all studies find a linear relationship University of Wisconsin Survey Center

Protection of human subjects Permission to do research Issues frequently raised by survey work • • • Where are you getting your participants? How will they be recruited? What is consent process? Written consent or waiver of written consent How will confidentiality be protected? Incentives University of Wisconsin Survey Center

Protection of human subjects Potential complicating factors: • • • Will you be linking data from medical records? Sensitive medical conditions or issues Children Illegal activities Prisoners Anything else? University of Wisconsin Survey Center

Data collection • • • Begin field period Discover quality of sample Progress reporting Sample management Close field period University of Wisconsin Survey Center

Field procedures to maximize response • • Train interviewers Method for initial contact and obtaining cooperation Follow-up procedures Converting refusals Quality control Supervision and reporting Recontact methods for panels University of Wisconsin Survey Center

Data management Data delivery • • Preliminary data Additional data deliveries as needed Recodes/post-processing Weights Final reports and documentation • Response rate report • Codebook • Methods University of Wisconsin Survey Center

Post-survey processing Coding and cleaning • • • Standardized coding systems Training coders Measure coding reliability Coding schemes for open questions Procedures for protecting respondents University of Wisconsin Survey Center

Questions? University of Wisconsin Survey Center

Thank you! Nathan R. Jones Senior Project Director University of Wisconsin Survey Center 4418 Sterling Hall (608) 890 -4724 nrjones@ssc. wisc. edu University of Wisconsin Survey Center