Journalism 614 Survey Research Survey Research Structured interview

Journalism 614: Survey Research

Survey Research ¨ Structured interview of: – Sample of individuals in order to generalize to a larger population ¨ Survey modes: – Face-to-face – Paper-and-pencil – RDD telephone – Direct mail – Internet

Advantages & Disadvantages 9 Generalizability - external validity 9 Representativeness - unbiased sample 9 Customizability - wide variety of research questions 4 4 Time - 6 months from start to finish Cost - Face-to-face vs. Telephone vs. Mail Hard to find facilities and experts Causality - Non-experimental design

Time and Survey Design ¨ Static designs: – Cross-sectional Survey - Poll ¨ Longitudinal designs: – Trend studies - Multiple Polls – Cohort studies – Study of Group Change – Panel studies – Study of Individual Change

Cross-sectional Study ¨ Static snapshot ¨ Slice of population at one point in time – E. g. , an opinion poll ¨ Inherent limitation: – Inability to capture change over time – Usually giving correlations not causal inferences

Cross Section Example

Trend Studies ¨ Measures changes over time ¨ Sequential cross-sections of the population – E. g. , Changes over time in: • Political knowledge levels • Concern about global warming • Presidential approval rating

Trend Example

Cohort Studies ¨ Tracking changes in a group as they age – Baby boomers (born during the post–World War II baby boom, approximately between 1946 and 1964) – Generation X – Millennials ¨ Measure change across the aging process – Do millenials become more conservative? – Why can’t you answer this question with a crosssectional design? – Untangle lifecycle vs. cohort differences

Cohort Example

Panel Studies ¨ Goes a step further: – interviewing the same people more than once ¨ Captures change in individuals over time – E. g. , NES Election Study • Pre-election and post-election ¨ Can begin to explain which individuals are changing and why they are changing ¨ The respondent mortality problem: – Are those who drop out different?

Panel Example

Misconceptions ¨ ¨ ¨ Single time-point >>> Longitudinal, Panel Designs Must be face-to-face >>> Can use telephone, mail Interviewers read questions >>> Self-administered Individuals as unit of observation >>> Family Non-experimental >>> Can embed experiments Atheoretical >>> Can test hypotheses Surveys are a very flexible research technique

Reliability of Survey Research ¨ Stability: – In panel designs, test-retest correlations ¨ Reproducibility: – In open-ended questions, coder agreement ¨ Internal Consistency: – In scales, the consistency of item response

Validity of Survey Research ¨ Face Validity: – Do items capture concepts? ¨ Content Validity: – Are relevant dimensions represented by indicators? ¨ Convergent Validity: – Are multiple indicators correlated? ¨ Divergent Validity: – Do indicators allow us to differentiate from other concepts? – Do indicators differentiate between distinct concept dimensions?

Stages in Survey Research ¨ General Research Questions ¨ Specific Research Questions ¨ Sampling Design ¨ Questionnaire Development ¨ Interviewer Training ¨ Pretest ¨ Fieldwork - Test ¨ Content Coding ¨ Analysis Computation ¨ Report Writing

Questionnaire Construction ¨ Length: – : 30 for telephone, longer for personal/self-administered ¨ Ordering: – Put an easy question first, funnel toward specific – Save sensitive question for the end ¨ Transitions: – Ease them from one section to another ¨ Probes: – Further information, elaboration

Training Interviewers ¨ Two parts: – 1. Basic interviewing skills – 2. Specific interview schedule, questionnaire

Motivations and Barriers ¨ Intrinsic and Extrinsic motivations: – expression, boredom, loneliness, politeness, curiosity, loyalty ¨ Barriers: – Suspicion, fear, inadequacy, privacy, distractions, time to answer ¨ Overcoming Barriers: – confidentiality, listening, probing, repeating, focus, and practice