University of Warwick Department of Sociology 201213 SO

University of Warwick, Department of Sociology, 2012/13 SO 201: SSAASS (Surveys and Statistics) (Richard Lampard) Survey Design: Some Implications for Secondary Analysis (Week 10) (Revised version)

Weighting • Sample designs often do not select respondents with a single, consistent probability of selection. • Unequal selection probabilities need to be taken into account at the analysis stage, to allow the results to be properly generalisable to the population of interest. • For a discussion of how weighting works, see pp 52 -53 of the online course extract by Pole and Lampard (2002), which also covers sampling more generally.

Complex Samples and Sampling Error • Weighting can increase the expected amount of sampling error. • Stratification can reduce the expected amount of sampling error. • Clustering within samples can increase the expected amount of sampling error. • For a concise discussion of the above, see Section 9. 4 of the online course extract by Sturgis (2008).

But. . . • Secondary analysts frequently treat samples as if they were simple random samples. • Hence the resulting analyses may underestimate (or over-estimate) the likely amount of sampling error. • In consequence the p-values may not be as reliable as they should be.

The solution. . . • Recent versions of SPSS (and various other forms of software) allow one to incorporate these aspects of sample design into the analysis, and thus take account of their effect on the standard errors of estimates in an appropriate way. • In SPSS the ANALYZE/COMPLEX SAMPLES menu allows one to specify variables relating to the strata, the clusters, and the weights, in the context of a small range of techniques, including logistic regression.

Merging files • Sometimes the design of a survey and/or a research focus means that a secondary analyst needs to merge data from two or more files. • For example, information on aspects of individuals’ life histories may be included in separate files. • Sometimes information about household members needs to be matched by saving subsets of members and matching them (e. g. matching individuals with their resident partners).

Example • In the case of a repeated survey like the British Social Attitudes survey, different years which contain some of the same questions can be matched to facilitate examinations of social change. • In this case the relevant SPSS command is DATA / MERGE FILES / ADD CASES. • Adding additional variables (ADD VARIABLES) is slightly more demanding, as matching variables in both sources of data need to be used to properly link the information relevant to each case from the two sources.