University of Warwick Department of Sociology 201415 SO

University of Warwick, Department of Sociology, 2014/15 SO 201: SSAASS (Surveys and Statistics) (Richard Lampard) Week 2 -3 Wedding Attitude by Sex Cross-tabulation Example

First version of cross-tabulation Wedding more celebration than commitment? by Sex Agree strongly Agree Neither Disagree strongly Can't choose Not answered Total Male 118 9. 7% 522 42. 8% 203 16. 6% 265 21. 7% 68 5. 6% 23 1. 9% 21 1. 7% 1220 Female 167 10. 7% 710 45. 7% 234 15. 0% 335 21. 5% 67 4. 3% 19 1. 2% 23 1. 5% 1555 Total 285 10. 3% 1232 44. 4% 437 15. 7% 600 21. 6% 135 4. 9% 42 1. 5% 44 1. 6% 2775

Commentary on first version • The attitude measure is a nominal variable (i. e. a set of categories), but the first five categories constitute an ordinal variable (i. e. ordered categories) • It is tempting to view the first five categories as an interval-level variable (i. e. a scale) • (Pearson) chi-square treats the measure as nominal, and has a statistically non-significant value (7. 628, 6 d. f. , p=0. 267 > 0. 05) • However the Linear-by-Linear Association statistic treats the variable as interval-level (N. B. the last two categories mean that it isn’t!), and is significant (4. 870, 1 d. f. , p=0. 027 < 0. 05)

Second version of cross-tabulation Wedding more celebration than commitment? by Sex Agree strongly Agree Neither Disagree strongly Total Male 118 10. 0% 522 44. 4% 203 17. 3% 265 22. 5% 68 5. 8% 1176 Female 167 11. 0% 710 46. 9% 234 15. 5% 335 22. 1% 67 4. 4% 1513 Total 285 10. 6% 1232 45. 8% 437 16. 3% 600 22. 3% 135 5. 0% 2689

Commentary on second version • When we render the attitude measure properly ordinal by removing the last two categories, the Linear-by-Linear Association statistic ‘unfortunately’ becomes nonsignificant (p=0. 071 > 0. 05) • However, this statistic doesn’t treat the variable as ordinal, but as interval-level (using the codes for the categories as a scale).

Third version of cross-tabulation Wedding more celebration than commitment? by Sex Agree strongly/Agree Neither/Disagree strongly Total Male Female Total 640 54. 4% 468 39. 8% 68 5. 8% 877 58. 0% 569 37. 6% 67 4. 4% 1517 56. 4% 1037 38. 6% 135 5. 0% 1176 1513 2689

Commentary on third version • However, if we consider the possibility that, substantively, there is not much difference between agreeing and agreeing strongly, and that there is not much different between answering ‘Neither’ and answering ‘Disagree’, then we can reduce the measure to three categories. • If these are coded 1, 2 and 3, and then treated as interval-level, then the Linear-by-Linear Association statistic become significant (p=0. 033 < 0. 05)

Conclusion • As is sometimes the case, our findings are ambiguous here. • Clearly the basic chi-square statistic ignores the apparent trend in the percentages in the table, and is not taking proper account of the variable’s ordinality. • But if we tinker with the categorisation in a way that simply reflects the observed pattern in the cross-tabulation, we can be viewed on one hand as teasing out a ‘real’, substantively plausible underlying relationship, or on the other hand as (inappropriately) nudging the p-value in the direction of significance, and hence increasing the risk of a Type I error (i. e. a false positive). • Either way, with p-values around the conventional 5% (0. 05) significance level, we should perhaps be cautious about viewing a finding as either ‘significant’ or ‘non-significant’…