How Sure is Sure Quantifying Uncertainty in Tables

How Sure is “Sure”? Quantifying Uncertainty in Tables Political Science 30: Political Inquiry

How Sure is Sure? Quantifying Uncertainty in Tables �Using Two-Way Tables SAT scores and UC admissions What’s the null hypothesis here? �Chi-Square Test The formula Interpreting your results Using a Chi-Square test to hold constant a confound or another independent variable

Using Two-Way Tables �A recent study of UC admissions over the past two years showed that thousands of students with SATs under 1000 have been admitted. Even flagships like UCSD, UCLA and Berkeley admitted hundreds. Many observers offered this hypothesis to explain the trend: Minority students with low SATs are more likely to be admitted.

Using Two-Way Tables Let’s examine the bivariate correlation between ethnicity and admittance. Ethnicity Admitted Not Admitted Total Underrepresented Minority 7, 605 4, 488 12, 093 White/Asian 7, 235 4, 170 11, 405 Total 14, 840 8658 23, 498 Data on UC applicants who scored lower than 1000 on the SAT, taken from the Los Angeles Times, November 3, 2003.

Using Two-Way Tables �If the ethnicity of a student with sub-1000 SAT scores makes him more likely to be admitted to the UC, admission rates in that table will differ by race. 62. 9% for underrepresented minorities 63. 4% for whites and Asian-Americans Opposite of what the hypothesis predicted!

Using Two-Way Tables �We still want to see if this small difference in our sample is“statistically significant” at the 95% confidence level. We could do a difference in proportions test. We will do a Chi-Square test, a nice option when we have more than two groups. The null hypothesis is that there is no association between ethnicity and UC admission rates for these students.

Chi-Square Test Under the null hypothesis, you would expect to see table entries that indicate the same admission rates for each ethnicity. To calculate the expected count in any cell of a two-way table when the null hypothesis is true:

Chi-Square Test For the Row #1, Column #1 entry in our UC admission table, which gives the number of underrepresented minorities who are admitted, this expected count is:

Chi-Square Test �To conduct a Chi-Square test, first fill in a table’s worth of expected counts: Ethnicity Observed Admitted Expected Not Admitted Underrepresented Minority 7, 605 4, 488 7, 638 4, 456 White/Asian 7, 235 4, 170 7, 203 4, 202

Chi-Square Test Then use this formula to get a single “Chi. Square statistic, ” which is a measure of how much the observed counts differ from the expected counts, for your table:

Chi-Square Test �So for each entry: Step #1. Subtract the expected count from the observed count. Step #2. Square that difference. Step #3. Divide that square by the expected count to get a quotient. �Then cells. add up your quotients for all of the

Chi-Square Test �When the null hypothesis is true, the resulting Chi-Square statistic has a sampling distribution called the “chisquare distribution, ” characterized by its degrees of freedom, which equals: df = (# of rows - 1) • (# of columns - 1) �Our UC admissions table has 1 degree of freedom: (2 -1) • (2 -1) = 1 • 1 =1

Chi-Square Test We can reject the null hypothesis at the 95% confidence level if our Chi -Square statistic is greater than the values given by this table for each number of degrees of freedom. Chi-Square is 0. 76 in UC example, so cannot reject degrees of freedom Threshold Value 1 3. 84 2 5. 99 3 7. 81 4 9. 49 5 11. 07 6 12. 59 7 14. 07 8 15. 51 9 16. 92

Chi-Square Test �If you have a confound, like“Did the students parent attend UC? , ”you can use the Chi-Square test to see if there is a relationship between ethnicity and admission, holding confound constant: Build one 2 X 2 table for students who parents attended a UC, and another 2 X 2 table for students with no legacy Conduct two separate Chi-Square tests to examine bivariate relationship for each group.