Megan Templin MPH M S H James Norton

Megan Templin, MPH, M. S. H. James Norton, Ph. D Dickson Advanced Analytics Research Division Carolinas Healthcare System 1

Chi-square Test 2

Jeopardy: Statistics for 100 The test that should be performed to answer: …“The general problem may be stated as follows: having given the number of instances in which things are both thus and so, in which they are thus but not so, in which they are so but not thus, and in which they are neither thus nor so, it is required …to determine the quantitative relativity between the thusness and the soness of the things. ” 3

Jeopardy: Statistics for 100 What is the chi-square test? Thus Not Thus So Not So Bulletin of the Philosophical Society of Washington, (1888). 4

Jeopardy: Statistics for 800 The book this is a quote from: …“Grownups love figures. When you tell them you have made a new friend, they never ask you about essential matters. They never ask you, “What games does he love best? …Instead they demand, “How old is he? How much money does his father make? ” Only from these figures do they think they have learned anything about him. ” 5

Jeopardy: Statistics for 800 What is “The Little Prince? ” 6

What is the chi-square test? • A statistical test used to compare groups on the percentage with a condition or outcome • Also known as contingency table 7

Assumptions of 1. Frequency data 2. Random sample 3. Adequate sample size 4. Adequate expected cell counts 5. Independent observations 8

Chi-square hypothesis • Null hypothesis: – There is no difference in the proportions of the two treatment groups – There is not an association between the two variables – The outcome (survival) is INDEPENDENT of the treatment group – H 0: P 1=P 2 • Alternative hypothesis: – There is a difference in the proportions of the two treatment groups – There is an association between the two variables – The outcome (survival) is DEPENDENT on the treatment group (the 2 variables are not independent and therefore survival depends on which group you are in) – HA: P 1≠P 2 9

Chi-square Example • Suppose we are interested in studying a drug that we hope will increase the 2 -year survival of patients following an M. I. • We have 46 patients in each of 2 groups (treatment vs. placebo). • Furthermore, suppose we lose to follow-up one patient in the experimental drug group. 10

Example (continued) • Null hypothesis: – H 0: P 1=P 2 – There is no difference in the proportions of the two treatment groups. – The outcome (survival) is INDEPENDENT of treatment group • Alternative hypothesis: – HA: P 1≠P 2 – There is a difference in the proportions of the two treatment groups; – The outcome (survival) is DEPENDENT of the treatment group (survival and treatment are not independent of each other and therefore survival depends on which treatment group you are in) 11

Example (continued) Drug Outcome Experimental Drug Control Survived 38 29 67 Died 7 17 24 45 46 91 Marginal totals Grand Total 12

Example (continued) • In the literature we could report this data as: ü % surviving at 2 years with experimental drug = = 84% ü % surviving at 2 years with control = = 63% 13

Example (continued) Drug Exp. Drug Outcome Survived Died Control 38 29 67 7 17 24 46 91 45 The 4 cells of this table are OBSERVED values We must calculate what is known as EXPECTED values one would expect if the 2 variables were INDEPENDENT 14

Example (continued) Drug Exp. Drug Outcome Survived Died Control 38 29 67 7 17 24 46 91 45 To calculate the critical value you need to first calculate the degrees of freedom and the expected value for each cell. Expected values: Chi-square: = # in row x # in column total number 15

Example (continued) 16

Chi-Square Table 17

Chi-Square Table 5. 024 6. 635 So, 0. 025 > p > 0. 01 Computers give exact calculation, p=0. 02 Conclude: The percentage of people who survived at 2 years was significantly higher in the experimental drug group (84%) than in the control group (63%). 18

Walther, A. E. , MD; Pritts, T. A. , MD, Ph. D; Falcone, R. A. , MD, MPH; Hanseman, D. J. , Ph. D; Robinson, B. R. H. , MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109 -116. Adult Trauma Center Pediatric Trauma Center Total N N N 3, 175 1, 676 4, 851 Penetrating 618 321 939 Burn 82 66 148 Asphyxia 12 26 38 3, 887 2, 089 5, 976 Injury Type Blunt Total The null hypothesis says that there is not an association between injury type and trauma center (they are independent) The alternative hypothesis says there is an association between injury type and trauma center (they are dependent) 19

Walther, A. E. , MD; Pritts, T. A. , MD, Ph. D; Falcone, R. A. , MD, MPH; Hanseman, D. J. , Ph. D; Robinson, B. R. H. , MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109 -116. Adult Trauma Center Pediatric Trauma Center Total Column % N Blunt 81. 7% 80. 2% 4, 851 Penetrating 15. 9% 15. 3% Burn 2. 1% Asphyxia Total Injury Type Adult Trauma Center Pediatric Trauma Center Total Row % N Blunt 65. 5% 34. 5% 4, 851 939 Penetrating 65. 8% 34. 2% 939 3. 1% 148 Burn 55. 4% 44. 6% 148 0. 3% 1. 2% 38 Asphyxia 31. 6% 68. 4% 38 3, 887 2, 089 5, 976 Total 3, 887 2, 089 Injury Type 5, 976 20

Walther, A. E. , MD; Pritts, T. A. , MD, Ph. D; Falcone, R. A. , MD, MPH; Hanseman, D. J. , Ph. D; Robinson, B. R. H. , MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109 -116. Adult Trauma Center Pediatric Trauma Center Total N N N 3, 175 1, 676 4, 851 Penetrating 618 321 939 Burn 82 66 148 Asphyxia 12 26 38 3, 887 2, 089 5, 976 Injury Type Blunt Total Degree of Freedom: • df = (# rows - 1) x (# columns - 1) • Injury Type: 4 - 1=3 • Type of Center: 2 - 1=1 • df = 3 x 1 = 3 21

Walther, A. E. , MD; Pritts, T. A. , MD, Ph. D; Falcone, R. A. , MD, MPH; Hanseman, D. J. , Ph. D; Robinson, B. R. H. , MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109 -116. Adult Trauma Center Pediatric Trauma Center Total N N N 3, 175 1, 676 4, 851 Penetrating 618 321 939 Burn 82 66 148 Asphyxia 12 26 38 3, 887 2, 089 5, 976 Injury Type Blunt Total Expected values: = # in row x # in column total number 22

Walther, A. E. , MD; Pritts, T. A. , MD, Ph. D; Falcone, R. A. , MD, MPH; Hanseman, D. J. , Ph. D; Robinson, B. R. H. , MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109 -116. Injury Type Adult Trauma Center Pediatric Trauma Center Total N N N 3, 175 1, 676 4, 851 Penetrating 618 321 939 Burn 82 66 148 Asphyxia 12 26 38 3, 887 2, 089 5, 976 Blunt Total Expected values: = # in row x # in column total number Blunt injury – Adult Trauma Center: = 4, 851 x 3, 887 = 3, 155. 3 5, 976 Observed n = 3, 175 Expected n = 3, 155. 3 23

Walther, A. E. , MD; Pritts, T. A. , MD, Ph. D; Falcone, R. A. , MD, MPH; Hanseman, D. J. , Ph. D; Robinson, B. R. H. , MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109 -116. Injury Type Adult Trauma Center Pediatric Trauma Center Total N N N 3, 175 1, 676 4, 851 Penetrating 618 321 939 Burn 82 66 148 Asphyxia 12 26 38 3, 887 2, 089 5, 976 Blunt Total Expected values: = # in row x # in column total number Burn injury – Peds. Trauma Center: = 148 x 2, 089 = 51. 7 5, 976 Observed n = 66 Expected n = 51. 7 24

Walther, A. E. , MD; Pritts, T. A. , MD, Ph. D; Falcone, R. A. , MD, MPH; Hanseman, D. J. , Ph. D; Robinson, B. R. H. , MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109 -116. Observed and Expected Values Adult Trauma Center Pediatric Trauma Center Total N N N 3, 175 (3155. 3) 1, 676 (1695. 7) 4, 851 618 (610. 8) 321 (328. 2) 939 Burn 82 (96. 3) 66 (51. 7) 148 Asphyxia 12 (24. 7) 26 (13. 3) 38 3, 887 2, 089 5, 976 Injury Type Blunt Penetrating Total 25

Walther, A. E. , MD; Pritts, T. A. , MD, Ph. D; Falcone, R. A. , MD, MPH; Hanseman, D. J. , Ph. D; Robinson, B. R. H. , MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109 -116. Chi-Square Calculation 26

Walther, A. E. , MD; Pritts, T. A. , MD, Ph. D; Falcone, R. A. , MD, MPH; Hanseman, D. J. , Ph. D; Robinson, B. R. H. , MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109 -116. Degrees of freedom = 3 Critical value = 25. 33 27

Walther, A. E. , MD; Pritts, T. A. , MD, Ph. D; Falcone, R. A. , MD, MPH; Hanseman, D. J. , Ph. D; Robinson, B. R. H. , MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109 -116. Degrees of freedom = 3 Critical value = 25. 33 28

Walther, A. E. , MD; Pritts, T. A. , MD, Ph. D; Falcone, R. A. , MD, MPH; Hanseman, D. J. , Ph. D; Robinson, B. R. H. , MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109 -116. Degrees of freedom = 3 Critical value = 25. 33 Chi-square value of 25. 33 and 3 degrees of freedom is greater than 12. 838. Therefore, the p-value is less than 0. 005. For adolescent patients, there is a statistically significant difference in the distribution of injury types between adult and pediatric trauma centers 29

Walther, A. E. , MD; Pritts, T. A. , MD, Ph. D; Falcone, R. A. , MD, MPH; Hanseman, D. J. , Ph. D; Robinson, B. R. H. , MD. (2014). Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult versus Pediatric Trauma Centers. Journal of Trauma Acute Care Surgery, vol. 77 (n. 1), pp. 109 -116. Our calculated significance: • P-value: < 0. 005 Therefore, reject the null hypothesis and state there is an association between type of trauma center and injury type. 30

Teen Trauma without the Drama: Outcomes of Adolescents Treated at Ohio Adult versus Pediatric Trauma Centers. SAS Output: Our calculations: • Df: 3 • Critical value: 25. 33 • P-value: < 0. 005 31

Key points Ø The further apart observed values are from expected values then the larger the chisquare value. This goes back to our null hypothesis that the two things we are studying are independent…in other words, the less independent two things are, the further apart the observed and expected values will be. Ø If the chi-square equals 0, then proportions perfectly match the hypothesis. Ø When the chi-square value is large, it leads to a smaller p-value and rejection of the null hypothesis. Ø You need a larger sample size for categorical comparisons compared to the sample size need for numeric comparisons (example, age in categories vs. numeric age) Ø Tests ASSOCIATION, NOT CAUSE & EFFECT!! Ø If the two variables being compared are dependent (paired) samples, Mc. Nemar’s test should be used. Ø If the grand total is <40 (some say<20) or if the expected value of any cell is <5 (some say <1), then use Fisher’s Exact test. 32

Sample Size Calculations for Chi-Square To calculate the number of patients needed in an experimental and a control group for a given probability of obtaining a significant result (two-sided test), you need to know (or educated guess): 1. Smaller proportion 2. Difference in proportions 3. Alpha 4. Power From: Career Medicine – Holland, JF, & Frei E. I. Chapter 8 33

Sample Size Example 1 Suppose the failure rate of a drug is 40%. You would like to decrease that rate to 20% with a new treatment and design a study comparing the two drugs. Using the table, how many subjects are needed in each group for an alpha level of 0. 05 and power of 0. 80? 34

Sample Size Example 1 Suppose the failure rate of a drug is 40%. You would like to decrease that rate to 20% with a new treatment and design a study comparing the two drugs. Using the table, how many subjects are needed in each group for an alpha level of 0. 05 and power of 0. 80? 1. Smaller proportion: 20% 2. Difference in proportions: 20% 3. Alpha: . 05 4. Power: . 80 35

Sample Size Example 1 Suppose the failure rate of a drug is 40%. You would like to decrease that rate to 20% with a new treatment and design a study comparing the two drugs. Using the table, how many subjects are needed in each group for an alpha level of 0. 05 and power of 0. 80? 1. Smaller proportion: 20% 2. Difference in proportions: 20% 3. Alpha: . 05 4. Power: . 80 Answer: At least 80 per group 36

Sample Size Example 2 Suppose the failure rate of a drug is 40%. You would like to decrease that rate to 20% with a new treatment and design a study comparing the two drugs. Using the table, how many subjects are needed in each group for an alpha level of 0. 05 and power of 0. 90? 37

Sample Size Example 2 Suppose the failure rate of a drug is 40%. You would like to decrease that rate to 20% with a new treatment and design a study comparing the two drugs. Using the table, how many subjects are needed in each group for an alpha level of 0. 05 and power of 0. 90? 1. Smaller proportion: 20% 2. Difference in proportions: 20% 3. Alpha: . 05 4. Power: . 90 38

Sample Size Example 2 Suppose the failure rate of a drug is 40%. You would like to decrease that rate to 20% with a new treatment and design a study comparing the two drugs. Using the table, how many subjects are needed in each group for an alpha level of 0. 05 and power of 0. 90? 1. Smaller proportion: 20% 2. Difference in proportions: 20% 3. Alpha: . 05 4. Power: . 90 Answer: At least 105 per group 39

Sample Size Example 3 Suppose the failure rate of a drug is 40%. You would like to decrease that rate to 30% with a new treatment and design a study comparing the two drugs. Using the table, how many subjects are needed in each group for an alpha level of 0. 05 and power of 0. 80? 40

Sample Size Example 3 Suppose the failure rate of a drug is 40%. You would like to decrease that rate to 30% with a new treatment and design a study comparing the two drugs. Using the table, how many subjects are needed in each group for an alpha level of 0. 05 and power of 0. 80? 1. Smaller proportion: 30% 2. Difference in proportions: 10% 3. Alpha: . 05 4. Power: . 80 41

Sample Size Example 3 Suppose the failure rate of a drug is 40%. You would like to decrease that rate to 30% with a new treatment and design a study comparing the two drugs. Using the table, how many subjects are needed in each group for an alpha level of 0. 05 and power of 0. 80? 1. Smaller proportion: 30% 2. Difference in proportions: 10% 3. Alpha: . 05 4. Power: . 80 Answer: At least 360 per group 42