Biostatistics statistical software IV Statistical estimation confidence intervals

Biostatistics, statistical software IV. Statistical estimation, confidence intervals. Hypothesis tests. One-and two sample t-tests. Krisztina Boda Ph. D Department of Medical Informatics, University of Szeged INTERREG

Statistical estimation n n A parameter is a number that describes the population (its value is not known). For example: § and are parameters of the normal distribution N( , ) § n, p are parameters of the binomial distribution § is parameter of the Poisson distribution n n Estimation: based on sample data, we can calculate a number that is an approximation of the corresponding parameter of the population. A point estimate is a single numerical value used to approximate the corresponding population parameter. § For example, the sample mean is an estimation of the population’s mean, . approximates Krisztina Boda INTERREG 2

Interval estimate, confidence interval n n Krisztina Boda Interval estimate: a range of values that we think includes the true value of the population parameter (with a given level of certainty). Confidence interval: an interval which contains the value of the (unknown) population parameter with high probability. The higher the probability assigned, the more confident we are that the interval does, in fact, include the true value. The probability assigned is the confidence level (generally: 0. 90, 0. 95, 0. 99 ) INTERREG 3

Interval estimate, confidence interval (cont. ) n n Krisztina Boda „high” probability: the probability assigned is the confidence level (generally: 0. 90, 0. 95, 0. 99 ). „small” probability: the „error” of the estimation (denoted by ) according to the confidence level is 1 -0. 90=0. 1, 1 -0. 95=0. 05, 1 -0. 99=0. 01 The most often used confidence level is 95% (0. 95), so the most often used value for is =0. 05 INTERREG 4

The confidence interval is based on the concept of repetition of the study under consideration http: //www. kuleuven. ac. be/ucs/java/index. htm n If the study were to be repeated 100 times, of the 100 resulting 95% confidence intervals, we would expect 95 of these to include the population parameter. Krisztina Boda INTERREG 5

Formula of the confidence interval for the population’s mean when is known n It can be shown that is a (1 - )100% confidence interval for . § u /2 is the /2 critical value of the standard normal distribution, it can be found in standard normal distribution table for =0. 05 u /2 =1. 96 for =0. 01 u /2 =2. 58 n Krisztina Boda 95%CI for the population’s mean INTERREG 6

The standard error of mean (SE or SEM) n is called the standard error of mean n Meaning: the dispersion of the sample means around the (unknown) population’s mean. When is unknown, the standard error of mean can be estimated from the sample by: n Krisztina Boda INTERREG 7

Formula of the confidence interval for the population’s mean when is unknown n n Krisztina Boda When is unknown, it can be estimated by the sample SD (standard deviation). But, if we place the sample SD in the place of , u /2 is no longer valid, it also must be replace by t /2. So is a (1 - )100 confidence interval for . t /2 is the /2 critical value of the Student's t statistic with n-1 degrees of freedom (see next slide) INTERREG 8

t-distributions (Student’s t-distributions) df=19 Krisztina Boda df=200 INTERREG 9

The Student’s t-distribution For =0. 05 and df=12, the critical value is t /2 =2. 179 Krisztina Boda INTERREG 10

Student’s t-distribution Degrees of freedom: 8 Krisztina Boda INTERREG 11

Student’s t-distribution Degrees of freedom: 10 Krisztina Boda INTERREG 12

Student’s t-distribution Degrees of freedom: 20 Krisztina Boda INTERREG 13

Student’s t-distribution Degrees of freedom: 100 Krisztina Boda INTERREG 14

Student’s t-distribution table Two sided alfa Degrees of freedom Krisztina Boda 0. 2 0. 1 0. 05 0. 02 0. 01 1 3. 077683537 6. 313752 12. 7062 31. 82052 63. 65674 2 1. 885618083 2. 919986 4. 302653 6. 964557 9. 924843 3 1. 637744352 2. 353363 3. 182446 4. 540703 5. 840909 4 1. 533206273 2. 131847 2. 776445 3. 746947 4. 604095 5 1. 475884037 2. 015048 2. 570582 3. 36493 4. 032143 6 1. 439755747 1. 94318 2. 446912 3. 142668 3. 707428 7 1. 414923928 1. 894579 2. 364624 2. 997952 3. 499483 8 1. 39681531 1. 859548 2. 306004 2. 896459 3. 355387 9 1. 383028739 1. 833113 2. 262157 2. 821438 3. 249836 10 1. 372183641 1. 812461 2. 228139 2. 763769 3. 169273 11 1. 363430318 1. 795885 2. 200985 2. 718079 3. 105807 INTERREG 15

Student’s t-distribution table two sided alfa degrees of freedom 0. 2 0. 1 0. 05 0. 02 0. 01 0. 001 1 3. 077683537 6. 313752 12. 7062 31. 82052 63. 65674 636. 6192 2 1. 885618083 2. 919986 4. 302653 6. 964557 9. 924843 31. 59905 3 1. 637744352 2. 353363 3. 182446 4. 540703 5. 840909 12. 92398 4 1. 533206273 2. 131847 2. 776445 3. 746947 4. 604095 8. 610302 5 1. 475884037 2. 015048 2. 570582 3. 36493 4. 032143 6. 868827 6 1. 439755747 1. 94318 2. 446912 3. 142668 3. 707428 5. 958816 7 1. 414923928 1. 894579 2. 364624 2. 997952 3. 499483 5. 407883 . . . … … … 100 1. 290074761 1. 660234 1. 983971 2. 364217 2. 625891 3. 390491 . . . … … … 500 1. 283247021 1. 647907 1. 96472 2. 333829 2. 585698 3. 310091 . . . … … … 1000000 1. 281552411 1. 644855 1. 959966 2. 326352 2. 575834 3. 290536 Krisztina Boda INTERREG 16

Example 1. n n n n n Krisztina Boda We wish to estimate the average number of heartbeats per minute for a certain population. The mean for a sample of 13 subjects was found to be 90, the standard deviation of the sample was SD=15. 5. Supposed that the population is normally distributed the 95 % confidence interval for : =0. 05, SD=15. 5 Degrees of freedom: df=n-1=13 -1=12 t /2 =2. 179 The lower limit is 90 – 2. 179· 15. 5/√ 13=90 -2. 179 · 4. 299=90 -9. 367=80. 6326 The upper limit is 90 + 2. 179· 15. 5/√ 13=90+2. 179 · 4. 299=90+9. 367=99. 367 The 95% confidence interval for the population mean is (80. 63, 99. 36) It means that the true (but unknown) population means lies it the interval (80. 63, 99. 36) with 0. 95 probability. We are 95% confident the true mean lies in that interval. INTERREG 17

Example 2. n n n We wish to estimate the average number of heartbeats per minute for a certain population. The mean for a sample of 36 subjects was found to be 90, the standard deviation of the sample was SD=15. 5. Supposed that the population is normally distributed the 95 % confidence interval for : =0. 05, SD=15. 5 Degrees of freedom: df=n-1=36 -1=35 t /2=2. 0301 The lower limit is 90 – 2. 0301· 15. 5/√ 36=90 -2. 0301 · 2. 5833=90 -5. 2444=84. 755 n The upper limit is 90 + 2. 0301· 15. 5/√ 36=90+2. 0301 · 2. 5833=90+5. 2444=95. 24 n n Krisztina Boda The 95% confidence interval for the population mean is (84. 76, 95. 24) It means that the true (but unknown) population means lies it the interval (84. 76, 95. 24) with 0. 95 probability. We are 95% confident the true mean lies in that interval. INTERREG 18

Example Krisztina Boda INTERREG 19

Presentation of results Krisztina Boda INTERREG 20

Hypothesis testing n n n Krisztina Boda Hypothesis: a statement about the population Based on our data (sample) we conclude to the whole phenomenon (population) We examine whether our result (difference in samples) is greater then the difference caused only by chance. INTERREG 21

Hypothesis n n Hypothesis: a statement about the population Examples § § n n Krisztina Boda H 1: =16 (the population mean is 16) H 2: ≠ 16 (the population mean is not 16) H 3: B=G (boys and girls score the same on mathematics exams) H 4: B≠G (boys and girls score differently on mathematics exams) Statisticians usually test the hypothesis which tells them what to expect by giving a specific value to work with. They refer to this hypothesis as the null hypothesis and symbolize it as H 0. The null hypothesis is often the one that assumes fairness, honesty or equality. The opposite hypothesis is called alternative hypothesis and is symbolized by Ha. This hypothesis, however, is often the one that is of interest. INTERREG 22

Steps of hypothesis-testing n n n n n Krisztina Boda Step 1. State the motivated (alternative) hypothesis Ha. Step 2. State the null hypothesis H 0. Step 3. You select the probability of „error”, or the α significance level. α =0. 05 or α =0. 01. Step 4. You choose the size n of the random sample Step 5. Select a random sample from the appropriate population and obtain your data. Step 6. Calculate the decision rule. Step 7. Decision. a) Reject the null hypothesis and claim that your alternative hypothesis was correct the difference is significant at α 100% level. b) Fail to reject the null hypothesis correct the difference is not significant at α 100% level. INTERREG 23

Testing the mean of a sample drawn from a normal population: one sample ttest n Problem, data. § The normal value of the systolic blood pressure is 120 mm Hg. § The following are the systolic blood pressures (mm Hg) of n=9 patients undergoing drug therapy for hypertension. § 182. 00 152. 00 178. 00 157. 00 194. 00 163. 00 144. 00 114. 00 174. 00 n Summary statistics § The mean=162 mm. Hg, the standard deviation SD=23. 92. n Question § Can we conclude with 95% confidence on the basis of these data that the population mean is =120? n n n Krisztina Boda HO: the population mean is 120, =120 Ha: the population mean is not 120 , 120 Assumption: the sample is drawn from a normally distributed population INTERREG 24

Decision rule based on confidence interval n n n Find the 95% CI for the above data! α=0. 05 mean=162 SD=23. 92 The standard error, SE=SD/ n=7. 97 t 8, 0. 05=2. 306) The confidence interval: (mean - t*SE, mean + t * SE )=(162 -2. 306*23. 92/ 9, 162+2. 306*7. 97)=(143. 61, 180. 386) Can we conclude with 95% confidence on the basis of these data that the population mean is =120? No, because the confidence interval does not contain 120. Decision rule based on confidence interval: is the given number (the number in the null hypothesis) in the confidence interval? § § n Krisztina Boda If yes: the difference is not significant at α level If not: the difference is significant at α level In our case 120 is not in the 95%Ci, the difference is significant at 5% level. INTERREG 25

Decision rule based on t-value n n n Calculate t= (mean - c)/SE=(162 -120)/7. 97=5. 26. Degrees of freedom: n-1=9 -1=8 Compare the absolute value of the calculated t to the critical t-value in the table: t 8, 0. 05=2. 306 5. 26>2. 306 Decision rule: § if |t|>ttable, the difference is significant at α level § if |t|<ttable, the difference is not significant at α level n n The acceptance (non-rejection) region is the set of values for which we accept the null hypothesis (- ttable, ttable) The critical region (rejection region) is the set of values for which the null hypothesis is rejected. Acceptance region t=5. 26 Krisztina Boda INTERREG 26

Decision rule based on p-value n n n Krisztina Boda The probability, computed assuming that H 0 is true, that the test statistic would take a value as extreme or more extreme than that actually observed. Decision: If p< , then the difference is significant at level If p> , then the difference is not significant at level In our case the difference is significant at 5% level, because p=0. 001<0. 05 INTERREG 27

One-sample t-test, example n n n Krisztina Boda A company produces a 16 ml bottle of some drug (solution). The bottles are filled by an automated bottle-filling process. If this process is substantially overfilling or under filling bottles, then this process must be shut down and readjusted. Overfilling results in lost profits for the company, while under filling is unfair to consumers. For a given adjustment of the bottles consider the infinite population of all the bottle fills that could potentially be produced. We let denote the mean of the infinite population of all the bottle fills. The company has decided that it will shut down and readjust the process if it can be very certain that the mean fill is above or below the desired 16 ml. Now suppose that the company observes the following sample of n=6 bottle fills: 15. 68, 16. 00, 15. 61, 15. 93, 15. 86, 15. 72 It can be verified that this sample has mean=15. 8 and standard deviation s=0. 156. Question: Is it true that the mean bottle fill in the population is 16? INTERREG 28

Krisztina Boda INTERREG 29

Krisztina Boda INTERREG 30

A one-sample t test for paired differences (paired t-test) Krisztina Boda INTERREG 31

Comparison of the means of two related samples: paired t-test n n Self-control experiment (measure the data before and after the treatment on the same patient) or Related data: § Before treatment – after treatment § Left side – right side § Matched pairs n n Krisztina Boda Null hypothesis: the two sample means are approximations of the same population mean (there is no treatment-effect, the difference is only by chance) HO: before= after or difference= 0 Alternative hypothesis: there is a treatment effect HA: before≠ after or difference≠ 0 INTERREG 32

Comparison of the means of two related samples: paired t-test (cont. ) n n n Calculation: take the difference of the two samples, calculate the mean and SE of the differences Fix The paired t-test is a one-sample t-test for the differences. The decision rule is based on: § Confidence interval for the mean difference § Calculation of t-value and comparison its absolute value with the ttable § p-value (software) Krisztina Boda INTERREG 33

Paired t-test, example n n n Krisztina Boda A study was conducted to determine weight loss, body composition, etc. in obese women before and after 12 weeks of treatment with a verylow-calorie diet. We wish to know if these data provide sufficient evidence to allow us to conclude that the treatment is effective in causing weight reduction in obese women. The mean difference is actually 4. Is it a real difference? Big or small? If the study were to be repeated, would we get the same result or less, even 0? Mean SD INTERREG Before 85 95 75 110 81 92 83 94 88 105 90. 8 10. 79 After 86 90 72 100 75 88 83 93 82 99 86. 8 9. 25 Difference -1 5 3 10 6 4 0 1 6 6 4. 3. 333 34

Paired t-test, example (cont). n n n n Krisztina Boda Idea: if the treatment is not effective, the mean sample difference is small (close to O), if it is effective, the mean difference is big. HO: before= after or difference= 0 HA: before≠ after or difference≠ 0 Let =0. Degrees of freedom=10 -1=9, t 0. 05, 9=2. 262 Mean=4, SD=3. 333 SE=3. 333/ 10=1. 054 INTERREG 35

Paired t-test, example (cont. ) n Decision based on confidence interval: § 95%CI: (4 -2. 262*1. 054, 4+2. 262*1. 054)=(1. 615, 6. 384) § If H 0 were true, 0 were inside the confidence interval § Now 0 is outside the confidence interval, the difference is significant at 5% level, the treatment was effective. § The mean loss of body weight was 4 kg, which could be even 6. 36 but minimum 1. 615, with 95% probability. Krisztina Boda INTERREG 36

n Decision based on test statistic (t-value): § t= (mean 0)/SE=mean/SE=4/1. 054=3. 7 95. This t has to be compared to the critical t-value in the table. § |t|=3. 795>2. 262(=t 0. 05, 9), the difference is significant at 5% level n Decision based on p-value: § p=0. 004, p<0. 05, the difference is significant at 5% level Acceptance region tcomputed, test statistic ttable, critical value Krisztina Boda INTERREG 37

Testing the mean of two independent samples from normal populations: twosample t-test n Independent samples: § § § n Control group, treatment group Male, female Ill, healthy Young, old etc. Assumptions: § Independent samples : x 1, x 2, …, xn and y 1, y 2, …, ym § the xi-s are distributed as N(µ 1, 1) and the yi-s are distributed as N µ 2, 2 ). n Krisztina Boda H 0: 1= 2, Ha: 1 2 INTERREG 38

n The case when the standard deviations are equal Assumptions: § 1. Both populations are approximately normal. § 2. The variances of the two populations are equal ( 1 = ). n That is the xi-s are distributed as N(µ 1, ) and the yi-s are distributed as N(µ 2, ) n H 0: 1= 2, Ha: 1 2 n If H 0 is true, then. has Student’s t distribution with n+m-2 degrees of freedom. • Decision: § If |t|>tα, n+m-2, the difference is significant at α level, we reject H 0 § If |t|<tα, n+m-2, the difference is not significant at α level, we do not reject H 0 Krisztina Boda INTERREG 39

The case when the standard deviations are not equal § Both populations are approximately normal. § 2. The variances of the two populations are not equal ( 1 1 ). § That is the xi-s are distributed as N(µ 1, 1) and the yi-s are distributed as N(µ 2, 2) n H 0: 1= 2, Ha: 1 2 n : If H 0 is true, then . has Student t distribution with df degrees of freedom. Krisztina Boda • Decision: § If |t|>tα, n+m-2, the difference is significant at α level, we reject H 0 § If |t|<tα, n+m-2, the difference is not significant at α level, we do not reject H 0 INTERREG 40

Comparison of the variances of two normal populations: F-test n n H 0: 1= 2 Ha: 1 > 2 (one sided test) F: the higher variance divided by the smaller variance: Degrees of freedom: § 1. Sample size of the nominator-1 § 2. Sample size of the denominator-1 n Decision based on F-table § If F>Fα, table, the twp variances are significantly different at α level Krisztina Boda INTERREG 41

Table of the F-distribution α=0. 05 Nominator-> Denominator| Krisztina Boda INTERREG 42

Example Krisztina Boda INTERREG 43

Result of SPSS Krisztina Boda INTERREG 44

Two sample t-test, example. n n n Krisztina Boda A study was conducted to determine weight loss, body composition, etc. in obese women before and after 12 weeks in two groups: Group I. treatment with a very-low-calorie diet. Group II. no diet Volunteers were randomly assigned to one of these groups. We wish to know if these data provide sufficient evidence to allow us to conclude that the treatment is effective in causing weight reduction in obese women compared to no treatment. INTERREG 45

Two sample t-test, cont. Data Krisztina Boda INTERREG 46

Two sample t-test, example, cont. n n n HO: diet= control, (the mean change in body weights are the same in populations) Ha: diet control (the mean change in body weights are different in the populations) Assumptions: § normality (now it cannot be checked because of small sample size) § Equality of variances (check: visually compare the two standard deviations) Krisztina Boda INTERREG 47

Two sample t-test, example, cont. n Assuming equal variances, compute the t test- statistic: t==2. 477 n Degrees of freedom: 10+11 -2=19 Critical t-value: t 0. 05, 19=2. 093 Comparison and decision: n n § |t|=2. 477>2. 093(=t 0. 05, 19), the difference is significant at 5% level n Krisztina Boda p=0. 023<0. 05 the difference is significant at 5% level INTERREG 48

Example from the literature Circulation, 2004; 109: 1212 -1214. Krisztina Boda INTERREG 49

Krisztina Boda INTERREG 50

Step 1. H 0, HA n n n n Krisztina Boda Basic Question is: Are these two groups different? Formalized as H 0: p 1 = p 2 (null, no difference) HA: p 1 ≠ p 2 (alternative, it is what we want to prove) or H 0: 1 = 2 (null, no difference) HA: 1 ≠ 2 (alternative, it is what we want to prove) The formalization of H 0 and HA depends on the data type and on the measure of the endpoint. INTERREG 51

Step 2. Fix is generally chosen to be 0. 05 n It represents the mistake of our later decision when reject H 0. n Krisztina Boda INTERREG 52

Step 3. Decide the sample size n n The sample size depends on… § The objective(s) of the research (estimation, hypothesis test, …) § Main outcome: categorical or continuous; one or more, primary or secondary, and the estimation of the distribution of the outcome – based on earlier trials. § Probability of type I. error, - see later § The power of the test (1 - ) (1 - probability of type II. error) –see later § The research design § effect size of clinical importance § … Krisztina Boda INTERREG 53

Step 4. Collect data (perform the experiment) Krisztina Boda INTERREG 54

Step 5. Decision rule n n n Krisztina Boda Test statistic: the summary statistics (signal/noise) of the data used in decision making between H 0 and Ha. Compute the t-statistic. When the null hypothesis is true, the probability distribution of the test statistic (null distribution) is known. Based on the null distribution, the critical values can be found according to the given . . the p-value can be computed (probability of the observed test statistic as is or more extreme in either direction when the null hypothesis is true). INTERREG 55

Step 4. Decision. n Decision based on p-value § If p<α: statistical significance at α level. H 0 is rejected in favour of Ha. § If p≥ α : non-significance at α level. H 0 cannot be rejected. n Decision based on test statistic § compare the computed t-value to the critical tvalue. The difference is significant, if the absolute value of the test statistics > critical t-value n Krisztina Boda Decision based on confidence interval § the confidence interval does not contain 0 INTERREG 56

Compare the mean age in the two groups! The sample means are „similar”. Is this small difference really caused by chance? Krisztina Boda INTERREG 57

n Step 1. § H 0: the means in the two populations are equal: 1= 2 § HA: the means in the two populations are not equal: 1≠ 2 n Step 2. § Let α=0. 05 n Step 3. § Decision rule: two-sample t-test. n Step 4. Decision. § Decision based on test statistic: n n n Compute the test statistics: t=-1. 059, the degrees of freedom is 14+13 -2=25 ttable=2. 059 |t|=1. 059<2. 059, the difference is not significant at 5% level. § p=0. 28, p>0. 05, the difference is not significant at 5% level. Krisztina Boda INTERREG 58

How to get the p-value? n n n If H 0 is true, the computed test statistic has a t-distribution with 25 degrees of freedom. Then with 95% probability, the t-value lies in the „acceptance region” Check it: now t=-1. 059 0. 025 0. 95 0. 025 ttable, critical value Krisztina Boda INTERREG 59

How to get the p-value? n n If H 0 is true, the computed test statistic has a t-distribution with 25 degrees of freedom Then with 95% probability, the t-value lies in the „acceptance region” Check it: now t=-1. 059 The p-value is the shaded area, p=0. 28. The probability of the observed test statistic as is or more extreme in either direction when the null hypothesis is true. 0. 025 ttable, critical value Krisztina Boda INTERREG 0. 95 0. 025 tcomputed, test statistic 60

Review questions and exercises n Problems to be solved by hand-calculations §. . HandoutsProblems hand IV. DOC, §. . HandoutsPractice_t-test. doc n Solutions §. . HandoutsProblems hand IV solutions. DOC, §. . HandoutsPractice_t-test, solutions. doc n Problems to be solved using computer §. . HandoutsPRACT 4 M. DOC Krisztina Boda INTERREG 61

Main tests to compare two groups n Comparing means, normality supposed (t-tests) § Paired samples (before-after treatment, left side-right side, etc. ): paired t-test § Independent samples (control-treatment, treatment 1 – treatment 2, etc): two-sample t-test n Comparing means medians, normality not supposed : nonparametric tests, e. g. tests based on ranks § Paired samples : Wilcoxon test § Independent samples Mann-Whitney U-test n Comparing „percentages” (frequencies): § Paired samples : chi-squared test § Independent samples: Mc. Nemar test Krisztina Boda INTERREG 62