Multivariate Statistical Methods Tests of Hypotheses of Means

Multivariate Statistical Methods Tests of Hypotheses of Means by Jen-pei Liu, Ph. D Division of Biometry, Department of Agronomy, National Taiwan University and Division of Biostatistics and Bioinformatics National Health Research Institutes 12/25/2021 Copyright by Jen-pei Liu, Ph. D 1

Tests of Hypotheses of Means n n n n Introduction Review of Univariate One-sample t Test Multivariate One-sample T 2 Test Review of Univariate Two-sample t Test Multivariate Two-sample T 2 Test Profile Analysis Summary 12/25/2021 Copyright by Jen-pei Liu, Ph. D 2

Introduction n Wechsler Adult Intelligence Scale scores of 101 elderly men and women (60 -64, years old, Morrison, 2005) Item verbal Performance 12/25/2021 Mean Variance 55. 24 210. 54 34. 97 119. 68 Copyright by Jen-pei Liu, Ph. D Covariance 126. 99 3

Introduction n The mean scores for the population between 18 and 59 are 60 and 50 for verbal and performance items Univariate question: Is the mean score of verbal item for elderly population different from that of the population between 18 and 59? Multivariate question: Is the Wechsler Adult Intelligence Scale scores for elderly population different from that of the population between 18 and 59? 12/25/2021 Copyright by Jen-pei Liu, Ph. D 4

Introduction Survivors and non-survivors for Bumpus’s female sparrows Survivors Variables mean Total length 157. 38 Alar extent 241. 00 Length of bead and head 31. 43 Length of humerus 18. 50 Length of keel of sterum 20. 81 12/25/2021 variance 11. 05 17. 50 0. 53 0. 18 0. 58 Copyright by Jen-pei Liu, Ph. D Nonsurvivors Mean 158. 43 241. 57 31. 48 18. 45 20. 84 Variance 15. 07 32. 55 0. 73 0. 43 1. 32 5

Introduction n n Univariate questions: Does difference exist in each of the five morphological measurements between the survivors and non-survivors? Multivariate questions: Is there any difference in morphology between the survivors and nonsurvivors? 12/25/2021 Copyright by Jen-pei Liu, Ph. D 6

Review of Univariate One-sample t test n Hypothesis of mean: H 0: v = o vs. Ha: v o 12/25/2021 Copyright by Jen-pei Liu, Ph. D 7

Review of Univariate One-sample t test n Wechsler Adult Intelligence Scale (WAIS) verbal scores 12/25/2021 Copyright by Jen-pei Liu, Ph. D 8

Review of Univariate One-sample t test n n Univaraite t 2 follow a F distribution with 1 and n-1 d. f. t and t 2 are unitless n n Example of verbal scores t 2 = (-4. 76)2/(210. 54/101) = (-4. 76)(210. 54/101)-1(-4. 76) = 10. 8695 > F 0. 01, 1, 100 = 6. 76 12/25/2021 Copyright by Jen-pei Liu, Ph. D 9

Review of Univariate One-sample test n Properties n It is the uniformly most powerful unbiased test for mean H 0: v = o vs. Ha: v o under the normal assumption when the variance is unknown in sense the probability of rejecting H 0 is less than or equal to if v = o and is at least if v o 12/25/2021 Copyright by Jen-pei Liu, Ph. D 10

Multivariate One-sample T 2 Test n Multivariate Hypothesis of One-sample Means 12/25/2021 Copyright by Jen-pei Liu, Ph. D 11

Multivariate One-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 12

Multivariate One-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 13

Multivariate One-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 14

Multivariate One-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 15

Multivariate One-sample T 2 Test n Union-intersection principle a’X ~ N 1(a’ , a’ a) n Univariate hypothesis H 0(a): a’ = a’ 0 vs. Ha(a): a’ 0 n 12/25/2021 Copyright by Jen-pei Liu, Ph. D 16

Multivariate One-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 17

Multivariate One-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 18

Multivariate One-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 19

Multivariate One-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 20

Multivariate One-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 21

Multivariate One-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 22

Multivariate One-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 23

Multivariate One-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 24

Multivariate One-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 25

Multivariate One-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 26

Multivariate One-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 27

Multivariate One-sample T 2 Test n Test Statistic n Hotelling T 2 Statistic: a generalization of univariate t statistic by union-intersection principle 12/25/2021 Copyright by Jen-pei Liu, Ph. D 28

Multivariate One-sample T 2 Test n Confidence Region and Simultaneous CIs 12/25/2021 Copyright by Jen-pei Liu, Ph. D 29

Multivariate One-sample T 2 Test n Wechsler Adult Intelligence Scale (WAIS) scores 12/25/2021 Copyright by Jen-pei Liu, Ph. D 30

Multivariate One-sample T 2 Test n Wechsler Adult Intelligence Scale verbal scores 12/25/2021 Copyright by Jen-pei Liu, Ph. D 31

Multivariate One-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 32

Multivariate One-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 33

Multivariate One-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 34

Review of Univariate Two-sample t Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 35

Review of Univariate Two-sample t Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 36

Review of Univariate Two-sample t Test n Example: The total length of female sparrows between survivors and nonsuvivors Statistics Sample size Mean Variance 12/25/2021 Survivors 21 157. 38 11. 05 Nonsurvivors 28 158. 43 15. 07 Copyright by Jen-pei Liu, Ph. D 37

Review of Univariate Two-sample t Test n Example: The total length of female sparrows between survivors and nonsuvivors 12/25/2021 Copyright by Jen-pei Liu, Ph. D 38

Multivariate Two-sample T 2 Test n Multivariate Hypothesis of Two-sample Means 12/25/2021 Copyright by Jen-pei Liu, Ph. D 39

Multivariate Two-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 40

Multivariate Two-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 41

Multivariate Two-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 42

Multivariate Two-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 43

Multivariate Two-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 44

Multivariate Two-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 45

Multivariate Two-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 46

Multivariate Two-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 47

Multivariate Two-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 48

Multivariate Two-sample T 2 Test n The Paired T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 49

Multivariate Two-sample T 2 Test n The Paired T 2 Test n n n Take the difference in the corresponding vectors of measurements within each object between the two conditions Obtain the sample mean difference vector and sample covariance matrix Apply one-sample T 2 test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 50

Multivariate Two-sample T 2 Test n The Paired T 2 Test – Example Dataset: Measurement of uraic acid (X 1) and total cholesterol level (X 2) before and after the treatment for six patients Before After Difference Pat. No. X 11 x 21 X 12 X 22 d 1 d 2 1 12. 5 220 6. 5 190 -6. 0 -30 2 14. 2 260 7. 8 250 -6. 4 -10 3 10. 8 180 6. 0 190 -4. 8 10 4 13. 4 200 7. 2 220 -6. 2 20 5 11. 9 280 6. 8 240 -5. 1 -40 6 12. 0 170 5. 9 180 -6. 1 -10 12/25/2021 Copyright by Jen-pei Liu, Ph. D 51

Multivariate Two-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 52

Multivariate Two-sample T 2 Test 12/25/2021 Copyright by Jen-pei Liu, Ph. D 53

Profile Analysis Forty-nine elderly men were classified into “senile factor present” (SFP, n 1=12) and “no senile factor” (NSF, n 2=37). The Wechsler Adult Intelligence Scale (WAIS) was administered to all subjects. The WAIS consists of four domains: information, similarity, arithmetic and picture completion. The sample means are given below (Morrison, 2005): Group NSF SFP Domain n 1 = 37 information 12. 57 similarity 9. 57 arithmetic 11. 49 picture completion 7. 97 12/25/2021 n 2 = 12 8. 75 5. 33 8. 50 4. 75 Copyright by Jen-pei Liu, Ph. D 54

Profile Analysis 12/25/2021 Copyright by Jen-pei Liu, Ph. D 55

Profile Analysis n n The two-sample Hotelling T 2 statistic is 22. 13 > F 0. 05, 4, 44 = 5. 18 The 95% simultaneous CI Domain CI Information ( 0. 12, 7. 52) Similarity ( 0. 18, 8. 30) Arithmetic (-0. 77, 6. 75) Picture Completion ( 0. 56, 5. 88) 12/25/2021 Copyright by Jen-pei Liu, Ph. D 56

Profile Analysis n Assumptions: n n n The same battery of psychological tests Measurements of continuous random variables The responses are commensurable or in comparable units 12/25/2021 Copyright by Jen-pei Liu, Ph. D 57

Profile Analysis 12/25/2021 Copyright by Jen-pei Liu, Ph. D 58

Profile Analysis n Three questions (in terms of priority): n n n Are the population mean profiles similar, in the sense that the line segments of adjacent tests are parallel? (Test for parallelism: response-by-group interaction) If the two population profiles are indeed parallel, are they also at the same level? (equal group effects) Again assuming parallelism, are the population means of the tests different? 12/25/2021 Copyright by Jen-pei Liu, Ph. D 59

Profile Analysis n n Parallelism: (response-by-group interaction) The slopes of the population profile segments are the same under each condition 12/25/2021 Copyright by Jen-pei Liu, Ph. D 60

Profile Analysis 12/25/2021 Copyright by Jen-pei Liu, Ph. D 61

Profile Analysis 12/25/2021 Copyright by Jen-pei Liu, Ph. D 62

Profile Analysis If the null hypothesis of parallelism is not rejected at the significance level, we can test the hypothesis of the same level 12/25/2021 Copyright by Jen-pei Liu, Ph. D 63

Profile Analysis If the null hypothesis of parallelism is not rejected at the significance level, we can test the hypothesis of the equal response 12/25/2021 Copyright by Jen-pei Liu, Ph. D 64

Profile Analysis n Example: WAIS Dataset n Hypothesis of parallelism 12/25/2021 Copyright by Jen-pei Liu, Ph. D 65

Profile Analysis 12/25/2021 Copyright by Jen-pei Liu, Ph. D 66

Profile Analysis n Example: WAIS Dataset n Hypothesis of the same level 12/25/2021 Copyright by Jen-pei Liu, Ph. D 67

Profile Analysis n Example: WAIS Dataset n Hypothesis of the equal mean responses 12/25/2021 Copyright by Jen-pei Liu, Ph. D 68

Profile Analysis n Example: WAIS Dataset n Hypothesis of the equal mean responses 12/25/2021 Copyright by Jen-pei Liu, Ph. D 69

Assumptions n n n Multivariate normality Equal covariance matrices Independent samples 12/25/2021 Copyright by Jen-pei Liu, Ph. D 70

Homogeneity of Covariance Matrix Hypothesis of Equal Covariance Matrices of m p-dimensional multivariate normal Distributions Ho: 1 = …= m 12/25/2021 Copyright by Jen-pei Liu, Ph. D 71

Homogeneity of Covariance Matrix 12/25/2021 Copyright by Jen-pei Liu, Ph. D 72

Homogeneity of Covariance Matrix 12/25/2021 Copyright by Jen-pei Liu, Ph. D 73

Homogeneity of Covariance Matrix n Reaction Times 32 male and 32 female normal subjects reacted to visual stimuli preceded by warning intervals of different lengths in 0. 5 and 0. 15 seconds 12/25/2021 Copyright by Jen-pei Liu, Ph. D 74

Homogeneity of Covariance Matrix 12/25/2021 Copyright by Jen-pei Liu, Ph. D 75

Multivariate Normality n n n Calculate squared Mahalanobis distance (MD 2) of each observed vector of p variables from the sample mean vector Order the MD 2 from the largest to the largest, MD 2(1) < MD 2(2) <…< MD 2(n) For each ordered MD 2, compute the (i-0. 5)/n percentile where i is the ith order observed vector 12/25/2021 Copyright by Jen-pei Liu, Ph. D 76

Multivariate Normality n n The 2 values for the percentiles are obtained from the 2 with p d. f. , which can be computed by CINV function in SAS Plot MD 2 vs. 2 (similar to Q-Q plot) The plot should be linear Deviation from linearity indicates nonnormality 12/25/2021 Copyright by Jen-pei Liu, Ph. D 77

Multivariate Normality n n Several tests available, see Seber (1984). However, methods are ad-hoc and are not implemented in most of statistical software Transformations n n n Counts – square-root Proportions – logit Skewed and positive - logarithm 12/25/2021 Copyright by Jen-pei Liu, Ph. D 78

Multivariate Outliers 12/25/2021 Copyright by Jen-pei Liu, Ph. D 79

Multivariate Outliers 12/25/2021 Copyright by Jen-pei Liu, Ph. D 80

Multivariate Outliers n n Hotelling T 2 is invariant under any fullrank linear transformation. The joint distribution of {T 2 i, i=1, …, n} is independent of parameter and Liu and Weng (1991, SIM) proposed a test procedure to identify multiple multivariate outliers 12/25/2021 Copyright by Jen-pei Liu, Ph. D 81

Multivariate Outliers n n Let T 2(1) , …, T 2(n) be the ordered statistics of T 21 , …, T 2 n and H 0(1) , …, H 0(n) be the corresponding sub-hypothesis based on T 2(i). Let {W 21 , …, W 2 n} be a vector of n Hotelling T 2 statistics computed from a sample of size n from a p-dimensional normal with mean 0 and covariance matrix Ip 12/25/2021 Copyright by Jen-pei Liu, Ph. D 82

Multivariate Outliers 12/25/2021 Copyright by Jen-pei Liu, Ph. D 83

Summary n Hotelling T 2 statistics n n n One sample Two independent samples (unpaired) Paired samples Confidence regions Simultaneous confidence interval Profile analysis 12/25/2021 Copyright by Jen-pei Liu, Ph. D 84