Sampling WLS and Mixed Models II ESAMP Meetings

Sampling, WLS, and Mixed Models II ESAMP Meetings Nov 6, 2009 Natal, Brazil Ed Stanek and Julio Singer U of Mass, Amherst, and U of Sao Paulo, Brazil SPH&HS, UMASS Amherst 1

Finite Population Mixed Models Research Group Luz Mery Gonzalez, Columbia; Viviana Lencina, Argentina; Julio Singer, Brazil; Silvina San Martino, Argentina; Wenjun Li, US; and Ed Stanek US 2

Background n Motivation: – 2 -stage cluster sample of hospitals § n Hospitals – m Appendectomy operations per hospital – What is the average cost of an operation at a selected hospital (latent value)? n Choices: – Use average cost of m operations for selected hospital – Use ‘shrunk’ cost- regressing to the mean for other sample hospitals. n Which should we use?

How do we make up models to get better insight from limited information? • Consider/Account for: Study Design Sampling Response Error • Model Assumptions An Example What is a subject’s saturated fat intake? SPH&HS, UMASS Amherst 4

Seasons Study UMASS Worc SPH&HS, UMASS Amherst 5

Seasons Study UMASS Worcfocus on 3 subjects SPH&HS, UMASS Amherst 6

The Problem-Simplified n Observe: n Assume: n Question: n Begin with a Response Error Model … which leads to…. – 1 Measure of SFat on each Subject – Response Error (RE) Variance known – How do we estimate Subject’s True Sat Fat intake? – – Mixed Model Finite Population Mixed Model Daisy Lily SPH&HS, UMASS Amherst Rose 7

Population SPH&HS, UMASS Amherst 8

Population Set SPH&HS, UMASS Amherst 9

Response 4 11 SPH&HS, UMASS Amherst 10

Response 0 11 SPH&HS, UMASS Amherst 11

Response 4 9 SPH&HS, UMASS Amherst 12

Response 4 11 SPH&HS, UMASS Amherst 13

Response 4 9 SPH&HS, UMASS Amherst 14

Response 0 11 SPH&HS, UMASS Amherst 15

Response……. . 04 11 SPH&HS, UMASS Amherst 16

Response Error Model for Set 9 11 0. 5 SPH&HS, UMASS Amherst 0 4 0. 5 17

Summary Response Error Model Latent Value SPH&HS, UMASS Amherst 18

Re-parameterized RE Model Mean Latent Value-of what? : SPH&HS, UMASS Amherst or the Set Population 19

Generating Response in the RE Model SPH&HS, UMASS Amherst 20

Generating Response in the RE Model SPH&HS, UMASS Amherst 21

Generating an Observed Response in the RE Model -1 1 -1 SPH&HS, UMASS Amherst -2 22

Sample Space n Response Error Model 4 9 9 SPH&HS, UMASS Amherst 0 11 11 4 0 23

Response Error Model 9 4 11 4 9 0 11 0 SPH&HS, UMASS Amherst 24

Mixed Model (MM) Random Effect SPH&HS, UMASS Amherst 25

Mixed Model (MM) Latent Value SPH&HS, UMASS Amherst 26

Mixed Model (MM) in action SPH&HS, UMASS Amherst 27

Mixed Model (MM) …. SPH&HS, UMASS Amherst 28

Mixed Model (MM) …. ? ? ? Who Are They? SPH&HS, UMASS Amherst ? 29

Mixed Model (MM) ? ? ? What Does it Mean? SPH&HS, UMASS Amherst ? 30

Sample Space (MM) Artificial 1 8 3 8 1 12 3 12 4 9 Real 9 0 SPH&HS, UMASS Amherst 11 11 4 0 31

MM-Latent Values? Daisy (j=1) 3 Samples 1 3 1 11 9 2 2 10 10 Rose (j=2) 1 12 -1 12 1 8 -1 8 1 4 -1 4 1 0 -1 0 SPH&HS, UMASS Amherst 10 10 2 2 2 -2 -2 32

What are they (for Daisy)? Daisy (j=1) 3 Samples 1 3 1 11 9 2 2 10 10 Rose (j=2) 1 12 -1 12 1 8 -1 8 1 4 -1 4 1 0 -1 0 SPH&HS, UMASS Amherst 10 10 2 2 2 -2 -2 33

What are they (for Rose)? Daisy (j=1) 3 Samples 1 3 1 11 9 2 2 10 10 Rose (j=2) 1 12 -1 12 1 8 -1 8 1 4 -1 4 1 0 -1 0 SPH&HS, UMASS Amherst 10 10 2 2 2 -2 -2 34

BLUPs of the MM-Latent Value SPH&HS, UMASS Amherst 35

MSE of BLUPs for MM-Latent Values Samples Daisy (j=1) 3. 1 1. 2 3. 1 10. 9 8. 9 10. 8 8. 9 2 2 10 10 0. 81 1. 15 0. 71 1. 28 0. 71 1. 15 0. 81 Ave=0. 986 Rose (j=2) 11. 5 11. 4 7. 7 7. 6 4. 4 4. 3 0. 64 0. 52 SPH&HS, UMASS Amherst 10 10 2 2 2. 19 1. 86 5. 24 5. 79 5. 24 1. 86 2. 19 Ave=3. 768 36

MSE of BLUPs |P=y Samples Daisy (j=1) 10. 90 10 0. 81 8. 93 10 1. 15 10. 84 10 0. 71 8. 87 10 1. 28 MSE=Ave=0. 986 Rose (j=2) 4. 41 2 5. 79 4. 29 2 5. 24 0. 64 2 1. 86 0. 52 2 2. 19 MSE=Ave=3. 768 SPH&HS, UMASS Amherst 37

Finite Population Mixed Model (FPMM) Population SPH&HS, UMASS Amherst 38

Response Error Model Latent Value SPH&HS, UMASS Amherst 39

Accounting for Sampling Indicator random variable, 1 if ith Selected sample subject is subject “s”

Finite Population Mixed Model (FPMM) SPH&HS, UMASS Amherst 41

Finite Population Mixed Model (FPMM) SPH&HS, UMASS Amherst 42

Finite Population Mixed Model (FPMM) SPH&HS, UMASS Amherst 43

Finite Population Mixed Model (FPMM) SPH&HS, UMASS Amherst 44

FPMM- Sample Space … 4 9 9 0 4 11 11 0 45

FPMM- Sample Space … 0 11 4 11 0 9 4 9 SPH&HS, UMASS Amherst 46

FPMM- Sample Space … 4 -7 -7 0 SPH&HS, UMASS Amherst 13 13 4 0 47

FPMM- Sample Space … 13 0 0 -7 SPH&HS, UMASS Amherst 4 4 13 -7 48

FPMM- Sample Space … 11 -7 -7 9 SPH&HS, UMASS Amherst 13 13 11 9 49

FPMM- Sample Space … 13 9 9 -7 SPH&HS, UMASS Amherst 11 11 13 -7 50

FPMM- Sample Space All sample points are Potentially Observable 11 11 -7 1 3 9 9 -7 13 11 13 9 -711 -7 11 11 0 4 0 94 9 9 0 4110 4 9 11 SPH&HS, UMASS Amherst 4130 4 -7 0 -7 13 134 -713 0 -7 0 4 51

FPMM- BLUPs of Realized Latent Values SPH&HS, UMASS Amherst 52

FPMM- BLUPs of Realized Latent Values SPH&HS, UMASS Amherst 53

FPMM- BLUPs of Realized Latent Values SPH&HS, UMASS Amherst 54

FPMM- BLUPs of Realized Latent Values Sample Sequence SPH&HS, UMASS Amherst 55

Comparison of MM-BLUP and FPMM -BLUP Target Random Variable MM-BLUP FPMM-BLUP MM-Latent Value SPH&HS, UMASS Amherst 56

Comparison of MM-BLUP and FPMM -BLUP MM-BLUP FPMM-BLUP Predictor SPH&HS, UMASS Amherst 57

Comparison of FPMM-BLUP and MM -BLUP-Sample Space l Ar a i c i f ti 12 3 812 1 8 1 3 11 11 -7 1 3 9 9 -7 13 11 13 9 -711 -7 9 0 4110 4 9 11 114 911 0 9 0 4 SPH&HS, UMASS Amherst 4130 4 -7 0 -7 13 134 -713 0 -7 0 4 58

To Compare, Focus on …THIS Sample Space 12 3 812 1 8 1 3 11 11 -7 1 3 9 9 -7 13 11 13 9 -711 -7 9 0 4110 4 9 11 114 911 0 9 0 4 SPH&HS, UMASS Amherst 4130 4 -7 0 -7 13 134 -713 0 -7 0 4 59

Bigger Sample (n=3) Population (N=4) SPH&HS, UMASS Amherst 60

n=3, What is Lily’s Latent value? • Use n=3 subject effects for MM 1 possible sample set 0 11 9 -7 4 0 11 9 SPH&HS, UMASS Amherst 4 4 130 11 11 3 3 4 -7 130 11 -7 -7 11 61

n=3, What is Lily’s Latent value? • 8 sample points 11 -71311 SPH&HS, UMASS Amherst 62

n=3, What is Lily’s Latent value? • 8 x(6 permutations)=48 sample points SPH&HS, UMASS Amherst 63

n=3, What is Lily’s Latent value? Combinations SPH&HS, UMASS Amherst 64

n=3, What is Lily’s Latent value? 192 Sample Points SPH&HS, UMASS Amherst 65

Select one sequence SPH&HS, UMASS Amherst 66

Select one sequence, Observe Sample Point SPH&HS, UMASS Amherst 67

FPMM-Average MSE of Predictor over Permutations SPH&HS, UMASS Amherst 68

Ave MSE 5. 0 X MM 11 4. 6 11 11 13 11 -7 11 16. 2 FPMM SPH&HS, UMASS Amherst 69

Ave MSE 29. 4 X MM 11 11 17. 7 11 13 11 -7 11 34. 3 FPMM SPH&HS, UMASS Amherst 70

Summary MSE Results j=3 Rose Target Mean Daisy Lily Rose MM MSE 2. 667 0. 9931 12. 3195 3. 7561 Lily Violet Mean Daisy Lily Violet 7. 409 0. 993 17. 765 18. 929 14. 000 18. 362 34. 311 18. 487 Daisy Rose Violet Mean Daisy Rose Violet 2. 464 0. 994 3. 540 13. 563 3. 333 3. 647 3. 304 17. 224 Lily Rose Violet Mean Lily Rose Violet 3. 066 4. 593 3. 345 4. 147 14. 333 16. 177 13. 751 15. 027 Set 1 1 j=1 Daisy 2 2 Daisy 3 3 4 4 Sample Set j=2 Lily FPMM MSE 11. 667 15. 679 34. 165 9. 785

Population Conclusions Design Based 11 FPMM-BLUP Sample Space 11 11 SPH&HS, UMASS Amherst 13 11 -7 11 72

Population Conclusions Evaluate Performance Conditional on the Sample FPMM-BLUP Design Based 11 11 11 SPH&HS, UMASS Amherst 13 11 -7 11 73

Conclusions Conceptual “Priors” Model Based MM-BLUP SPH&HS, UMASS Amherst 13 11 -7 11 74

Conclusions Evaluate Performance Conditional on the Sample Model Based MM-BLUP SPH&HS, UMASS Amherst 13 11 -7 11 75

Conclusions n To Evaluate Performance of BLUP Estimators: – For Mixed Model: Condition on P=y § i. e. MM Latent Values match subject Latent Values – For the FPMM: Condition on the sample set n MSE for BLUPs not evaluated Correctly – Extends to WLS estimate of mean n MM-BLUP not always best SPH&HS, UMASS Amherst 13 11 -7 11 76

Thanks SPH&HS, UMASS Amherst 77

Any thoughts? Next steps? Questions? SPH&HS, UMASS Amherst 78