response variable missing indicator variable the joint distribution
response variable missing indicator variable the joint distribution of x and the marginal distribution of the observed data r
MCARの下では,発生メカニズムは無視できる No systematic difference between complete cases and incomplete cases CC 法, 平均値の代入 unbiased estimates of underlying marginal means/profiles
Growth Curve Data (Potthoff & Roy, 1964) x 10 r 10 x 8 x 12 , x 14 means the missing produced through a MAR mechanism by
Response Propensity スコア Probability of missing based on covariate. Rosenbaum & Rubin (1983) Missing at Random and approximately
Imputation(代入法) 欠測データに何らかの値を代入 擬似的な完全データの作成 ü Marginal or Conditional imputation ü Explicit or Implicit model imputation ü Deterministic or Stochastic imputation (using random numbers) ü Univariate or Multivariate imputation ü Single or Multiple imputation
Partial loglikelihood – 欠測発生メカニズムを無視 has much simpler form than Missing at Random
EM algorithm A general algorithm for incomplete data problems that provides an interesting link with imputation methods q(k) converges to a maximum likelihood estimate of q based on Lpartial , if a unique finite MLE of q exists.
DLR(1977) E-step :To calculate the conditional expectation of Lc(q) M-step :To find q which maximize the conditional expectation calculated in the previous E-step
EM の適用(Ignorable case) 1. 欠測を含む多変量正規モデル 2. 欠測を含む多変量回帰モデル 3. 尺度混合正規モデルの下でのロバスト 4. 5. 6. 推定 Logistic 回帰( missing covariates) Unbalanced repeated-measures models with structured covariance and with missing data 潜在構造モデル
E-step :Sufficient statistics
E-step :Sufficient statistics
Imputation(代入法) 欠測データに何らかの値を代入 擬似的な完全データの作成 ü Marginal or Conditional imputation ü Explicit or Implicit model imputation ü Deterministic or Stochastic imputation (using random numbers) ü Univariate or Multivariate imputation ü Single or Multiple imputation
Mean Imputation 分布(ばらつき)を再現しない Marginal distributions and associations distorted ( no residual variance) Conditional better than unconditional Standard errors from filled-in data too small – – – no residual variance n actually smaller uncertainty of prediction Stochastic Imputation
Deterministic imputation (非確率的代入) Hot deck and Cold deck methods Overall (unconditional) mean Group (adjusted cell) mean Predictive mean by regression model More accuracy, but distort the distribution The distribution becomes too peaked and the variance is underestimated
Stochastic imputation 確率的代入 非確率的代入法+確率的要素 ばらつきを保持する(代入値の分散・共分散を意識) EX. 1. Add a random residual from N ( m , s 2 ) Stochastic Predictive mean imputation 回帰による推定値+乱数による誤差 2. Impute the value of a randomly selected case Random hot deck method
Stochastic Predictive Mean Imputation (Imputation from a Distribution) Add a random residual from N ( m , s 2 ) to the predictive mean Impute c. f. Predictive Mean Matching (more robust to misspecification) Predictive Mean Stratification & Random Hot Deck
Imputation(代入法) 欠測データに何らかの値を代入 擬似的な完全データの作成 ü Marginal or Conditional imputation ü Explicit or Implicit model imputation ü Deterministic or Stochastic imputation (using random numbers) ü Univariate or Multivariate imputation ü Single or Multiple imputation
Multiple Imputation Combined Estimator Total variability
- Slides: 58