Assessing and Dealing with the Impact of Imputation

  • Slides: 18
Download presentation
Assessing and Dealing with the Impact of Imputation through Variance Estimation Eric Rancourt Statistics

Assessing and Dealing with the Impact of Imputation through Variance Estimation Eric Rancourt Statistics Canada UN/ECE Work Session on Statistical Data Editing Ottawa, 16 -18 May 2005

Outline n n n Context Impact of imputation GENESIS SEVANI Outlook Conclusion

Outline n n n Context Impact of imputation GENESIS SEVANI Outlook Conclusion

Context n n Nonresponse / unusable data Imputation n Attractive (complete files, simple) Quality

Context n n Nonresponse / unusable data Imputation n Attractive (complete files, simple) Quality (auxiliary information) Impacts

Measuring the impact of Imputation n Model assessment n Simulations n Variance estimation

Measuring the impact of Imputation n Model assessment n Simulations n Variance estimation

Model assessment n Always a model! n n n Response Imputation Issues n n

Model assessment n Always a model! n n n Response Imputation Issues n n Auxiliary variables? Modelling Robustness Simplicity

Simulations n Comparisons of methods n Quantitative information n Controlled experiment n Learning tool

Simulations n Comparisons of methods n Quantitative information n Controlled experiment n Learning tool

Variance estimation Purpose n Knowledge of quality n Correct inference n Reporting to users

Variance estimation Purpose n Knowledge of quality n Correct inference n Reporting to users n Evaluation of methods n Budget (re-)allocation

Variance estimation Methods n Two-phase U S R Two approaches n Response model n

Variance estimation Methods n Two-phase U S R Two approaches n Response model n Imputation model

Variance estimation Methods n Reversed framework U UR SR Two approaches n Response model

Variance estimation Methods n Reversed framework U UR SR Two approaches n Response model n Imputation model

Variance estimation Implementation Issues n Full response method n Need for separate components n

Variance estimation Implementation Issues n Full response method n Need for separate components n Internal vs external users n Design & estimation n Simplicity

Measuring the impact of imputation at Statistics Canada Modelling: Training, Research Simulations: GENESIS Variance

Measuring the impact of imputation at Statistics Canada Modelling: Training, Research Simulations: GENESIS Variance estimation: SEVANI

GENESIS n Evaluation n Tests n Simulations

GENESIS n Evaluation n Tests n Simulations

GENESIS 3 modules: n Full response n Imputation/re-weighting classes

GENESIS 3 modules: n Full response n Imputation/re-weighting classes

SEVANI n n n Production (variance estimation) Nonresponse and Imputation Provides n n Components

SEVANI n n n Production (variance estimation) Nonresponse and Imputation Provides n n Components Relative importance

SEVANI n Any design n Broad families of imputation n Quasi-multi-phase framework (Nonresponse and/or

SEVANI n Any design n Broad families of imputation n Quasi-multi-phase framework (Nonresponse and/or imputation model)

Outlook Applications of variance estimation n Manual imputation n Editing

Outlook Applications of variance estimation n Manual imputation n Editing

Outlook Research avenues n Combining surveys and administrative data n Rolling surveys & Censuses

Outlook Research avenues n Combining surveys and administrative data n Rolling surveys & Censuses n Multi-level imputation

Conclusion n Imputation n Evaluation methods n n n Impact Model assessment Simulations Variance

Conclusion n Imputation n Evaluation methods n n n Impact Model assessment Simulations Variance estimation STC tools GENESIS, SEVANI A lot accomplished, need full implementation