Achieving Quality and Efficiency Using a Topdown Approach

Achieving Quality and Efficiency Using a Top-down Approach in the Canadian Integrated Business Statistics Program UNECE - Work Session on Statistical Editing (Topic iii – Macro Editing Methods) Serge Godbout, Yanick Beaucage and Claude Turmelle, Statistics Canada Ljubljana, Slovenia, May 9 th 2011

Integrated Business Statistics Program – What do we need? § Quality Indicators = Focal point of the Common Editing Strategy and the Rolling Estimate model § In order to decide when to stop collection Quality Indicators • Sufficient weighted response rate • Minimum risk of non-response bias • Acceptable CVs (below a target CV) § In order to continue collection Measure of Impact Scores • List of influential non-respondents • List of influential respondents with failed edits Ø Influential : in terms of variance, bias or estimates § In order to minimize analysis work • List of respondents influencing estimates and quality § We need quality indicators and scores 2 Statistics Canada • Statistique Canada 2021 -09 -13

Quality Indicators § Measure the quality for a given variable and domain § Can be item-based • Variance, CV or imputation rate of a variable of interest § Can be covariate-based • Response rates, Response representativeness (R-indicator, Schouten et al, 2009) Measure of Impact (MI) Scores § Estimate-related or quality-related score functions § Measure the impact of a unit on estimate or quality 3 Statistics Canada • Statistique Canada 2021 -09 -13

Estimate-related MI Scores § MI Scores on the estimated total: Consistent with the predicted difference in estimates (Hedlin, 2008) § Requires to impute for respondents as well as nonrespondents Quality-related MI Scores § MI Scores from item-based quality indicators • Estimated sampling variance for expansion estimators (Under a stratified Bernoulli sampling design) 4 Statistics Canada • Statistique Canada 2021 -09 -13

Quality-related MI Scores § MI Scores from covariate-based quality indicators § Estimate the units response propensities • From covariates, including paradata § Measure the estimated MSE: • Standardized maximum bias • Standardized variance (or item-based variance) § Predicted response propensity • Derived from estimated model with updated covariates (paradata) • Predict and MI Scores 5 Statistics Canada • Statistique Canada 2021 -09 -13

Managing Response Representativeness • Goal: Increase the response propensities average while improving their homogeneity. 6 Statistics Canada • Statistique Canada 2021 -09 -13

Managing Response Representativeness • Goal: Increase the response propensities average while improving their homogeneity. 7 Statistics Canada • Statistique Canada 2021 -09 -13

Managing Response Representativeness • Goal: Increase the response propensities average while improving their homogeneity. 8 Statistics Canada • Statistique Canada 2021 -09 -13

Active Management § A large number of variables of interest to monitor • Monitoring all of them is a challenge • Not equally important § Identify a limited number of key variables • For each key variable • Identify relevant item-based quality indicator(s) and MI Score(s) • May set targets • For the non-key variables • Manage response representativeness using covariate-based quality indicator(s) and MI Score(s) • Global MI Score • Combination of the MI scores (maximum, average or other) 9 Statistics Canada • Statistique Canada 2021 -09 -13

Future Work § Methodology development • Global MI score • Response propensity model • In order to build a quality indicator for bias • Based on auxiliary data and paradata § Strategy validation • Conduct simulation studies • Develop a prototype of the Rolling Estimates • Using weighted response rate and variance as quality indicators § Evaluation • Recommendations for relevant and consistent quality indicators and MI scores 10 Statistics Canada • Statistique Canada 2021 -09 -13

• For more information, please contact: Pour plus d’information, veuillez contacter: Serge. Godbout@statcan. gc. ca Yanick. Beaucage@statcan. gc. ca Claude. Turmelle@statcan. gc. ca 11 Statistics Canada • Statistique Canada 2021 -09 -13