Increasing Survey Statistics Precision Using Split Questionnaire Design

Increasing Survey Statistics Precision Using Split Questionnaire Design: An Application of Small Area Estimation 1

Content Introduction Split Questionnaire Design Population Characteristics Estimation • Introduction • Small Area Models • Nested Error Regression Model A Simulation Study • Steps of Procedure • Measures of Comparisons • Results 2

Introduction • Issue: – Effects of a lengthy survey questionnaire on: • Increasing response burden • declining response rate and precision of survey statistics. • A solution: Splitting the questionnaire into sub-questionnaires and assigning each one to a group of sample units. Procedure of sub-sample selection is at random, therefore, the resulting nonresponse is completely at random. The resulting nonresponse would be imputed by the common imputation methods. • Our method: – Designing and analyzing the split questionnaire, using Small Area Estimation technique. – The method is applicable where the efficient survey estimates are required. – Complete data set is not provided in our method. 3

Content Introduction Split Questionnaire Design Population Characteristics Estimation • Introduction • Small Area Models • Nested Error Regression Model A Simulation Study • Steps of Procedure • Measures of Comparisons • Results 4

Split Questionnaire Design steps: (To apply small area estimation) i. The original questionnaire is divided to (m) sub-questionnaires. Some common items as covariates are assigned to the all sub-questionnaires. Therefore, all sample units respond to them. ii. All sample units are classified with respect to a known auxiliary variable. Consequently, we make homogeneity within classes. Each class is considered as an area. iii. Sample units belong to each area randomly divided into (m) sub-samples. In each class, each sub-questionnaire is administrated to a sub-sample. iv. Step iii is repeated for all classes. Note: In each class, the number of sub-questionnaires and number of sub samples should be equal. 5

Split Questionnaire Design (To apply small area estimation) (continued) Pattern of administering subquestionnaires to sub-samples in small area estimation approach: 6

Content Introduction Split Questionnaire Design Population Characteristics Estimation • Introduction • Small Area Models • Nested Error Regression Model A Simulation Study • Steps of Procedure • Measures of Comparisons • Results 7

Population Characteristics Estimation: Introduction • There is not large enough sample to support direct estimates of appropriate precision based on the proposed design. • Small area estimation method as a solution of insufficient sample size in split questionnaire method would be useful, in order to improve the efficiency of survey statistics. 8

Population characteristics Estimation: Small Area Models • Area level models – Fay-Herriot Model – Model with Correlated Sampling Errors – … • Unit level models – Nested Error Regression Model ✓ – Random Error Variance Linear Model – … 9

Population characteristics Estimation Nested Error Regression model (Rao 2003) • 10

Population characteristics Estimation: Nested Error Regression model (Continued) (Rao 2003) • 11

Population characteristics Estimation: Nested Error Regression model (Continued) (Rao 2003) • 12

Population characteristics Estimation Nested Error Regression Model (Continued) • 13

Content Introduction Split Questionnaire Design Population Characteristics Estimation • Introduction • Small Area Models • Nested Error Regression Model A Simulation Study • Steps of Procedure • Measures of Comparisons • Results 14

A Simulation Study: Steps of Procedure • Split questionnaire design: i. ii. Creating a questionnaire with 17 questions. Splitting the questionnaire into five different components (based on iii. Considering five items (which are highly correlated with other twelve items) as a core part. Administering the core part to all sample units. Assigning three items to each component in such a way that the within component correlation is small whereas, items in different components are highly correlated. Creating 6 subquestionnaires consist of each double combination of four components plus the core part. iv. v. vi. split questionnaire design (Raghunathan and Grizzle 1995)). 15

A Simulation Study Steps of Procedure (Continued) • Data generator i. Generating a multivariate normal random vector (50, 000 times), under the described correlation pattern. ii. Producing a multinomial variable as a stratification variable which is strongly correlated with the other variables. iii. Classifying the population units based on the stratification variable. iv. Selecting a simple random sample (without replacement) of a fixed size n=2000 from the population. v. Assigning sample units in each stratum to the all 6 subquestionnaires. vi. Estimating the population mean of each item by applying multiple imputation approach using the predictive mean matching method (Rubin 1987) and the small area estimation technique. vii. Generating 1000 simulated bootstrap samples to compare two approaches. 16

A simulation Study: Measures of Comparisons • 17

A simulation Study: Measures of Comparisons (Continued) • 18

Results of the Study • The estimation of absolute relative bias, MSE and relative efficiency for 1000 bootstrap samples using sample auxiliary information 19

Results of the Study (Continued) • Absolute relative bias, MSE and relative efficiency for 1000 bootstrap samples using population auxiliary information 20

Results of the Study (Continued) • Small area estimators mostly have lower ARB respect to multiple imputation based estimators. • There were no cases in which the multiple imputation approach gave a smaller MSE than the small area method across all items. • Small area estimates are more efficient than multiple imputation approach estimates. • Small Area technique requires less computation compare to multiple imputation method. • Small Area method does not require to produce data, Hence it would be more applicable, where the goals is estimation of population auxiliary and not the improvement of data quality. 21

References 22

References (Continued) 23

Thanks for your attention… 24