Sources of Measurement Errors in Business Surveys Mojca

Sources of Measurement Errors in Business Surveys Mojca Bavdaz

Outline • Measurement errors & their sources. • Empirical study. • Sources of measurement errors redefined. • Implications.

Measurement errors • ‘Theoretical’ definition of a measurement error: – ‘deviation’ of the observed survey value from the true value (Groves, 1991); – ‘difference’ or ‘discrepancy’ between the observed survey value and the true value (Hansen et al. , 1951; Sukhatme and Sukhatme, 1970). • ‘Practical’ definition of a measurement error: – observational gap between the ideal measurement and the response obtained (Groves et al. , 2004); – error which occurs at the time of data collection (Biemer and Lyberg, 2003).

Sources of measurement errors (1) • In household surveys (e. g. Groves, 1989): – – respondent; interviewer; instrument (the survey questionnaire); mode of data collection.

Sources of measurement errors (2) • In business surveys, the addition of the fifth source: – records (Ponikowski, Meily, 1989); – information system (Biemer, Fecso, 1995; Biemer, Lyberg, 2003); – organization (O'Brien, 2000).

Empirical study • Business survey under study: – Quarterly Business Survey on Trade (SORS). • Aim: – examination of actual response process. • Research methods: – in-depth interview; – observation. • Sample size (n=28). • Challenges of implementation.

Three-dimensional model of business survey response

Sources of measurement error redefined Existing sources Identified sources Respondent →+ Business participants Interviewers →+ Survey staff Survey questionnaire (Survey instrument) → Survey instrument Mode of data collection →+ Survey characteristics Records/Information system/Organization → Organizational context INTERACTIONS!

Implications • Improved tools: – Three-dimensional model for comprehensive & systematic analysis of response process in business surveys. – Redefined sources of measurement errors for development of approaches to measurement error prevention & reduction.

Implications: Example 1 • • Self-administered business survey. No interviewer. But really no survey staff involved? One-time & rare contacts make huge impact. • Training of (all) survey staff.

Implications: Example 2 • • • Recurring business survey. No source of measurement error. But really no impact of recurrence? Learning curve effect & routine. Support & verification of first-time questionnaire completion (at business & respondent level).

Future research • More research on specific aspects: – – – comprehension of economic concepts; role of business participants & survey staff; circulation of survey instrument; impact of ‘soft’ issues; . . . • Quantification through experiments. • Generalization to non-business organizational surveys.

Sources of Measurement Errors in Business Surveys Mojca Bavdaz Mojca. Bavdaz@ef. uni-lj. si