RESPONSE Eurostat Presented by Andrzej Modak Central Statistical

RESPONSE Eurostat

Presented by • Andrzej Młodak • Central Statistical Office, Statistical Office in Poznań, Poland.

Modules • Response process (author: Paweł Lańduch, Central Statistical Office, Statistical Office in Poznań, Poland) • Response burden (authors: Monika Natkowska and Andrzej Młodak, Central Statistical Office, Statistical Office in Poznań, Poland)

Response process • Background – response process is a step of the statistical production procedure which results from the interaction between a respondent and a questionnaire – the key role of the relationship between model steps and observations of real respondent behaviour in relation to survey requests – it enables us to identify places and time points where the burden is especially heavy.

Response process • Response models for business surveys – Tourangeau’s Model (1984) and its adjustment to business surveys (Edwards and Cantor (1991)) Tourangeau’s Model Comprehension Retrieval Judgment Communication Edwards’ and Cantor’s Model Encoding /Record formation Comprehension Source decision Retrieval /Record look-up Judgment Communication

Response process • Hybrid Response Process Model for Establishment Surveys (Sudman et al. (2000)) Encoding in memory/record formation Selection and identification of respondent(-s) Assessment of priorities Comprehension of the data request Retrieval of information (memory/record) Judgment of the adequacy of the response Communication of the response Release of the data

Response process • MISBR – Multidimensional Integral Business Survey Response Process Model (Bavdaž (2010))

Response process • Three layers of the MISBR model – Organizational layer § Includes a complex list of factors which influence the consecutive steps of the response process § Respondent selection and assessment of priorities usually concern both individual and organizational circumstances – Typical organizational factors: tradition, customary practices, procedures, location of information – Both organizational and individual factors: competing tasks and delivery of requested data § Business information system – Retrieval from business records and authorization of the business response – External factors: legal obligations, standards, benchmarking practices – Internal factors: various aspects of management needs § Record formation problem is conditioned by the kind of business activity, its environment and data availability.

Response process • Three layers of the MISBR Model (cont. ) – Individual layer § Moves the response process from the organizational level to the individual level § The multidimensional integral business response process distingishes three types of necessary knowledge: – Knowledge of business reality – Knowledge of record formation – Knowledge of business records § Stages: – Comprehension: matching survey variables with business activity and determining their relevance for business statistics – Retrieval: knowledge and recognitions of business records – Judgment: compilation of processed information and the record formation process – Communication: required edition and categorization of business knowledge from records.

Response process • Three layers of the MISBR Model (cont. ) – Survey layer § Refers to the general implementation of the survey response as well as repeated response to the same surveys. § It is used to conceptualize the influence of various elements of a survey on the response process (e. g. respondents’ tendency to routine performance of tasks and possible participation of persons outside of an organization) § Different roles to people taking part in the process resulting from their internal cognitive importance are assigned: – Gate-keeper: bringing information into an organization or sends information from an organization to its surrounding environment – Supervisor: person/unit with some authority concerning data collection – Coordinator: person/unit coordinating the response preparation – Data provider: person/unit which provides the required data for a given field/variable – Respondent: person/unit which transmits responses to the receiver (e. g. NSI).

Response process • Application of the response process model – Results of the study of the response process model § A framework for evaluating questionnaires and detecting and understanding problems (burden, errors, etc. ) § For data users: awareness of possible cases on non-availability of the required data § For data collectors: improve data collection instruments and facilitate the process – Encoding in memory/record formation step § Categorical data are retrieved from memory and figures – from transactional systems; their complexity depends on the size of a company. – Selection and identification of respondent(-s) § It can reduce the measurement error; respondent can be also a coordinator.

Response process • Application of the response process model (cont. ) – Assessment of priorities § Important factors for respondent: mandatory status of the survey, clear due date, advance notice of new surveys – Comprehension of data request § Cognitive step based on the knowledge of business reality and standards – Retrieval of relevant information § Recognise to what extent survey concepts match business practice and who has the ability to access company data; compile pieces of data – Judgment of the adequacy of the response § Assess whether the collected data meet the requirements; § Questionnaire instructions are tools to judge the correctness of data.

Response process • Application of the response process model (cont. ) – Communication of the response § Matching prepared data to fit the options of the measuring instrument § Correctional operation to ensure data consistency § Importance of electronic reporting; burden of communicating are perceived by respondents as unwarranted – Release of the data § This step may require authority § Complying with external obligations and the concern to project a good publish image may entail an internal policy, whereby any information leaving the company must first be approved by its management § Trust towards a statistical agency (confidentiality of business activity).

Response burden • The concept and awareness of response burden – What is response burden? § Generally: All circumstances and factors negatively affecting the quality and cost of collecting statistical data directly from respondents or other external sources are regarded as response burden. § The response burden is a negative effect of growing demand for economic data and, in consequence, a wide scope of detailed statistical surveys § Such an effort (additional task in relation to the main activity of an economic entity) to satisfy a number of needs in a relatively short time can lead to gaps and errors in provided data § NSI should be concerned about response burden also in their own self interest since excessive burden cause problems with data quality

Response burden • The concept and awareness of response burden – An example (EU Project on Baseline Measurement and Reduction of Administrative Costs) § Total administrative burden in 13 priority areas = ca € 102 billion. Burden caused by statistics is estimated at € 552 million (0. 5% of the total burden). § Statistics is one of the three priority areas that cause the highest irritation. – Excessive response burden results from § refusal to participate in the survey – no data from a unit is available § refusal to provide some data (item non–response problem) § provision of data of too low quality, e. g. presenting rough figures, errors in estimation or computation, etc. § in the case of similar surveys some data can be mechanically copied from one questionnaire to another without any concern for accuracy § deliberate provision of false data (an extreme situation).

Response burden • Classification of response burden by type – Dichotomies and preferences by Willeboordse et al. (1997) 1 2 3 4 objective gross imposed maximalistic subjective net accepted minimalistic

Response burden • Classification of response burden by type – Other classifications § Hedlin et al. (2005): – actual burden – perceived burden; § Fisher and Kydoniefs (2001): – respondent burden – design burden – interaction burden. – Interactions of types § Perceived burden is influenced both by respondents’ ability to answer, by the design of the survey and by the combination of these elements § Standard Cost Model ignores perceived (subjective) burden § Response burden has two dimensions: qualitative and quantitative.

Response burden • Classification of response burden by type – Suggested universal classification gross respondent accepted maximal gross actual design accepted maximal gross interaction Response burden accepted maximal observed respondent accepted maximal observed perceived design accepted maximal observed interaction accepted maximal

Response burden • Factors affecting response burden – Factors affecting perceived burden (Haraldsen (2004) Survey properties • Instrument features (number and flow of questions, questionnaire content, etc. ) • Data collection (contact mode, recruiting strategy, security measures, etc. ) Respondent’s characteristics • Interest of in the topic of survey • Competence of the respondent • Availability (amount of required time and effort)

Response burden • Factors affecting response burden – Six main factors by Hedlin et al. (2005) Survey organization/sponsor Publicity Implementation strategy Questionnaire length Question comprehension Mode of data collection

Response burden • Measurement of response burden – Indicators of response burden § Scheme of Perceived Response Burden Study (PRB) Core Question Set Perceived burden Actual burden Preceived causes of burden Motivation Perception of time Time to collect information Reason for time consuming Usefulness for own business Perception of burden Time to complete questionnaire Conditions for burden Usefulness for society

Response burden • Measurement of response burden – General Attitude Towards Survey Scale (Loosveldt and Storms (2004)) [1 – completely agree, 2 – agree, 3 – neither agree nor disagree, 4 – disagree and 5 – completely disagree] § § § § Surveys like this are a waste of time for people participating in it. By means of surveys like this one can express their opinion. Results of surveys like this are useful to make policy decisions. Surveys like this are an invasion of people’s privacy. Everyone is obliged to cooperate with surveys like this. Results of surveys like this are mostly correct. With surveys like this the government gets a good picture of what’s going on in the population.

Response burden • Measurement of response burden – Administrative burden for businesses caused by statistical obligations can be measured by a special barometer of burdens (e. g. in Germany) – Indirect measures – indicators of response § unit response rate § item response rate – Difficulties affecting response burden § discrepancies between the time when survey data are transferred to NSIs and rules applied in accounting systems and the timetable of wage and salary payments in different economic entities § obligation in the next round depends on the current number of employees.

Response burden • Measurement of response burden – Standard Cost Model Regulation Information obligation 1 Data requirement 2 Activity 2 Data requirement n Activity n Information obligation 2 Information obligation n Activity 1 Internal costs - Hourly rate - Time - Overheads External costs - Hourly rate - Time Acquisitions (monetary value)

Response burden • Measurement of response burden – Standard Cost Model § Computational formula – – Price=Time × Tariff Quantity = Population × Frequency Combining these elements give the basic SCM formula: Activity Cost = Price × Quantity = (tariff × time) × (population × frequency) § SCM example – an administrative activity takes 3 hours to complete (time) and the hourly cost of one member of staff in the business completing it is £ 10 (tariff). The price = 3 × £ 10 = £ 30. For 100, 000 businesses (population), each of which had to comply twice a year (frequency), the quantity would be 200, 000. Hence the total cost of the activity amounts to 200, 000 × £ 30 = £ 6, 000 § Respondent burden cost – Respondent Burden Cost=(weighted mean cost per questionnaire + uplift for recontacting business) × number of questionnaires in survey sample × survey frequency where cost per questionnaire=[(internal cost + overhead - adjustment for business-as-usual + external cost]

Response burden • Measurement of response burden – Recommendations § Cost Benefit Analysis (Prest and Turvey (1965). Haraldsen et al. (2013)) – the respondent burden as an effect of participation in a survey which can generate both costs or benefits for users and institution conducting the survey – cost of a survey is divided into respondent burden costs and survey organization costs § Measurement should include obligatory as well as voluntary data collection from businesses (Rainer (2008)) § Each NSI should have a central unit coordinating the problem of measuring response burden § Complex measure of response burden, combining actual burden and perceived burden (θ is the cost of conduction of the survey for businesses obtained using the SCM model, Θ denotes the total cost of conduction of this survey, is the value of j–th category in the i-th question concerning perceived burden and is the percentage of a given answer in surveyed businesses )

Response burden • Reducing response burden – Basic instruments and factors affecting the reduction of response burden § Coordination, concentration or integration of data collection § Rationalization of the number of questionnaires and indicators where they should be reported § Coordinated delimitation of sampling frames § Coordinated sampling § Electronic Data Interchange – EDI § Information on response burden § Policies applying at the level of individual surveys § Cut down on the information output and the need for a data input from respondent (burden reduction actions may also reduce the quality!)

Response burden • Reducing response burden – Main factors affecting the total reduction of response burden (Hedlin (2011)) Time per question Number of ques-tions per questionnaire Use of registers General survey tasks Range of questions in different survey rounds Frequency of re–contacs for the same questionnaire Number of respondents Spreading actual response burden out Frequency of the questionnaire

Response burden • Reducing response burden – Some practical solutions § Survey Help Desk in Statistics Sweden § Use of the XBRL (e. Xtensible Business Reporting Language) technology § Bulgarian project aimed at reduction of administrative burden (based on the Information System of Business Surveys – ISBS) § Experiences of United States of America: reducing instruction/explanatory materials and item redundancy distributing subsets of items strategically across units using available data or imputation to complete analyses and automating field completion by means of NLP (Non-Linear Programming) and scrapers § Poland – central database indicating for each economic entity which surveys it is actually involved in.

Response burden • Reducing response burden – International recommendations § Eurostat (2011) – “the reporting burden should be proportionate to the needs of the users and should not be excessive for respondents. The statistical authority monitors the response burden and sets targets for its reduction over time”. – Principle 9: “the range and detail of European statistics demands is limited to what is absolutely necessary”. § Eurostat (2009) – treating respondent burden should include, e. g. : – – – assessment of annual respondent burden in financial terms and/or hours, the definition of respondent burden reduction targets, recent efforts made to reduce respondent burden, assessment of absolute necessity of range, detail and scope of data collected by surveys, etc. the scope of data collected from businesses should be verified to obtain an answer whether such data are readily available from their accounts.

Response burden • Reducing response burden – International recommendations (cont. ) § Conclusions (Hedlin (2011)) – Eurostat should initiate the development and implementation of a standardised methodology for the measurement of response burden caused by official business surveys. The standardised methodology may include multiple indicators and a minimum version of the measurement, to accommodate NSIs differences regarding the purposes of and resources for response burden measurement. – 2 Research concerning business data collection methodology must move on from qualitative, explorative research to quantitative and preferably experimental research designs. – Effects of actions intended to reduce response burden should be monitored, reviewed, documented and published. – Burden reduction measurement and burden reduction actions should be coordinated within NSIs. Within NSIs the knowledge on response burden measurement and response burden reduction actions seems to be rather fragmented and scattered.

Thank you very much for your attention