Applying CAWI ready steady go Examples from Finnish

Applying CAWI – ready, steady, … go Examples from Finnish LFS mixed mode tests Pertti Taskinen

Finnish LFS pilots – web and mixed mode 1 st step • 2013: only web data was collected for 8, 000 gross size pilot • First time LFS questionnaire in web • 30 per cent responded (with extended answering period) 2 nd step (slides 3 -8) • 2016 -2017: CAWI-CATI mixed mode pilot • Aimed applying by three rounds • Rejected time – 4 days - to answer first in web • Interviews were started after web option • Pilot was interrupted due to technical problems during the 2 nd round and overall response rate was low (42%) • However, valuable information on web answering was achieved 3 th step (slides 9 -11) • 2019: new CAWI-CATI pilot 2 6 May 2019 Pertti Taskinen

Web answering: what kind of problem(s) did you encounter? Share of respondents (r=496), pilot 11/2016 em s er pr o th Q of y bi lit ita Su O at th e co nn k or et w bl re en io n ec t e th in g Lo N tru ct ns of i ng nd i Fi Pertti Taskinen Q nf. . i io n s /c q: s of g in an d st nd er U dd at in oo s ch in s tie ul ic 6 May 2019 /a g eg or ca ie s t: s m le ob pr o N iff D 3 Technique Content sc 100 90 80 70 60 50 40 30 20 10 0

”Did you encounter problems in web, what kind”, pilot 11/2016 • • No problems 70% Content related problems 17. 5% Technical problems ~10% Other problems ~7% • Note: can be more than one kind of problem (sum over 100%) 4 6 May 2019 Pertti Taskinen

Web answering: what was it like? Share of respondents (r=477), pilot 11/2016 100 90 80 70 60 50 40 30 20 10 0 Fluent 5 6 May 2019 Pertti Taskinen Some issues Troubled

6 6 May 2019 Pertti Taskinen to lo g he in Q n op tio s co de at t eb w ag e tm an n' id e em s tp ro bl bu ic no e m In principle no Id d rie It tn ot no e ly t ul ic iff rd ta pp no av Ih s D oe s de co s rie qu e e us , o st re in te o tt w an eb eb 50 45 40 35 30 25 20 15 10 5 0 Id id o N 't w rw on as re er o pu t er go t or If tim e/ ot h m co an sw er Id on 't Id on se tu on ' Id An o N Respondents in telephone: why did not you respond in web (first option), share (r=305), pilot 11/2016 Technical problems

”Why did you not respond in web (first option)”, pilot 11/2016 • ~42% no time/forget • 26 -27% in principle no • (of them, do not use computer/web ~12% … dimishing? ) • 11 -12% technical problems (dimishing? ) • 20% other reasons 7 6 May 2019 Pertti Taskinen

Open-ended answers in web 2/2017 • At the second round over 50% (r=122) answered • 30% of them were positive comments • 20% of them were negative comments • (Major of them quite neutral) • Due to problems with the content of the survey and partly negative feed back, we recommended to start with the telephone interview and during the following waves to give a web option 8 6 May 2019 Pertti Taskinen

LFS mixed mode pilot 2019 • First round was planned in 2/2019 but is was deferred to 5/2019 • Planned rounds also 8/2019 and 11/2019 • The ”whole packet” will be delivered for all (=household) in telephone first • Household (minimum set of variables) perhaps later in web • The second/third wave in pilot are web (and telephone as option for those not answering in web) • We connected mixed mode project with the IESS break in times project • Project compares IESS questionnaire results with the current LFS results (telephone only) • Correction factors for time series break 9 6 May 2019 Pertti Taskinen

2019 pilot data collection plan compared to ongoing LFS 5/2019 8/2019 Normal LFS CATI Pilot Group A -Invitation letter -CATI -Gross sample 1, 800 -IESS questionnaire -Minimum set of variables Pilot Group A -SMS/email -CAWI (or CATI) -IESS questionnaire Pilot Group B -Invitation letter -CATI -Gross sample 1, 800 -IESS questionnaire -Minimum set of variables Mode efect: vs. Nonresp. ef. : + Content ef. : vs. 10 25. 2. 2019 Anna Pärnänen vs. 11/2019 Normal LFS CATI Pilot Group A -SMS/email -CAWI (or CATI) -IESS questionnaire Pilot Group B -SMS/email -CAWI (or CATI) -IESS questionnaire Pilot Group C -Invitation letter -CATI -Gross sample 900 -IESS questionnaire -Minimum set of variables

Pros and cons for mixing IESS and mixed mode test • Data at one stroke for two projects • Learning best practices for mixed mode • One and only time series break (IESS + mixed mode) • Also weighting, proxy, and imputation rules same as in future • Same model for using register based data • Building of new system architecture (data collection system) • Vulnerability • If something goes wrong … multiple efects • Workload • Building of new system architecture (data collection system) 11 6 May 2019 Pertti Taskinen

Pertti Taskinen pertti. taskinen@stat. fi Anna Pärnänen anna. parnanen@stat. f LFS Workshop Hungary 20. 5. 2019