The Annual Dutch Business Inquiry Developing and Testing

The Annual Dutch Business Inquiry Developing and Testing the electronic form Ger Snijkers Evrim Onat Jo Tonglet Statistics Netherlands Division of Business Statistics 13 June 2005 CES, 13 -15 June 2005, Geneva 1

Developing the electronic form In three steps: 1. Small scale usability test (31 -1 -’ 05) - Small development team - Prototype of the e-form 2. Improving the e-form (1 -10 -’ 05) - additional usability tests 3. Implementation (15 -2 -’ 06) 13 June 2005 CES, 13 -15 June 2005, Geneva 2

1 Tested form 2 Test waves 3 Research issues 4 Conclusions 1. Tested e-form 13 June 2005 CES, 13 -15 June 2005, Geneva 3

1 Tested form 2 Test waves 3 Research issues 4 Conclusions 2. Test waves Test 1 Test 2 Test 3 Period Aug. 2004 Oct. 2004 Nov. /Dec. 2004 Form Blaise IS Blaise EDR Blaise CBSquest On/off-line On-line Off-line Mode Browser CD-rom Downloadable via internet Number of interviews 15 CBS employees 37 businesses 6 businesses Tested by CBS employees, external designer 6 business interviewers 2 cogn. lab interviewer + 2 business interviewer Results Most bugs identified Experiences from interviewers Most results; Inexperienced respondents 13 June 2005 CES, 13 -15 June 2005, Geneva 4

1 Tested form 2 Test waves 3 Research issues 4 Conclusions 3. Research issues 1. How does the e-form work in practice? - Completing the questionnaire - Question-and-answer process 2. What features should be included to make it easy to use? 1. 2. make it respondent friendly - ‘Computer-assisted’ tools to User wishes 3. 3. How should the e-form be designed in relation to the paper form? 4. 5. 13 June 2005 The same or a different design ‘look-and-feel’ of paper and e-5 CES, 13 -15 June 2005, Geneva

1 Tested form 2 Test waves 3 Research issues 4 Conclusions 3. Research issues 1. How does the questionnaire work? • Laborious and complex process - Long, complex questionnaire (≥ 25 items) Complex completion process: several sessions, several informants kick-and-rush behaviour Imagine. . . a respondent sitting behind his/her computer. . . Respondent got lost in the questionnaire 13 June 2005 CES, 13 -15 June 2005, Geneva 6

1 Tested form 2 Test waves 3 Research issues 4 Conclusions 3. Research issues 2. Features to make it easy to use? 1. What am I supposed to do (next)? 2. 3. 4. - Easy to download and install Clear instructions and explanations Easy to send data back 5. How is the questionnaire built up? 6. 7. 8. - Show the questionnaire is structured Help to find the way in the questionnaire No hidden rules, no unexpected functionalities 9. Where am I? What did I do so far? 10. 11. 12. - Provide overview of the completion process Clear navigation, no scrolling Printing function 13. Where can I put these data? 14. 13 June 2005 - Entry-search (google) CES, 13 -15 June 2005, Geneva 7

1 Tested form 2 Test waves 3 Research issues 4 Conclusions 3. Research issues 3. Design of paper and e-form? • The computer is different than paper - The e-form reacts to the respondent Reading from the screen is different Navigating and getting an overview works differently Kick-and-rush behaviour, even stronger than on paper In order to get good data in time … 13 June 2005 CES, 13 -15 June 2005, Geneva 8

1 Tested form 2 Test waves 3 Research issues 4 Conclusions 4. Conclusions • Visual design Clear and logical Simple, transparent, consistent No hidden and unexpected functionalities • Support the completion process other mode, other features, other visual design Different than paper form, same ‘look-and-feel’ • Tailor to kick-and-rush behaviour Small sections, small tasks Short and clear explanations 13 June 2005 CES, 13 -15 June 2005, Geneva 9