Minimizing Nonresponse in a Survey of New Businesses
Minimizing Non-response in a Survey of New Businesses Tom Barton David Des. Roches Third International Conference on Establishment Surveys June 19, 2007 Montréal Québec, Canada
Kauffman Firm Survey (KFS) Background l Sponsored by the Ewing Marion Kauffman Foundation- www. kauffman. org/ l Longitudinal survey of businesses that began operations in 2004 l Baseline Survey completed in 2006, annual follow-up surveys planned for three years
Survey Design l Web and CATI data collection l Advance letter and a reminder postcard l “Web-First” approach – No CATI follow -up mentioned in advance materials l Web non-responders were called
Barriers to Survey Completion l “Surveys” l Lack used as marketing tools of time for business owners l Gatekeepers l No contacts
Overcoming Barriers to Completion l Use a multi-mode approach, emphasizing Web option l Streamline l $50 survey instrument post-pay incentive
How Interviewers Address Barriers l Stress research l Not a sales pitch l Not soliciting money – “Kauffman Foundation” l Acknowledge how busy they are
Telephone Interviewer Training l Be honest about the difficulty of the study l Use what was learned from Pilot Study l Practice a smooth and short introduction l Practice quick response to common objections, such as lack of time and “I’m not interested. ” l Role plays with scenarios, including those with gatekeepers.
Managing Gatekeepers l Ask for the business owner first l Stress 5 minutes of the owner’s time l Push the incentive l Don’t just leave the toll-free number l If you talked to the owner on a previous call, tell the gatekeeper.
Data Collection Results l 12 -month data collection period l Total sample size = 32, 469 l 4, 930 interviews completed - CATI (77%), Web (23%) l A low eligibility rate (35%) l 49% weighted response rate
Data Collection Results- Continued l 375, 000 calls made to complete 3, 783 CATI interviews l Converted 538 refusals, or 11% of the total completes and 8% of total refusals - screened out another 1, 062 l Of all refusal conversions, 478 (89%) completed by phone, and 60 (11%) by Web
Research Question l When we compare business owners who initially refused and then completed with business owners who completed without an initial refusal, do they differ on: Ø Measures of size Ø Type of industry Ø Form and location of business Ø Demographics
Size Comparison Total Assets No initial refusal (N= 4, 392) Initial refusal(N = 538) Median = $19, 500 Median = $18, 000 Total Revenue Median = $7, 000 Median = $3, 500 Number of employees Mean = 1. 87 Mean 1. 90
Technology Status No initial refusal (N= 4, 392) 2% Initial refusal(N = 538) 2% Medium Tech Low Tech 13% 85% Intellectual Property 1. 30 1. 35 High tech
Legal Status Comparison No initial refusal (N= 4, 392) 36% Initial refusal(N = 538) 35% 5% 6% LLC 31% 29% Sub-S Corp 20% C-Corp 8% 9% Sole Proprietorship Partnership
Business Location Comparison Home No initial refusal (N= 4, 392) 49% Initial refusal(N = 538) 48% Rented Space 41% 42% Purchased Space 5% 6% Client Location or other 5% 4%
Demographics of Business Principal Age Gender Female Male Education Associate’s or lower Bachelor’s or higher No initial refusal (N= 4, 392) 44 Initial refusal 32% 68% 29% 71% 54% 46% 52% 48% (N = 538) 46
Summary l No major differences between refusals and non-refusals l Surveys of new businesses offer significant challenges l Prepare interviewers for the challenges l Ask only what you need l Consider multiple modes and incentives
Questions, Comments and Paper Requests l Please contact Tom Barton at: Tbarton@mathematica-mpr. com l Mathematica’s Web site is: www. mathematica-mpr. com/
- Slides: 18