Community Life Survey Moving to address based online

Community Life Survey: Moving to address based online surveying Rosanna White- DCMS Joel Williams- Kantar Public

Background • The Community Life Survey (CLS) is a self completion survey of adults (16+) in England. • The Survey is a key evidence source for government, covering community cohesion and engagement in England. • Fieldwork conducted by Kantar Public.

Time line Citizenship Survey Commissioned by Home Office 2001 Online/postal testing begins Citizenship Survey decommissioned 2006 Citizenship Survey Moved to MHCLG 2010 -11 2012 -13 Community Life Survey Commissioned by Cabinet Office 2013 -14 2016 -17 Move to DCMS, Survey moved fully to self-completion 2017 -18 2018 -19

What does the survey collect? • Identity and social networks • Your Community • Civil Engagement • Volunteering and charitable giving • Social Action • Loneliness and subjective wellbeing

How did the survey run in the past? • Face to face household interview. • One adult per household (‘random adults’ approach) • Conditional £ 5 incentive • c. 60% response rate and c. 2, 000 -5, 000 sample size

Why Change? • Cost saving • Increased sample size • Increased survey capability • Reduced risk of social desirability bias • Removes need to cluster sample geographically

What do we do now? • Around 30, 000 addresses invited to take part online • 4 adults (16+) can take part per household • Sub sample of non-responding households given two paper questionnaires in second reminder letter • Aim to achieve final sample size of c. 10, 000, including 2, 000 respondents from ethnic minority groups

Login codes Contact details

Enter login codes

Initial Testing • Initial testing conducted between 2013 -2016 by Kantar Public • Full details of testing published on gov. uk https: //www. gov. uk/government/publications/community-life-surveyexperimental-online-survey-findings

Issues considered during testing • Within household sampling • Response rates and sample size • Incentives • Profile of web sample compared with face to face • Change in time series/ responses ?

Key stages of testing Stage 1: September-October 2012 • • • ‘last birthday’ sample 4 incentive packages tested Small scale sample Stage 2: April 2013 -March 2014 • • • Full year and full scale test. £ 10 conditional incentive Same sampling method as stage 1 Stage 3: Stage 4: September 2013 July-September 2014 • • • All adult within household sampling experiment Small scale sample Sample vs mode experiments

Within household sampling Random adults approach All adults approach ! fraud risk! For more info: https: //www. gov. uk/government/uploads/system/uploads/attachment_data/file/466925/The_Community_Life_Survey_Investigating_t he_Feasibility_of_Sampling_All_Adults_in_the_Household_FINAL. pdf

Response rates Face to face Self-completion Online Additional due to paper 2013 -14 61% 27 -28% 25 -26% 1%-3% 2014 -15 59% 18% 1% 2015 -16 61% 25% 17% 8% 2016 -17 - 21% 15% 6% Source: https: //www. gov. uk/government/uploads/system/uploads/attachment_data/file/631932/Community_Life_Survey__Online_Paper_Estimates_Technical_Note_2016 -17_FINAL. pdf

Sample size Face to Face Self-completion Online Paper 2013 -14 5, 105 10, 215* 9, 387 828 2014 -15 2, 022 2, 323* 2, 226 97 2015 -16 3, 027 3, 256* 2, 233 1, 023 2016 -17 - 10, 256 7, 365 2, 891 *collected for methodological purposes Source: https: //www. gov. uk/government/uploads/system/uploads/attachment_data/file/631932/Community_Life_Survey__Online_Paper_Estimates_Technical_Note_2016 -17_FINAL. pdf

Testing Incentives Response rate (%) by incentive 70 60 60 50 39 40 35 31 30 27 20 10 0 No incentive Conditional £ 5 incentive Conditional £ 10 incetive Web/ postal Unconditional £ 5 incentive Conditional £ 5 incentive Face to Face Source: Kantar Public: Investigating the viability of moving from a face to an online postal model https: //www. gov. uk/government/uploads/system/uploads/attachment_data/file/466921/Investigating_the_viability_of_moving_from_a_f ace-to-face_to_an_online_postal_mode_FINAL. pdf

Profile of self completion sample compared with face to face Higher concentration of: • Those who use the internet everyday • Home owners • 45 -75 year-olds • Higher earners • Degree educated Lower concentration of: • Younger people • Social renters • Lived at address for less than a year • First language not English Source: Kantar Public: Investigating the viability of moving from a face to an online postal model https: //www. gov. uk/government/uploads/system/uploads/attachment_data/file/466921/Investigating_the_viability_of_moving_from_a_f ace-to-face_to_an_online_postal_mode_FINAL. pdf

Sample breakdown by age group 0 10 Population 15 Face to face 7 Web and Postal (random adults) 6 30 50 60 22 26 13 25 28 16 -24 30 25 25 -34 80 90 11 14 31 26 15 70 26 14 12 9 40 17 11 Online only (random adults) Web and Postal (All adults) 20 35 -49 26 50 -64 65 -74 75+ 100 10 10 15 6 17 8 16 9

What have we done to address imbalance? t • All adult approach reduced under-representation of younger people • Target postal paper questionnaires • Ensure paper versions available to all on request • Boost sample in areas with higher BAME populations

What is the impact on the time series? • Some difference in responses between face to face and self-completion • Collecting test data alongside the face to face allows for longer time series • Advise users against directly comparing selfcompletion and face to face data

Feeling of belonging to Neighbourhood & Britain Adults feeling they belong to Britain and their immediate neighbourhood; face-to-face and online/paper, 2005 to 2016 -17 100 80 60 40 2012 -13 2013 -14 2014 -15 2015 -16 2016 -17 Belonging to neighbourhood Face to face Belonging to neighbourhood online/paper Belonging to Britain face to face Belonging to Briatin Online/paper

Well-being Average rating of well-being; face-to-face and online/paper, 2012 -13 to 2016 -17 10 10 8 8 6 6 4 4 2 2 0 2012 -13 2013 -14 2014 -15 2015 -16 2016 -17 0 Satisfaction Worthwhile 10 10 8 8 6 6 4 4 2 2 0 2012 -13 2013 -14 2014 -15 2015 -16 2016 -17 Happiness 2012 -13 2013 -14 2014 -15 2015 -16 2016 -17 0 2012 -13 2013 -14 2014 -15 2015 -16 2016 -17 Online/paper Anxiety Face to face

Charitable Giving Proportion who gave to charity 4 -weeks prior to their survey; face-to-face and online/paper, 2005 to 2016 -17 100 80 60 40 2012 -13 2013 -14 2014 -15 Face to face 2015 -16 Online/ Paper 2016 -17

Looking forward… • Continue to build on knowledge of mode and sample effects • Maintain/increase response rates • Keeping up to date with best practice • Keeping up to date with technology

Any Questions? Useful links: Community life 2016/17 publication documents: https: //www. gov. uk/government/statistics/community-life-survey-201617 Previous publications and testing documentation: https: //www. gov. uk/government/collections/community-life-survey Department website: https: //www. gov. uk/government/organisations/department-for-digitalculture-media-sport Contact details: Rosanna. white@culture. gov. uk