Measuring attitudes and behaviors toward the 2020 Census

Measuring attitudes and behaviors toward the 2020 Census across time Yazmín A. García Trejo and Jennifer Hunter Childs November 20, 2020 Contact information: Yazmin. argen. garcia. Trejo@census. gov Jennifer. Hunter. Childs@census. gov Disclaimer: Any views expressed are those of the authors and not those of the U. S. Census Bureau. Acknowledgements: We would like to thank the contractor TYR and PSBInsights for the data collection work. We also would like to thank our colleagues Nancy Bates, Monica Vines, Gina Walejko and Kyley Mc. Geeney who worked on this survey project while working at the Census 2020 CENSUS. GOV in 2019 -2020 Census Tracking Survey Data are from the Integrated Communications Contract Order 18: Campaign Monitoring and Optimization.

Background and Methods: 2020 Census Attitudes Survey § The survey collected data on awareness, attitudes, self-reported intent to participate in the U. S. 2020 Census, perceptions about data usage concerns and selected topics on messages. Modes: Phone (Nationally representative) and web (non-probability sample) Data collection: Monthly (September to December 2019) and weekly (from January to September 2020). § § 2 Phone Web § Nationally representative probability-based sample survey using dual-frame Random Digit Dialing (RDD) telephone interviews of US adults § The telephone tracking survey collected 1, 400 completed cases per month from Sept. to Dec. 2019. Starting on Jan. 2020, the survey increased to 200 interviews completed daily for a total sample size of 1, 400 completed cases weekly through the end of Sept. 2020. § Data were weighted and estimates reported monthly in 2019 and reported daily on a seven-day rolling basis in 2020. § Response Rates were reported using AAPOR RR 3 each month from Sept. 2019 to Dec. 2019 and daily from Jan. 2020 to Sept. 2020. § Conducted in English and Spanish. § A nonprobability-based sample from two web panels § The online survey collected 2, 100 interviews per month between Sept. and Dec. 2019. From Jan. to Sept. 2020, the data collected was of 300 per day and, thus, 2100 per week. § Survey respondent statuses (e. g. , number terminated and qualified) were reported monthly from Sept. to Dec. 2019 and daily from Jan. to Sept. 2020. § Data were weighted and estimates reported monthly in 2019 and daily on a three-day rolling basis in 2020. § Because the survey was a nonprobability sample, a response rate cannot be calculated however the AAPOR Participation Rate was an alternative that can be used. § Conducted in English and Spanish. 2020 CENSUS. GOV

Example #1. Self-reported “Intends to Participate” and “Already Responded” to the 2020 Census to the form Intent only (Definite)* by Date (Phone vs. Web) Intent + answered census form (Definite + Answered)* by Date (Phone vs. Web) Weighted data used for the chart Survey questions: ASK ALL BEGINNING MARCH 12, 2020: ANSWERED. Have you or someone in your household answered the 2020 Census questions, or has your household not answered them yet? (1) Yes, have answered (2) No, have not answered BEGINNING MARCH 12, ASK ONLY OF PEOPLE WHO HAVE NOT RESPONDED: INTENT. [In 2019: In 2020; In 2020: Later this year], the Census Bureau will mail nearly every household in the U. S. a questionnaire and an invitation to participate online. How likely do you think it is that you or someone in your household will mail back the questionnaire or submit it online? (1) Definitely will, (2) Probably will, (3) Might or might not, (4) Probably will not (5) Definitely will not 3 2020 CENSUS. GOV

Example #2: Background and Methods Corrective Information Messaging to Debunk Misperceptions: A Case Study from the 2020 Census (submitted to the JEPS for consideration) Background: In December 2019, the 2020 Census Attitudes Survey reported that many respondents had the misperception that a citizenship question status was going to be in the Census form. Research question: Does exposure to a corrective message debunking the misperception of a citizenship status question in the Census form increase intention to participate in the 2020 Census? Methods: Randomized survey experiment conducted with the online survey data only. The experiment ran for four weeks between January 27 and February 23 rd. The online participation rate was 43. 5% Analyzed cases for control/treatment Daily quote Question Wording Control Treatment* 8, 424 3, 627 300 130 Respondents are not exposed to the question that debunks the misperception. The Department of Commerce requested that the 2020 Census ask which people living in each household are U. S. citizens. The U. S. Supreme Court later decided the 2020 Census could not ask that. Are you aware the 2020 Census will not ask about citizenship? *To embed the experiment within the existing daily online tracking survey, we first conducted a power analysis. One week of the existing tracking study had a quota of 2, 100 cases, or 300 cases per day. In order to maintain continuity for the tracking survey, we did not change these quotas for Group 2 (control). One in three qualified respondents were randomly assigned to Group 1 until Group 1 reached its target of 130 cases. Once the Group 1 quota was full, remaining respondents were assigned to Group 2 until Group 2 reached its target of 300 cases. 4 Source: Bates, Nancy, Yazmín A. García Trejo, Monica Vines and Laura Kail. Corrective Information Messaging to Debunk Misperceptions: A Case Study from the 2020 Census. Under review on the Journal of Experimental Political Science. 2020 CENSUS. GOV

Example #2. Corrective Information Messaging to Debunk Misperceptions: A Case Study from the 2020 Census Treatment: The Department of Commerce requested that the 2020 Census ask which people living in each household are U. S. citizens. The U. S. Supreme Court later decided the 2020 Census could not ask that. Are you aware the 2020 Census will not ask about citizenship? INTENT TO RESPOND (ALL CATEGORIES) Asian Black White Hispanic General population Definitely will Probably will not 52 Control (2) 16 25 42 Control (2) Definitely will not 23 49 Treatment (1) Might or might not 18 26 37 19 26 24 57 Control (2) 23 54 Treatment (1) Control (2) 43 Treatment (1) 43 Control (2) 42 Treatment (1) 42 26 27 28 3 5 3 8 4 9 4 14 24 23 5 4 3 15 21 4 3 8 21 23 21 6 4 6 5 8 4 2 Chi-square tests significantly different between Control and Treatment at p=. 05 level. Survey questions: NTENT. [In 2019: In 2020; In 2020: Later this year], the Census Bureau will mail nearly every household in the U. S. a questionnaire and an invitation to participate online. How likely do you think it is that you or someone in your household will mail back the questionnaire or submit it online? (1) Definitely will, (2) Probably will, (3) Might or might not, (4) Probably will not (5) Definitely will not 5 2020 CENSUS. GOV

Example #3: Background and Methods Interviewer Face Coverings and Response to Personal Visit Surveys: A Case Study of the 2020 U. S. Census Background: During the non-response follow-up phase, census enumerators go door-to-door to interview households that have not self-responded. Census management sought information on public attitudes regarding census enumerators’ use of face coverings when visiting households. Research question: Does the notion of face coverings worn by enumerators influence a household’s decision to participate in the 2020 Census nonresponse follow-up operation? Methods: A randomized questionnaire wording experiment was conducted adding two new questions about wearing face coverings to a split panel for the online survey. The data was collected from June 12 -July 18, 2020. Online participation rate was 34. 4% Version 1: Positive Version 2: Negative If the census taker came to your door and was wearing a face mask, would you be more likely to respond, less likely to respond, or would it not make a difference? If the census taker came to your door and was NOT wearing a mask, would you be more likely to respond, less likely to respond, or would it not make a difference? Source: Bates, Nancy, Laura Kail, Amanda Price. 2020. “Interviewer Face Coverings and Response to Personal Visit Surveys: A Case Study of the 2020 U. S. Census. Survey Practice 3: 1. https: //doi. org/10. 29115/SP-2020 -0011 6 2020 CENSUS. GOV

Example #3: Interviewer Face Coverings and Response to Personal Visit Surveys: A Case Study of the 2020 Census Version 1 - Positive: "If the census taker came to your door and was wearing a face mask, would you be more likely to respond, less likely to respond, or would it not make a difference? Version 2 – Negative: “If the census taker came to your door and was NOT wearing a face mask, would you be more likely to respond, less likely to respond, or would it not make a difference? ” * Source: Bates, Nancy, Laura Kail, Amanda Price. 2020. “Interviewer Face Coverings and Response to Personal Visit Surveys: A Case Study of the 2020 U. S. Census. Survey Practice 3: 1. https: //doi. org/10. 29115/SP-2020 -0011 7 2020 CENSUS. GOV *

References and research projects § Bates, Nancy, Laura Kail, Amanda Price. 2020. “Interviewer Face Coverings and Response to Personal Visit Surveys: A Case Study of the 2020 U. S. Census. Survey Practice 3: 1. https: //doi. org/10. 29115/SP-2020 -0011 § Bates, Nancy, Yazmín A. García Trejo, Monica Vines and Laura Kail. Corrective Information Messaging to Debunk Misperceptions: A Case Study from the 2020 Census. Under review on the Journal of Experimental Political Science. § Comparing probability-sampled telephone and nonprobability-sampled web survey data across time (forthcoming) 8 2020 CENSUS. GOV