AddressBased Sampling What Do We Need To Know

Address-Based Sampling: What Do We Need To Know? Michael W. Link Chief Methodologist The Nielsen Company DC-AAPOR/WSS ABS Seminar 1/10/09 Washington, DC

Well Documented Problems with RDD surveys • Rapidly declining response rates • Systematic under-coverage of cell phone only-households • Lack of sufficient participation among 18 -34 – Even with oversampling techniques, unable to reach complete targets among this group • Costs sky-rocketing • Losing confidence in data collected and rating produced DC-AAPOR/WSS ABS Seminar 11/10/09 Washington, DC 2 Confidential & Proprietary Copyright © 2008 The Nielsen Company

New sample frame option Address Based Sampling (ABS) • Concept: Sample addresses rather than sample telephone numbers – Based on an address database, not traditional counting & listing approach • Current frame source: US Postal Service Delivery Sequence File (DSF): • Switch from the landline telephone frame to ABS is designed to address the COVERAGE problem – With increased opportunity to address nonresponse problem DC-AAPOR/WSS ABS Seminar 11/10/09 Washington, DC 3 Confidential & Proprietary Copyright © 2008 The Nielsen Company

Residential Household Coverage with Landline Random Digit Dialing model Landline Telephone Frame Covered ~65% Not Covered Systematically Excluded ~35% Business / not in service / non-residential DC-AAPOR/WSS ABS Seminar 11/10/09 Washington, DC 4 Confidential & Proprietary Copyright © 2008 The Nielsen Company

Residential Household Coverage with Address Based Sample Model Address Based Frame Covered ~98% DC-AAPOR/WSS ABS Seminar 11/10/09 Washington, DC 5 Confidential & Proprietary Copyright © 2008 The Nielsen Company

Completing the transition • In November 2008, Nielsen became the first organization to completely migrate an on-going data collection effort from RDD to ABS – The new Nielsen approach is 100% ABS; no RDD components • Successfully completed four TV measurements – 190 markets per measurement – 4 weekly survey periods per market • Successfully completed Radio measurement – 51 markets per measurement – 8 weekly survey periods per market DC-AAPOR/WSS ABS Seminar 11/10/09 Washington, DC 6 Confidential & Proprietary Copyright © 2008 The Nielsen Company

RDD design (Old) Sample landline telephone numbers Match to address Yes Prerecruitment Postcard Diary placement calls Mail diaries No Diary week Reminder calls Diaries Returned By mail DC-AAPOR/WSS ABS Seminar 11/10/09 Washington, DC 7

ABS Design (New) Sample residential addresses Match to landline telephone number? Yes Diary placement calls Prerecruitment mailing Mail diaries No Reminder calls & mailings Yes Pre-recruitment Survey -Mail -Web --Call-in Return telephone numbers? No Mail diaries Diaries Returned by mail Reminder mailings DC-AAPOR/WSS ABS Seminar 11/10/09 Washington, DC 8

What are the key lesson’s learned (to date)? DC-AAPOR/WSS ABS Seminar 11/10/09 Washington, DC

Source of sample file makes a difference • Addresses purchased from a third-party vendor, not USPS – Vendors vary significantly in their ability to draw samples from the file • Raw file has basic information – enhanced file from vendors has a wealth of sample information – Base frame augmented with matches to commercial and Census data information – Unless you have very skilled staff on hand, go with a knowledgeable sample vendor for access to the file or individual samples DC-AAPOR/WSS ABS Seminar 11/10/09 Washington, DC 10 Confidential & Proprietary Copyright © 2008 The Nielsen Company

Not all addresses are equal • File contains indicator of type of address: – City style, PO Box, seasonal, vacant, etc. • Conducted analyses of returns by address types • Best to exclude from the sample: – Seasonal units – Throwback units • Best to include: City style 80. 8% PO Box 7. 3% Augmented RRs 5. 7% Vacation 4. 7% Seasonal 0. 7% Throwback unit 0. 4% Drop point 0. 3% Educational 0. 1% – Vacant units DC-AAPOR/WSS ABS Seminar 11/10/09 Washington, DC 11 Confidential & Proprietary Copyright © 2008 The Nielsen Company

ABS is a Sampling Approach, Not a Data Collection Mode • With RDD, sample frame & mode were closely tied; with ABS, greater mode potential • ABS-based study recruitment modes: – In-person (~92%), telephone (~55%), or mail (100%) • ABS-based study interview modes: – Just about any mode provided individuals have access – In-person – Telephone (outbound/inbound) – Web – Mail – Interactive Voice Response/TACASI – Etc. DC-AAPOR/WSS ABS Seminar 11/10/09 Washington, DC 12 Confidential & Proprietary Copyright © 2008 The Nielsen Company

ABS can reach cell phone-only households & help bring in younger demographics • ABS offers a means of reaching CPO households without the need to sample cell phone exchanges • If a cell phone number is provided by the respondent as their preferred (or only) contact number, can use autodialing for phone-based studies • Does require thinking through some enhanced calling protocols when a cell phone number is dialed – AAPOR Cell Phone Task Force offers guidance in this area DC-AAPOR/WSS ABS Seminar 11/10/09 Washington, DC 13 Confidential & Proprietary Copyright © 2008 The Nielsen Company

Percentage of cell phone-only households 16. 3%* 14. 3% 15. 2%* 10. 4% 0% 0% DC-AAPOR/WSS ABS Seminar *Reflects model-based estimates; CPO population parameters 14 11/10/09 Washington, DC not available at individual market level

Percentage of 18 -34 Householders (Regular sample only) RDD DC-AAPOR/WSS ABS Seminar 11/10/09 Washington, DC 15

Percentage of 18 -34 Householders (Regular sample only) ABS RDD DC-AAPOR/WSS ABS Seminar 11/10/09 Washington, DC 16

“Unmatched” cases diamonds in the rough, but a lot of rough • “Unmatched” households – those with an identifiable address, but no identifiable telephone number • Predominantly cell phone only, unlisted, & households with no telephone – Younger adults, blacks, Hispanics • In the absence of field staff, must recruit initially by mail • Obtain ~25 -30% response from this group, but need to find ways to improve cooperation DC-AAPOR/WSS ABS Seminar 11/10/09 Washington, DC 17 Confidential & Proprietary Copyright © 2008 The Nielsen Company

Response Rates in an ABS Environment • Because ABS facilitates inclusion of “hard-to-reach” demographics (those with lower than average participation rates), response rates tend to go down. • Due to near universal coverage of the ABS frame, ABS response rates provide more meaningful measures of participation – Rates are not based on a subset of more compliant homes, with harder-toreach homes excluded • Significant issue that is not much discussed within survey research: Are “response rates” meaningful if based on samples with poor or low coverage? – Statistical –vs- layman understanding of what “response rate” measures – Sample-based measure that does not account for any coverage bias DC-AAPOR/WSS ABS Seminar 11/10/09 Washington, DC 18 Confidential & Proprietary Copyright © 2008 The Nielsen Company

Participation with Landline RDD Model Landline Telephone Frame Nonrespondents Respondents 35% Systematically Excluded Respondents Nonrespondents Business / not in service / non-residential DC-AAPOR/WSS ABS Seminar 11/10/09 Washington, DC 19 Confidential & Proprietary Copyright © 2008 The Nielsen Company

Participation with Address Based Sample Model Address Based Frame Nonrespondents Respondents Respondents Nonrespondents DC-AAPOR/WSS ABS Seminar 11/10/09 Washington, DC 20 Confidential & Proprietary Copyright © 2008 The Nielsen Company

Maximizing the utility of a rich sample frame: developing & optimizing sample indicators • Having an address as the base sample unit facilitates database matching to commercial & Census-based data • Enhanced sample indicators allows greater focus: – Model-based age indicator – Black Census block penetration – Hispanic surname and Census block penetration – Other study-relevant indicators • Sample indicators can improve: – Targeted sampling/stratification – Tailors materials, processes, & incentives DC-AAPOR/WSS ABS Seminar 11/10/09 Washington, DC 21 Confidential & Proprietary Copyright © 2008 The Nielsen Company

ABS next steps … • ABS solves the coverage problems seen in RDD, but not necessarily the response issues • ABS provides a solid foundation for use of different designs, approaches, & mode combinations • ABS is still in its infancy, but the growing number of studies adopting & testing this approach will allow the industry to rapidly optimize the use of this important sample frame. DC-AAPOR/WSS ABS Seminar 11/10/09 Washington, DC 22 Confidential & Proprietary Copyright © 2008 The Nielsen Company

Thank you Michael Link Michael. Link@Nielsen. com DC-AAPOR/WSS ABS Seminar 11/10/09 Washington, DC 23 Confidential & Proprietary Copyright © 2008 The Nielsen. Company Confidential & Proprietary • Copyright © 2007 The Nielsen