DESIGN FEATURES OF NCHS SURVEYS By Iris Shimizu

DESIGN FEATURES OF NCHS SURVEYS By Iris Shimizu Mathematical Statistician Office of Research and Methodology, NCHS Disclaimer: The opinions in this presentation are those of the presenter and not necessarily those of NCHS.

OUTLINE • DESIGN FEATURES OF ESTABLISHMENT SURVEYS o SAMPLE DESIGN o DATA WEIGHTS • FEATURES COMMON TO ALL COMPLEX SAMPLE SURVEYS (both establishment and population surveys) 3

ESTABLISHMENT SURVEYS TARGETED ANALYSIS UNITS: • CLIENTS OF ESTABLISHMENT • EVENTS OCCURRING AT/WITH ESTABLISHMENT • ESTABLISHMENTS THEMSELVES 4

National Health Care Survey • • • National Hospital Discharge Survey National Survey of Ambulatory Surgery National Ambulatory Medical Care Survey National Hospital Ambulatory Medical Care Survey National Nursing Home Survey National Home and Hospice Care Survey 5

FEATURES • MULTI-STAGE SAMPLING • PRIMARY SAMPLING UNITS (PSUs) o ESTABLISHMENTS o AREAS (USED TO SAVE COSTS) 6

FEATURES (CONTINUED) • STRATIFICATION o GEOGRAPHY o PROVIDER SPECIALTY o SIZE (INPATIENT BEDS, VISIT VOLUME) o ESTABLISHMENT TYPE o OWNERSHIP TYPE • SELECTION WITH PROBABILITY PROPORTIONAL TO SIZE (PPS) 7

FEATURES (CONTINUED) • SAMPLING FREQUENCY o EVERY YEAR FOR PHYSICIANS o PERIODICALLY FOR OTHER ESTABLISHMENTS § BASIC SAMPLE –NEW DESIGN § UPDATES PERIODICALLY 8

FEATURES (CONTINUED) • WITHIN ESTABLISHMENT SAMPLING o TIME SAMPLE o VISIT SAMPLE –FROM FRAME PROVIDED BY ESTABLISHMENT o STRATIFICATION o SYSTEMATIC RANDOM SAMPLING o PPS FOR SELECTING SERVICE AREAS 9

OVERALL PROBABILITY • PRODUCT OF PROBABILITIES AT EACH SAMPLING STAGE • ACCOUNTS FOR SAMPLING DESIGN FEATURES 10

DATA WEIGHTS • INVERSE OF SELECTION PROBABILITIES • ADJUSTMENT FOR UNIT NON-RESPONSE • CALIBRATION – USES DATA FROM NONSAMPLE SOURCE FOR UNIVERSE 11

VARIANCES USING PUBLIC USE FILES • REFER TO DATA FILE DOCUMENTATION • FOR RECENT YEARS AND BARRING RISKS, NEEDED DESIGN VARIABLES ARE IN FILES • RESEARCH DATA CENTER 12

SUMMARY FOR ESTABLISHMENT SURVEY DESIGN • DESIGNS USE MULTI-STAGE STRATIFIED SAMPLES • WEIGHTS AND VARANCES REFLECT THE COMPLEX SAMPLES 13

TURNING ATTENTION TO FEATURES COMMON TO ALL SURVEYS (BOTH POPULATION & ESTABLISHMENT 14

DANGERS OF NOT USING SAMPLE WEIGHTS UNWEIGHTED ESTIMATES: • OF TOTALS WILL BE TOO SMALL • OF RATES AND OTHER RATIOS COULD BE DISTORTED. I. E. , UNWEIGHTED SAMPLE PROPORTIONS COULD DIFFER FROM THE CORRESPONDING CENSUS PROPORTIONS 15

VARIABILITY OF SURVEY ESTIMATES • ESTIMATES BASED ON SAMPLES ARE SUBJECT TO SAMPLING VARIABILITY • ESTIMATES OF SAMPLING VARIANCES MUST ACCOUNT FOR SAMPLE DESIGNS FOR VALIDITY 16

COMPLEX SURVEY FEATURES AFFECTING VARIANCE ESTIMATION • CLUSTERING ANALYTIC UNITS WITHIN PRIMARY SAMPLING UNITS (PSUs) • STRATIFICATION OF PSUs 17

DANGER OF USING SAMPLE SUBSETS TO ESTIMATE VARIANCES • VARIANCE ESTIMATES BASED ONLY ON SUBSETS OF SAMPLE MAY NOT CORRECTLY REFLECT SAMPLE DESIGN • COULD UNDERSTATE SAMPLING VARIANCE 18

DANGERS OF IGNORING SAMPLE DESIGN IN VARIANCE ESTIMATION • VARIANCE ESTIMATES PROBABLY TOO SMALL • “DEGREES OF FREEDOM” WOULD BE TOO LARGE 19

GENERAL REFERENCE FOR SURVEY ANALYSIS SOFTWARE http: //www. hcp. med. harvard. edu/statistics/ survey-soft/ Provides descriptions and links to software packages that do variance estimation with complex sample data. 20

SUMMARY FOR ALL COMPLEX SAMPLE SURVEYS • SAMPLING WEIGHTS SHOULD BE USED • SHOULD USE COMPLEX SAMPLE ANALYSIS SOFTWARE TO ESTIMATE VARIANCES • SHOULD USE WHOLE SAMPLE TO ESTIMATE VARIANCES 21