The Epidemiologists Toolbox Cari Olson Bonnie Kerker NYC

The Epidemiologist’s Toolbox Cari Olson Bonnie Kerker NYC Department of Health and Mental Hygiene November 29, 2012

Questioning Health Data � Is there a comparison? ◦ Are the groups really comparable? � Are the differences being reported real? � Can anything else explain this association? ◦ Are they worth reporting? ◦ How much confidence do we have in them? � What � How can (and can’t) this study tell us? should findings be accurately presented?

Comparison / Control Groups � Most data interpretation requires context – a comparison group. ◦ Same group compared over time; ◦ Different groups compared within same timeframe; ◦ Different groups compared over time. � Without a comparison, the likelihood that findings are due to factors other than the hypothesized cause cannot be assessed. �Selection of study participants; �Chance; �Other factors or trends.

Rates �A basic epidemiologic tool because they allow for appropriate comparisons. ◦ Comparing counts can be misleading. # of events in a specific time period Rate = ------------------------- x 10 n Avg. pop during that time period …per 100 (%) …per 1000 …per 100, 000

Start with a Fact…. � There were 1, 765 heart disease deaths in Flushing, Queens in 2002 and 882 in Pelham Bay, Bronx.

Where are residents at greater risk for dying from heart disease? A. Flushing, Queens B. Pelham Bay, Bronx

Where are residents at greater risk for dying from heart disease? A. Flushing, Queens 354/100, 000 pop B. Pelham Bay, Bronx 361/100, 000 pop

Why? � Because Flushing (n = 498, 318) has a larger population than Pelham Bay (n = 244, 452). � Same as saying 25 miles-per-hour is faster than 50 miles-per-day: ◦ 25 miles 1 hour 50 miles 1 day (24 hours)

Questioning Health Data � Is there a comparison? ◦ Are the groups really comparable? � Are the differences being reported real? � Can anything else explain this association? ◦ Are they worth reporting? How much confidence do we have in them? � What � How can (and can’t) this study tell us? should findings be accurately presented?

What is Statistical Testing? The process of inferring from your data whether an observed difference is likely due to chance. � Commonly, significance set at 0. 05 (5%): 95% sure that the association is not due to chance. � � sig=0. 01 (1%): 99% sure. � sig=. 10 (10%): 90% sure. � ◦ The smaller the sample, the more difficult it is to find a significant difference. In larger samples, it is often easy to find significance – but is it meaningful?

Notes on Interpretation Ø Statistical Ø Not significance ≠ importance significant ≠ no association Ø Statistical significance ≠ causation

What is a Confidence Interval (CI)? � An interval or range of values that reflects the precision of an estimate of a population parameter. � Statistically, how confident are we that the number is real? � E. g. , Smoking prevalence (2010): 14. 0% (12. 9, 15. 3) � The more confidence you want (90% vs. 95% vs. 99%), the wider the interval.

Applied Interpretation of a CI Ø What does it mean if 2 CIs overlap? – Prevalence of smoking among: • Men: 16. 1% (14. 3%-18. 1%) • Women: 12. 2% (10. 8%-13. 7%) – Prevalence of diabetes among: • Men: 9. 4% (8. 3%-10. 8%) • Women: 9. 1% (8. 2%-10. 2%) Ø What does it mean if a CI includes 0?

Questioning Health Data � Is there a comparison? ◦ Are the groups really comparable? � Are the differences being reported real? � Can anything else explain this association? ◦ Are they worth reporting? How much confidence do we have in them? � What � How can (and can’t) this study tell us? should findings be accurately presented?

Confounding � � � A third factor that influences the relationship between exposure and disease. If you are interested in actual differences in prevalence across populations, confounders are not that important. However, if you are interested in assessing risk differences, confounders can and should be controlled for in analyses. 16

Confounding � Example: When comparing cardiac disease between men and women, what other factor may confound the relationship between sex and illness? � Age! If we don’t adjust for age, and find a higher prevalence among women, it might be due to the fact that in the general population, women are (on average) older than men. � Age-adjustment is one way to limit confounding. � Ensures that any differences you see between groups are NOT due to age.

Questioning Health Data � Is there a comparison? ◦ Are the groups really comparable? � Are the differences being reported real? � Can anything else explain this association? ◦ Are they worth reporting? ◦ How much confidence do we have in them? � What � How can (and can’t) this study tell us? should findings be accurately presented?

Types of Studies � Cross-sectional ◦ Select a sample from the population and measure predictor and outcome variables at the same time. � Yields prevalence; � Cannot talk about incidence or risk of developing a disease; � Cannot establish sequence of events; � Cannot infer causation; � Can be generalizable. � Case-control ◦ Select two samples from the population - one with disease and one without, then look back and measure predictor variable. � Yields odds ratio (measure of association); � Cannot talk about incidence or risk of developing a disease; � Can be generalizable.

Types of Studies � Prospective cohort ◦ Select a sample from the population, measure predictor variable (presence or absence), then follow up and measure the outcome variable. �Yields incidence, relative risk; �Can be generalizable. � Randomized Control Trial (RCT) ◦ Randomly assign people to treatment or control (exposure), then follow up and measure outcome. �Can be generalizable; �STRONGEST STUDY DESIGN FOR CAUSATION.

Types of Studies � Ecologic Study ◦ Unit of analysis is a population, rather than an individual. For example, looking at rates of disease across countries. �Can’t infer anything about individuals; �Cannot infer causality. � Qualitative Study ◦ Aims to gather an in-depth understanding; ◦ Includes focus groups, in-depth interviews; ◦ Subjects are not systematically chosen to represent a target population. �Data cannot be generalized.

Causality � Time sequence of events � Biological plausibility � Consistency � Rule and replications out confounding

How meaningful is this study? � Size of study � New knowledge vs. replicated finding ◦ The bigger the study, the more power you have to detect findings and the more generalizable it will be. ◦ First study ever finding this result? ◦ Scientific method requires ability to replicate findings.

Questioning Health Data � Is there a comparison? ◦ Are the groups really comparable? � Are the differences being reported real? � Can anything else explain this association? ◦ Are they worth reporting? ◦ How much confidence do we have in them? � What � How can (and can’t) this study tell us? should findings be accurately presented?

Presentation of Data Findings � Provide clear context of literature base and importance of findings. ◦ How big is the population that these findings apply to and what population exactly is referenced? � Always source the data clearly, providing link to/information on original research for audience. � Question data. researchers on limitations to their ◦ Researcher “headlines” (titles/abstracts) can be misleading!

The WHY Question � Best answered by qualitative data (focus groups, interviews). � Speculation � Reporting vs. Evidence. “could be” rather than “is. ”

Illustrating Data with “Human Interest” Stories � Anecdotes can make data come alive, but… ◦ “Anecdotal evidence” is an oxymoron. � Anecdotes should not be the only “counterfactual” argument against data. ◦ “Fairness” in reporting must insist on data (with stated limitations) from both sides.

Illustrating Data with “Human Interest” Stories � Anecdotes must be presented in the context of the data. ◦ Source says “Everyone does X” vs. data showing that 35% of people do X.

Remember Health Department Data Sources for NYC � Epi. Query ◦ Web-based, interactive data tool ◦ Multiple data sources � My Community’s Health: Data and Statistics ◦ www. nyc. gov/health

THANK YOU! Contact: colson@health. nyc. gov bkerker@health. nyc. gov