EvidenceBased Public Health Nutrition Eric Poortvliet Unit for

Evidence-Based Public Health Nutrition Eric Poortvliet Unit for Public Health Nutrition, KI

There they are again. . The five most terrifying words in TV news ! ”According to a new study. . ”

Why is evidence-based public health important ? l During the past century, average life expectancy increased by approximately 30 years in industrialized countries l Only about 5 years of that improvement is attributable to preventive services and medical care Bunker et al, 1994

Why do we need evidence-based PHN (1) l Millions of lost working days per year l Link between nutrition exposure (saturated fat, fruit and vegetable, starchy food) and coronary heart disease l Any public health strategy to reduce ill-health needs to address the question of cost-effectiveness l Need to justify actions and demonstrate benefits of public health interventions

Why do we need evidence-based PHN (2) l Decisions often made by politicians with limited health backgrounds l Plethora of information of varying quality l Need high quality, filtered information to make informed decisions l Value of scientific knowledge for decision making l Decisions should not be based only on intuition, opinion or anecdotal information

Evidence for what ? ? • Evidence about the scientific basis of recommendations (e. g. folate. . ) • Evidence about strategies that work (e. g. childhood obesity) • Evidence about policy impact (breastfeeding)

Quote from Public Health Professionals (1): “Some things have simply always been done a certain way – are common practice, but there is really no research to back it up. ”

Quote from Public Health Professionals (2): “I make a lot of decisions about how money is going to be spent, and I would like to always be able to back it up and say that this is proven, or evidencebased. ”

Define ‘evidence-based’ ”Evidence-based healthcare is the conscientious use of current best evidence in making decisions about the care of individual patients or the delivery of health services” (Nordic Workshop on how to critically appraise and use evidence in decisions about healthcare, National Institute of Public Health, Oslo, Norway, 1996)

Five Steps of Evidence-Based Medicine 1. Convert information needs into answerable questions 2. Track down with maximum efficiency the best evidence with which to answer them 3. Critically appraise that evidence for its validity and usefulness 4. Apply the results of this appraisal in your practice 5. Evaluate your performance

Public Health Medicine l Focus on populations l Focus on individuals l Prevention & health promotion l Diagnosis & treatment l Environment & human behavior interventions l Clinical interventions l Well-established profession, standardized education & certification l Diverse workforce, variable education & certifications l Social sciences integral; clinical sciences peripheral to education l Clinical sciences integral; social sciences less emphasized l Observational studies: casecontrol & cohort studies l Experimental studies with control groups: RCTs.

Medline search: nutrition OR diet OR food AND (PY = year) Brunner E, et. al. Public Health Nutrition 2001 -

Evidence-based nutrition ? l Cochrane libary contains dozens of complete reviews on nutrition interventions l The Cochrane model is heavily based on randomized clinical trials (RCT) l RCTs do not translate comfortably into the field of nutrition

www. cochrane. org

Cochrane reviews Initiated in 1993 by Archie Cochrane Voluntary organisation of researcher and practitioners Typically, a meta-analysis of randomized controlled trials Does not translate comfortably into the field of nutrition: l l l Longer time periods needed to study diet Larger populations than feasible Participants agree to ‘engineering of diet ?

SBU: Statens beredning för Medicinsk Utvärdering

Focus on randomized trials ? l best available tool for reducing measured and unmeasured confounding l unique strenghts in supporting causal inference l might increase efficiency of searching l understandable l maybe easier to synthesize than diverse designs

The effectiveness of parachutes has not been subjected to rigorous evaluation by using randomized controlled trials Smith and Pell, BMJ, 2004

Reasons for alternatives to the RCT l Randomisation often not appropriate or feasible l Randomized studies are not free of threats to internal and especially external validity l Several recent systematic comparisons show close correlation between randomized and ’observational’ studies

Sometimes, RCTs are more reliable than ’obvervational’ data l Trials on estrogen replacement and cardiovascular disease have raised important questions about observational studies

Sometimes, ’obvervational’ data are more reliable than RCTs l RCTs often underestimate drug-related harms and observational data (e. g. from well-conduced cohort studies) are preferable to RCTs (underpowered or too short term)

Evidence about the scientific basis of recommendations • RCTs are good for single nutrients, but is is difficult to control all aspects of dietary behaviour/change • Complexity of the diet as exposure needs to be taken into account (e. g. people who eat more fruit tend also to consume less meat)

Guidelines for gathering and reviewing evidence • Judgement based on systematic review of all available evidence • Explicit search strategy and inclusion criteria

Review guided by: l Type of epidemiologic study (RCT, cohort, etc. . ) l Consistency of results between studies l Quality of studies reviewed l Validity of measure of exposure l Consideration of effects of sample size, power, bias and confouding l Design issues (loss to follow-up etc l Size of risk estimate (relative risk, odds ratio) l Evidence that exposure preceeds outcome l Plausible mechanisms

Where to set the evidence bar for effectiveness ? How much evidence is enough ? l How to balance type 1 vs. Type 2 error ? l ”The scientific purist, who will wait for medical statistics until they are. . . exact, is no wiser than Horace’s rustic waiting for the river to flow away”

The Evidence Pyramid

Sources of Evidence-Based Knowledge 1. Reports of Original Research 2. Summaries, Critiques and Commentaries 3. Systematic Reviews, Meta-Analyses, and Evidence. Based Guidelines 4. Comprehensive Knowledge Bases

1. Reports of Original Research • Journal articles • Book chapters • Reports (government & other sources) • Newsletter articles • Conference proceedings and abstracts • Filtered searches of bibliographic databases

2. Summaries, Critiques and Commentaries l Narrative reviews and summaries of original studies l Critiques of original studies l Expert commentary based on original studies l Structured abstracts of individual research articles

3. Systematic Reviews, Meta-Analyses, and Evidence-Based Guidelines l Systematic reviews with explicit criteria l Meta-analyses of data from original studies l Evidence-based guidelines based on original studies, systematic reviews, and meta-analyses

4. Comprehensive Knowledge Bases Searchable online textbooks with up-to-date information l Collections of multiple online resources l Journal articles l E-textbook chapters l Recommendations l Multiple databases with integrated searching l Internet links to related sources l

Common vs. evidence-based reviews 1. Common literature reviews Non-systematic Subjective 2. Evidence-based reviews Systematic review of scientific literature Includes ‘grey’ literature

Systematic reviews l A systematic review brings together the findings from a range of research reports done around a specific intervention l A systematic review is explicit in its reporting of the search for studies and the criteria for including and excluding studies l Systematic reviews may or may not include a meta-analysis on the findings.

Systematic reviews reduce bias by: (1) l Using an explicit detailed search strategy to find as many published and unpublished reports (grey literature) of relevant trials as possible l Conducting reviews according to a written protocol l Using explicit, pre-specified inclusion and exclusion criteria for determining which trials are reviewed l Using set methods to assess the quality of each trial

Systematic reviews reduce bias by: (2) l Employing at least two people to independently extract the data l Analysing trials using meta-analysis (using actual numerical results) where possible l Presenting the review in a clear and transparent manner In summary, systematic reviews attempt to give the best estimate of what the result of all relevant and appropriately conducted trials mean.

Search strategies Which database? (Pub. Medline all information) 1. 2. 3. 4. 5. 6. Systematic review databases Cochrane SBU CDC… Database with original studies and reviews Reference list (paper), grey literature, conference abstracts, Internet Limit list to relevant studies Quality check of the remaining references Write down your search words and protocols

Systematic reviews: a (good) example Efficacy of Interventions To Modify Dietary Behavior Related to Cancer Risk Agency for Healthcare Research and Quality (AHRQ) (www. ahcpr. gov/clinic/epcsums/dietsumm. htm)

Reporting the Evidence: 3 main questions • Is there evidence that one type of intervention or combination of interventions is more effective than another in helping individuals or groups modify their diets to consume less fat? • What is the evidence for the efficacy of dietary interventions by population subgroup, particularly groups defined by ethnicity and sex? • What conclusions (if any) can be reached about the cost -effectiveness of these types of interventions?

Methodology (1) Six databases: MEDLINE, EMBASE, Psyc. INFO, CINAHL, AGELINE, and AGRICOLA Search strategy excluded studies: • • • published before 1975 or in languages other than English conducted outside of North America, Europe, or Australia conducted with infants, institutionalized populations, or populations with insulin-dependent diabetes mellitus. with sample sizes of fewer than 40 subjects at follow-up. in which dietary intake was externally controlled

Methodology (2) Three sets of secondary analyses: • A meta-analysis (for the outcome of total fat as a percentage of energy intake) • A standardized, quantitative analysis of the differences in dietary changes between intervention and comparison or control groups from baseline to follow-up • A semi quantitative analysis summarizing the statistical significance of the intervention effects

Findings (general) • considerable evidence regarding the efficacy or effectiveness of different types or components of interventions in helping individuals or groups modify their dietary intake • Very few studies were appropriately designed or reported their findings to permit interpretation of the evidence for the efficacy of interventions by subgroup, particularly lowincome or ethnic subgroups • No studies that met the authors' review criteria provided data on the cost-effectiveness of dietary interventions.

Dietary Fat- results • nearly 80 percent (80/104) of the articles reviewed reported results for dietary fat • similar decreases in intake of total fat and saturated fat • median difference between intervention and control groups in the change in total fat intake translates into a 7. 3 percent reduction in the percentage of calories from fat.

Serum cholesterol • the decrease in total fat intake was significantly correlated with concomitant decreases in total blood cholesterol (r = 0. 763, p = 0. 004) • the change in saturated fat was not associated with statistically significant decreases in total blood cholesterol.

Conclusions l If we aim to show effectiveness of PHN interventions and policy implications, we need to work evidence based in PHN – which is not the same as evidence based medicine! l In order to be seen and heard we need to develop our own criteria for high quality interventions l We need to use assessment techniques that are valid and appropriate l We need to show evidence of cost effectiveness