Laurie Anderson Ph D MPH Centers for Disease

Laurie Anderson, Ph. D, MPH Centers for Disease Control & Prevention February 10, 2004

Outline I. Difference between a literature review and a systematic review II. Purpose of the Guide to Community Preventive Services: systematic reviews & evidence-based recommendations III. Steps in carrying out a systematic reviews IV. An example

The difference between a literature review and a systematic review

When making decisions about the choice of an intervention The body of intervention literature can be quite large, inconsistent, and uneven in quality.

Literature Reviews A “literature review” has typically been used to provide background information for intervention selection. These reviews present a group of studies, with strengths and weaknesses discussed selectively and informally.

Traditional literature reviews have several shortcomings: • The process is subjective. There are few formal rules, two reviewers might reach different conclusions. • Lack explicit criteria for excluding inappropriate or poorly done studies.

Typically a literature review counts the number of studies supporting, or not supporting, an intervention i. e. positive, negative, or no effect but ignores sample size, effect size and research design. This can lead to erroneous conclusions about intervention effectiveness.

In literature reviews conflicting findings may lead to a conclusion that an intervention is ineffective or the research is uninterpretable. Systematic reviews exploit divergent findings by examining potential explanations --- treatment differences, setting differences, etc. --- because conflicting outcomes may tell us where an intervention is likely to succeed or fail.

Systematic reviews use numeric and narrative information fully: • a small effect across several studies may be significant • program characteristics can be used to explain the effect.

Literature reviews are an inefficient way to extract program & outcome information, particularly if the number of studies is large, e. g. >30. • It is impossible to mentally juggle relationships among so many variables.

Systematic Reviews Another approach to the literature is systematic reviews. A systematic review takes in account: • the precise purpose of the review i. e. stating a research hypothesis • how studies are selected & included

Systematic reviews can answer: • is there publication bias? • are intervention programs similar enough to combine? • what is the distribution of study outcomes?

Systematic review can answer: • are outcomes related to research design? • are outcomes related to characteristics of programs, participants, and setting? • what are the needs for future research?

Purpose of the Guide to Community Preventive Services: systematic reviews & evidence-based recommendations

Are we building on a foundation of existing knowledge? Explosive growth of scientific information • too much to keep up with • contradictory results Increasing public doubt about scientific findings

Systematic reviews for research synthesis • combine many studies with different methods and results • look for consistencies in set of findings • more robust than single study • may pinpoint why studies differ • shows what is effective and why

What counts as evidence? Type 1 Type 2 Determinants or associations between risk and an outcome Relative effectiveness of different interventions “Something should be done” “This should be done”

The Guide to Community Preventive Services What strategies, targeted to which groups will: • promote healthy choices? • prevent disease and injury? • improve environmental conditions to promote health?

Community Guide Topics Environmental Influences • Sociocultural Environment • Physical Environment Risk Behaviors Specific Conditions • Tobacco Use • Alcohol Abuse/Misuse • Other Substance Abuse • Poor Nutrition • Inadequate Physical Activity • Unhealthy Sexual Behaviors • Vaccine Preventable Disease • Pregnancy Outcomes • Violence • Motor Vehicle Injuries • Depression • Cancer • Diabetes • Oral Health

Central Questions • • • program effectiveness feasibility of implementation acceptability to the population unanticipated harms (or benefits) cost-effectiveness

Steps to Conducting a Review • • Assemble a review team Develop conceptual framework Prioritize review topics Define specific intervention for review Search for and retrieve evidence Rate quality of evidence Summarize evidence Translate into a recommendation

Assemble a Review Team • Multiple perspectives and backgrounds – Improve completeness and accuracy of information – Reduce impact of individual/institutional perspectives – Enhance usefulness of products

Develop Conceptual Approach to the Topic • The Logic Model

LOGIC FRAMEWORK ILLUSTRATING CONCEPTUAL APPROACH TO NUTRITION & COMMUNITY HEALTH Interventions Modifiable Determinants Intermediate Outcomes Community Health Outcomes POPULATION FOOD INTAKE Food Supply Factors -Agriculture policy -Nutrition policy -Science and technology -Food production, processing, storage and distribution -Food fortification -Food safety Environmental Factors Availability & Price -Neighborhoods -Schools -Worksites -Homes -State and National -Food and nutrition assistance programs Food Consumption Patterns (e. g. fruits, vegetables) Intake of Nutrients and Food Components Vitamins Minerals Fiber Fats Other food constituents Dietary supplements Alcohol Energy balance Consumer Demand -Household resources -Nutrition knowledge -Cultural practices -Psychosocial characteristics -Taste and preferences -Advertising and marketing Physical Activity Patterns Life Stage Requirements Pregnancy Lactation Childhood Adolescence Adulthood Older Adulthood Physiologic Indicators Growth Adipose tissue Musculoskeletal Gastrointestinal Metabolic Cardiovascular Reproductive Immunological Neurological Genetics, Co-morbidities Morbidity Mortality Measures of Health & Fitness

Prioritize Intervention Topics • • Preventable disease burden Common practices that are questionable New approaches that are promising Topic of keen public health interest

Priority Ranked List of Topics 1. Food & beverage availability and price in schools. 2. Comprehensive community approaches to increase fruity & vegetable intake. 3. Food and beverage advertising to children. 4. Food & beverage availability, price, portion size, and labeling in restaurants. 5. Tax on sodas and snack foods. 6. Farm subsidies and production of fruits, vegetables, & whole grains.

7. Food choice and nutrition education in food assistance programs. 8. Nutrition and weight management counseling in healthcare settings. 9. Breast-feeding. 10 Product labeling in grocery stores, restaurants and vending machines. 11. Food & beverage availability and price in worksites. 12. Use of dietary supplements across the lifespan.

Specify the Review Question • • What population? What interventions or risk factors? What comparisons? What outcomes?

Example Question Do multi-component, school-based nutrition interventions which may include: – curricula (nutrition & physical activity) – food availability, accessibility, price – policy and environmental changes improve nutritional behavior and nutritional status of children and adolescents in developed countries?

Analytic Framework Behaviors dietary intake physical activity School Policy Environmental support for healthy choices Multicomponent School-based Nutrition Interventions Nutrition & health messages Promotion of self-awareness, self-efficacy Knowledge nutritional needs food content Attitudes self-care body image Abilities self-assessment behavioral change skills media literacy Physiologic Indicators body size & composition growth fitness Health status blood pressure cholesterol School Achievement attendance participation performance

Inclusion Criteria • Characteristics derived from the focused question • Additional characteristics – Methodological – Publication dates – Languages – Relevant outcomes

Find Information Select strategies for finding studies: – Database searches – Reviews of reference lists – Gray literature – Consultation with experts

Assess Quality • Suitability of study design • see handout • Quality of study execution • see handout

Suitability of Study Design • Greatest – Prospective and – Concurrent comparison • Moderate – Retrospective or – Multiple measurements over time; no concurrent comparison • Least – Single before and after measurement; no concurrent comparison

Strength of Study Execution • Description – Population – Intervention • Sampling • Measurement – Exposure – Outcome • Analysis • Interpretation of results • Other

Exclude Studies Below Some Quality Threshold • Exclude studies with limited execution (i. e. , with many important threats to validity) • Sometimes exclude studies with least or moderate suitability of design

Research Synthesis for Public Health Policy & Practice Decisions: Systematic review of United States studies of. 08 blood alcohol concentration laws

Lowering legal blood alcohol concentration (BAC) limits • Alcohol-related crashes cause 16, 000 deaths and 300, 000 injuries each year in the US • Laws that lower BAC from 0. 10 to 0. 08 existed in less that half of US states

Analytic Framework: Lower Blood Alcohol Concentration Laws Lower B. A. C. Laws Perceived Risk of Arrest Social norms regarding drinking and driving Drinking & driving behaviors OUTCOME Alcoholrelated crashes Fatal & non-fatal injuries

Evidence base for. 06 BAC Laws Searched journal articles, technical reports and conference proceedings. Nine studies were identified evaluated state BAC laws– all of sufficient design and quality. All studies reported data from police incident reports of crashes on public roadways.

Evidence base for. 06 BAC Laws Eight of the nine studies reported percent change in alcohol-related fatalities (post-law period vs. pre-law period. Other outcome data included public knowledge and perception of impaired driving laws, selfreport of impaired driving, impaired driving arrests.

Percent Change in Measures of Alcohol -Related Motor Vehicle Fatalities, by State Median percent change: -7% Interquartile range: -15%, -4% States 15 States (1)* VA (2) VT (3, 4, 5) UT (5, 3, 4) OR (4, 5, 3) NC (6, 2) NM(2) NH (2) ME(4, 5, 3) KS (2) FL (2) CA (5, 7, 8, 3, 4) – 50 -40 -30 -20 -10 20 30 Percent Change Median percent change: -7%

States with. 08 BAC Laws, 2002 Passed. 08 BAC law before the new national standard Passed. 08 BAC law in 2001 Passed. 08 BAC law in 2002

Who Is the Audience? • Public health departments • Health care systems • Purchasers of health care • Government and foundations • Community organizations

Lack of Persuasive Evidence • Lack of evidence does not mean that interventions don’t work • Insufficient evidence may point to a research agenda

Issues of Evidence • We don’t have evidence about everything • Enormous amount of evidence yet to review • New evidence may change recommendations • Capturing complexity • Urgent needs and limited resources • Participatory research

www. The. Community. Guide. org