What evidence informs our practice Roger F Soll

What evidence informs our practice? Roger F. Soll, MD H. Wallace Professor of Neonatology Larner College of Medicine, University of Vermont Coordinating Editor, Cochrane Neonatal President, Vermont Oxford Network Trusted evidence. Informed decisions. Better health. 1

Editorial Team Roger F. Soll Coordinating Editor Colleen Ovelman Managing Editor Caitlin O’Connell Assistant Managing Editor 2

Editorial Team Michael Bracken Yale University Jeffrey Horbar Bill Mc. Guire University of Vermont Hull York Medical School Gautham Suresh Baylor University 3

Associate Editors 4

Guest Discussants Danielle Ehret, MD, MPH Assistant Professor, University of Vermont Director, Global Health, Vermont Oxford Network Deirdre O'Reilly, MD, MPH Assistant Professor, University of Vermont Director, NPM Fellowship, University of Vermont

Sponsors Section on Neonatal-Perinatal Medicine

What Evidence Informs Our Practice? Roger F. Soll, M. D. is the President of the Vermont Oxford Network and the Coordinating Editor of Cochrane Neonatal No other relevant financial issues to disclose

What Evidence Informs Our Practice? To develop an understanding of the strengths and weaknesses of various sources of evidence used to inform our practice of neonatal-perinatal medicine.

Evidence Based Medicine “The conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. ” Sackett DL, et al. Evidence-Based Medicine: How to Practice and Teach EBM. Churchill-Livingstone. 1997.

Evidence Based Medicine

Changes in Delivery Room Practice Vermont Oxford Network 2001 to 2009 Soll and colleagues. Obstetric and Neonatal Care Practices for Infants 501 to 1500 g From 2000 to 2009. Pediatrics 2013. 132. 10. 1542/peds. 2013 -0501.

Vermont Oxford Network Infants Gestational Age 27 to 29 Weeks Interquartile Ranges 2017 Lowest Quartile Highest Quartile Antenatal Steroids Cesarean Delivery 80% 64% 97% 84% DR CPAP Tracheal Intubation DR Surfactant 22% 65% 0% 56% 97% 36% Over 22, 000 Infants at NICUs in the Vermont Oxford Network

Vermont Oxford Network Infants Gestational Age 27 to 29 Weeks Interquartile Ranges 2017 Mortality CLD @ 36 wks Pneumothorax Severe IVH Lowest Quartile Highest Quartile 0% 7% 0% 0% 11% 31% 6% 7% Over 22, 000 Infants at NICUs in the Vermont Oxford Network

Evidence Based Medicine If we are all reading the same information… Why aren’t we operating from the same playbook?

What evidence should I use to inform my practice?

The Spectrum of Evidence Strength of the evidence: Perry Mason • Multiple eyewitnesses • One sober eyewitness who got a good look • Physical evidence at the crime scene • Pattern of previous criminal activity • Round up the usual suspects

The Evidence Hierarchy EBM hierarchies rank study types based on the strength and precision of their research methods. Different hierarchies exist for different question types and experts may disagree on the exact rank of information in the evidence hierarchies. However, most agree that current, well designed systematic reviews and meta-analyses are at the top of the pyramid, and that expert opinion and anecdotal experience are at the bottom.

Classifying the Quality of the Evidence 1. Systematic review of multiple well designed randomized controlled trials. 2. Properly designed randomized controlled trial of appropriate size. 3. Well-designed trials without randomization 4. Well-designed non-experimental studies 5. Opinions of respected authorities (based on clinical evidence, descriptive studies or reports of expert committees) (no longer seen as “evidence”)

The Evidence Hierarchy

Hierarchy of Study Designs for Intended Effects of Therapy 1. 2. 3. 4. 5. Randomized Controlled Trials Prospective Follow-up Studies Retrospective Follow-up Studies Case-Control Studies Anecdotal: Case Reports and Series Hierarchy of Study Designs for Discovery and Explanation 1. 2. 3. 4. 5. Anecdotal: Case Reports and Series Case-Control Studies Retrospective Follow-up Studies Prospective Follow-up Studies Randomized Controlled Trials Vandenbroucke JP (2008). "Observational research, randomised trials, and two views of medical science". PLo. S Med. 5 (3): e 67. doi: 10. 1371/journal. pmed. 0050067. PMC 2265762. PMID 18336067.

Evidence Based Medicine The strengths and weaknesses of the evidence • clinical impressions • case series without formal controls • studies with formal controls • randomized controlled trials • meta-analyses

Clinical Impressions Fundamentally the source of all discovery Also, the source of all erroneous conclusions

Clinical Impressions One case: “in my experience…” Two cases: “in case after case…” “time and again…” Three cases: “in my series…”

Clinical Impressions During the 1960’s, obstetricians interested in parturition noted that lambs whose mothers were given corticosteroids in an attempt to initiate labor, had less respiratory distress and increased survival… followed by detailed randomized controlled trials both in the animal model and in humans.

Clinical Impressions During the 1950’s, obstetricians reported the successful use of diethylstilbestrol (DES) in maintaining the pregnancy of a woman who previously had a series of miscarriages…. followed by widespread use despite limited data from randomized controlled trials

Cases and Case Series Without Formal Controls Sometimes the outcome of current forms of therapy can be predicted with such certainty that past experience provides a valid basis for interpreting current observation

Hazardous Journeys Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials Smith Gordon C S, Pell Jill P. BMJ 2003; 327 doi: https: //doi. org/10. 1136/bmj. 327. 7429. 1459

Hazardous Journeys Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials Objectives: To determine whether parachutes are effective in preventing major trauma related to gravitational challenge. Design: Systematic review of randomized controlled trials. Data sources: Medline, Web of Science, Embase, and the Cochrane Library databases; appropriate internet sites and citation lists. Study selection: Studies showing the effects of using a parachute during free fall. Main outcome measure: Death or major trauma, defined as an injury severity score > 15. Results: We were unable to identify any randomized controlled trials of parachute intervention.

Conclusions As with many interventions intended to prevent ill health, the effectiveness of parachutes has not been subjected to rigorous evaluation by using randomized controlled trials. Advocates of evidence based medicine have criticized the adoption of interventions evaluated by using only observational data. We think that everyone might benefit if the most radical protagonists of evidence based medicine organized and participated in a double-blind, randomized, placebo controlled, crossover trial of the parachute.

Look Before You Leap Parachute use to prevent death and major trauma when jumping from aircraft: randomized controlled trial PArticipation in RAndomized trials Compromised by widely Held beliefs abo. Ut lack of Treatment Equipoise (PARACHUTE) Trial Yeh and colleagues. BMJ 2018; 363: k 5094

Look Before You Leap Parachute use to prevent death and major trauma when jumping from aircraft: randomized controlled trial Yeh and colleagues. BMJ 2018; 363: k 5094

Parachute use to prevent death and major trauma when jumping from aircraft: randomized controlled trial Yeh and colleagues. BMJ 2018; 363: k 5094 Parachute use did not significantly reduce death or major injury (0% for parachute v 0% for control; P > 0. 9). This finding was consistent across multiple subgroups. Compared with individuals screened but not enrolled, participants included in the study were on aircraft at significantly lower altitude (mean of 0. 6 m for participants v mean of 9146 m for non-participants; P < 0. 001) and lower velocity (mean of 0 km/h v mean of 800 km/h; P < 0. 001). “Representative study participant jumping from aircraft with an empty backpack. This individual did not incur death or major injury upon impact with the ground”

Cases and Case Series Without Formal Controls If a particular form of care is associated with an abnormal outcome that has not been seen before, observations without formal controls may be an adequate basis for abandoning the form of care in question

The Thalidomide Disaster Thalidomide was first marketed in 1957 in West Germany. Primarily prescribed as a sedative or hypnotic, thalidomide also claimed to cure "anxiety, insomnia, gastritis, and tension". Afterwards, it was used against nausea and to alleviate morning sickness in pregnant women. Thalidomide became an over-the-counter drug in West Germany on October 1, 1957.

Shortly after the drug was sold in West Germany, between 5, 000 and 7, 000 infants were born with phocomelia (malformation of the limbs). Only 40% of these children survived.

“Heroine” of FDA Keeps Bad Drug Off of Market Washington Post 1962 Frances Oldham Kelsey, Ph. D

E-Ferol Injection In April 1984, the US FDA was notified of an unusual clinical syndrome consisting of ascites, liver and renal failure, thrombocytopenia, and death among low birth weight infants exposed to an intravenous vitamin E preparation, E-Ferol. The product, which had not been tested for safety prior to marketing, was voluntarily withdrawn from the market in early April.

Morbidity and mortality among low birth weight infants exposed to an intravenous vitamin E product, E-Ferol. To further investigate the reported associations, the FDA conducted a retrospective cohort study among seven neonatal intensive care units where the product had been used. Included in the study were 379 infants weighing 2, 000 grams or less and surviving at least two days; 148 (39%) had been exposed to E-Ferol. Compared with the unexposed infants, the exposed infants were more likely to die and to have ascites, hepatomegaly, thrombocytopenia, and a combination of clinical events similar to the syndrome initially reported. Arrowsmith and colleagues. Pediatrics. 1989 Feb; 83(2): 244 -9.

Cases and Case Series Without Formal Controls • although past experience suggests that caution is appropriate in interpreting the results of uncontrolled case series, it is often not exercised • prone to selective reporting • enthusiasm for treatment has been shown to be inversely related to number of patients studied

Evidence Based Medicine “Therapeutic reports with controls tend to have no enthusiasm, and reports with enthusiasm tend to have no controls” - Sackett 1986

Studies With Formal Controls • Historical controls • Case controls • Non-randomized concurrent controls • Randomized controls

Historical Controls Comparison between people who have received a relatively recently introduced form of care with other individuals cared for in a different way during an earlier era

Red blood cell transfusion, feeding and necrotizing enterocolitis in preterm infants. OBJECTIVE: Preliminary studies suggested an association between red blood cell (RBC) transfusion and necrotizing enterocolitis (NEC) in premature neonates. An advantageous effect of withholding feeds during transfusion has never been studied. We aimed, first, to determine whether preterm infants who developed NEC were more likely to be transfused in the 48 to 72 h before the diagnosis of NEC; second, to test if a strict policy of withholding feeds during transfusion would decrease the incidence of transfusion-associated NEC. El-Dib M and colleagues. Red blood cell transfusion, feeding and necrotizing enterocolitis in preterm infants. J Perinatol. 2011 Mar; 31(3): 183 -7. doi: 10. 1038/jp. 2010. 157.

Red blood cell transfusion, feeding and necrotizing enterocolitis in preterm infants. STUDY DESIGN: The study was conducted in two phases. Phase 1: a retrospective case-control study of premature low birth weight (< 32 weeks and < 2500 grams) infants who developed NEC over a 6 -year period. Phase 2: a comparison study of the incidence of NEC during the 18 -months preceding, and the 18 months following the change of practice to withholding feeds during RBC transfusion. El-Dib M and colleagues. Red blood cell transfusion, feeding and necrotizing enterocolitis in preterm infants. J Perinatol. 2011 Mar; 31(3): 183 -7. doi: 10. 1038/jp. 2010. 157.

The incidence of necrotizing enterocolitis during the 18 -months preceding and the 18 months following the change of practice to withholding feeds during RBC transfusion. Implementing the policy of withholding feeds during transfusion was associated with a decrease in the incidence of NEC from 5. 3 to 1. 3% (P=0. 047). El-Dib M and colleagues. Red blood cell transfusion, feeding and necrotizing enterocolitis in preterm infants. J Perinatol. 2011 Mar; 31(3): 183 -7. doi: 10. 1038/jp. 2010. 157.

Historical Controls Randomized (RCT) vs Historical (HCT) Controlled Trials Six Therapies: 50 RCT’s, 56 HCT’s • 79% Historical Controlled Trials supported intervention • 20% Randomized Controlled Trials supported intervention Sacks 1982

Historical Controls • When substantial differences in outcome are noted between two different time frames, this may only reflect changes in other undocumented factors that have modified outcome • Inferences based solely on studies that use historical controls tend to lead to conclusions that new forms of care effective, when less biased comparisons suggest that they are not, or that the estimate is exaggerated

Case Controls Studies that start by identifying persons with and without a disease of interest (cases and controls, respectively) and then look back in time to find differences in exposure to risk factors.

An outbreak of necrotizing enterocolitis. Association with transfusions of packed red blood cells. Of 187 newborns admitted to a 33 -bed, level III neonatal intensive care unit between 1/1/1985 and 6/23/1985, 33 developed necrotizing enterocolitis during their hospital stay. A case-control study, with matching on birth weight, date of admission and duration of stay, failed to reveal any association of the syndrome with type or timing of feeding, perinatal hypoxic events, or specific microbial organisms. By contrast, however, transfusion of packed red blood cells was highly and significantly associated with the syndrome (odds ratio = 15. 1, 95% confidence interval = 2. 59 to 92. 51). During this outbreak period, the incidence of necrotizing enterocolitis by birth weight was 30. 6% in infants less than 1, 500 gm, 10. 8% in infants 1, 500 to 2, 500 grams, and 11. 9% in infants 2, 500 gm or more. Mc. Grady GA and colleagues. Am J Epidemiol. 1987 Dec; 126(6): 1165 -72.

Non-Randomized Concurrent Controls Comparison of two or more cohorts of individuals who happen to have received alternative forms of care concurrently

Reducing Nosocomial Infection Ignaz Philipp Semmelweis • Hungarian physician • Puerperal fever • Hand washing 1818 -1865 • Maternal Mortality

Non-Randomized Concurrent Controls Two wards in hospital were on intake for admissions on different days of the week • one staffed by medical students and physicians • the other by midwives and midwifery students Astonishingly different rates of puerperal infection: • Women delivered by physicians and medical students (13 to 18%) vs. • Women delivered by midwife trainees or midwives (2%).

Reducing Nosocomial Infection QI Initiative Semmelweis Outcome Courtesy of R. Pfister

Non-Randomized Concurrent Controls Maternal Mortality in a Religious Group Avoiding Obstetric Care 1975 -1982 Members Other Indiana Residents Total Births 334 681, 142 Maternal Mortality Rate/ 100, 000 Livebirths 872 9 Kaunitz AM and colleagues. Perinatal and maternal mortality in a religious group avoiding obstetric care. Am J Obstet Gynecol. 1984 Dec 1; 150(7): 826 -31.

Non Randomized Concurrent Controls Delivery of the extremely low birth weight infant: cesarean section or vaginal delivery? - Many reports of improved outcome with cesarean section - Does not account for vastly different risk of death or morbidity in the two groups

Impact of cesarean section on neonatal mortality rates among very preterm infants in the United States, 2000 -2003 OBJECTIVE: To compare the neonatal mortality rates for infants delivered through primary cesarean section versus vaginal delivery, taking into consideration a number of potentially risk-modifying conditions. Malloy MH. Pediatrics. 2008 Aug; 122(2): 285 -92.

OUR CHANGING PRACTICE… IS IT EVIDENCE BASED? Cesarean Section Demographic, medical, and labor and delivery complications were abstracted from US linked birth and infant death certificate files for 2000 -2003. 13, 733 neonatal deaths and 106, 809 survivors available from the trimmed data set for analysis for the 4 -year period. Malloy MH. Pediatrics. 2008

OUR CHANGING PRACTICE… IS IT EVIDENCE BASED? Cesarean Section Risk of Neonatal Death Gestational Age Adjusted odds ratios and 95% CI 22 weeks 23 weeks 24 weeks 25 weeks 0. 58 (0. 38 to 0. 87) 0. 52 (0. 42 to 0. 64) 0. 72 (0. 62 to 0. 82) 0. 81 (0. 69 to 0. 94) Malloy MH. Pediatrics. 2008.

Cesarean Section VERMONT OXFORD NETWORK ANNUAL REPORTS 2000 -2017

Non-Randomized Concurrent Controls Bias from unrecorded and often unrecognized risk factors will undermine confidence in causal inferences based on comparisons of different forms of care using non-randomized concurrent controls

Randomized Controlled Trials Minimizing bias……

Randomized Controlled Trials

Randomized Controlled Trials Scurvy “On the 20 th of May 1747, I took twelve patients in the scurvy, on board the Salisbury at sea. Their cases were as similar as I could have them. They all in general had putrid gums, the spots and lassitude, with weakness of their knees. They lay together in one place, being a proper apartment for the sick in the fore-hold; and had one diet common to all…”

Randomized Controlled Trials • • • cider elixir vitriol vinegar sea water oranges/lemons nutmeg

Randomized Controlled Trials “The consequence was that the most sudden and visible good effects were perceived from the use of the oranges and lemons; one of those who had taken them, being at the end of six days, fit for duty. ” James Lind 1753

The Real World Results Lind's therapeutic findings made little impact on medical opinion in Britain. The year after publication of the treatise, the Navy's 'Sick and Hurt Board' rejected a proposal to provide sailors with supplies of fruit juice. It was not used for 30 years…

Randomized Controlled Trials Need for randomization: • likely to provide equivalent groups at study entry • provides theoretical basis for statistical comparisons • minimizes selection bias in treatment assignment

Randomized Controlled Trials Minimize systematic error (bias) • • Study entry (selection bias) Exposure to intervention (performance bias) Completeness of follow up (exclusion bias) Measurement of outcomes (assessment bias)

Limitations of Randomized Controlled Trials • • Evaluation of preventive therapy Multiple therapeutic candidates Minor changes in therapeutic agents “Instability” of available therapy Long-term adverse effects of therapy Evaluation of etiologic agents Evaluation of diagnostic technology Evaluation of process or structure

Randomized Controlled Trials The need for collaborative research: sample size requirements effect size 10% 15% 20% 30% change in rate 25% to 22. 5% 25% to 21. 25% to 20% 25% to 17. 5% sample size 9254 3574 2268 986

Systematic Overview - Applies specific research strategies to identify, appraise, and synthesize data from all relevant clinical studies Quantitative systematic reviews include meta-analyses: - statistical methods to combine the results of similar randomized controlled trials to produce a typical estimate of the effect size

Meta-Analysis Meta-analysis demands the same methodological quality expected in a randomized controlled trial: • prospectively designed protocol • comprehensive and explicit search strategy • strict criteria for inclusion of studies • standard definitions of outcomes • standard statistical techniques

Meta-Analysis What’s the use of meta-analysis? • increase statistical power • increase precision of estimate • explore differences between study results • create structure for incorporating new evidence

Preparing, maintaining and promoting the accessibility of systematic reviews of the effects of health care interventions www. cochrane. org

Why we need observational studies to evaluate the effectiveness of health care. Experimentation may be unnecessary Experimentation may be inappropriate • Rarely large enough to measure infrequent adverse outcomes • Difficulty in evaluating interventions designed to prevent rare events • Outcome of interest far in the future • RCT may be inherently “self-defeating” (patient engagement) Experimentation may be impossible (social and political) Experimentation may be inadequate (external validity) Black N. BMJ 1996; 312(7040): 1215 -1218. doi: 10. 1136/bmj. 312. 7040. 1215.

Comparison of evidence of treatment effects in randomized and nonrandomized studies Objectives: To compare results of randomized and nonrandomized studies that evaluated medical interventions and to examine characteristics that may explain discrepancies between randomized and nonrandomized studies. Study Selection: Forty-five diverse topics were identified for which both randomized trials (n = 240) and nonrandomized studies (n = 168) had been performed and had been considered in meta-analyses of binary outcomes. Data Synthesis: Very good correlation was observed between the summary odds ratios of randomized and nonrandomized studies (r = 0. 75; P <. 001); however, nonrandomized studies tended to show larger treatment effects (28 vs 11; P =. 009). The summary results of the 2 types of designs differed beyond chance in 7 cases (16%). Conclusions: Despite good correlation between randomized trials and nonrandomized studies—in particular, prospective studies—discrepancies beyond chance do occur and differences in estimated magnitude of treatment effect are very common. Ioannidis JP and colleagues. JAMA 2001. 286 (7): 821– 30. doi: 10. 1001/jama. 286. 7. 821.

Welcome to the Live Classroom Session How to Participate? • Follow the slides on your screen. • Listen to the Audio Broadcast via your computer speakers. If the computer audio is not working well, click at the bottom of the Participants panel and follow the prompts to call in on the telephone. • Interactive Classroom: • • Live Poll Everywhere questions • Send questions and comments via Chat to “All Panelists”

Guest Discussants Danielle Ehret, MD, MPH Assistant Professor, University of Vermont Director, Global Health, Vermont Oxford Network Deirdre O'Reilly, MD, MPH Assistant Professor, University of Vermont Director, NPM Fellowship, University of Vermont

How can I prove to the world that ECMO works for babies? Dr. Robert Bartlett Director of the Surgical Intensive Care Unit, Program Director of the Surgical Critical Care Fellowship and Director of the Extracorporeal Life Support Program at the University of Michigan Medical Center

Extracorporeal membrane oxygenation (ECMO) cardiopulmonary support in infancy. We have used prolonged extracorporeal membrane oxygenation (ECMO) in the treatment of 13 moribund infants (including 9 neonates), with 4 survivors (3 neonates). Successfully treated cases include post-operative cardiac failure, infant respiratory distress syndrome, massive meconium aspiration, and persistent fetal circulation. All cases have been managed with veno-arterial bypass at flow rates of 80 -100 cc/Kg/min. Carotid cannulation for arterial access and careful control of heparin anticoagulation based on whole blood activated clotting time are among the techniques which have contributed to this success. Progressive pulmonary or cardiac failure has been the major problem in older infants, intracranial bleeding is the major problem in neonates. Both of these problems could be minimized by instituting ECMO earlier in the clinical course, but this awaits development of reliable early predictors of mortality. Bartlett RH and colleagues. Trans Am Soc Artif Intern Organs. 1976; 22: 80 -93.

Extracorporeal membrane oxygenation (ECMO) cardiopulmonary support in infancy. What type of study is this? • • • Case series without formal controls Case control study Historical control study Non-randomized concurrent control study Randomized controlled study

Extracorporeal membrane oxygenation (ECMO) cardiopulmonary support in infancy. Does this study convince me to use ECMO in late preterm or term infants with severe respiratory failure? • Yes • No

Extracorporeal circulation (ECMO) in neonatal respiratory failure. Sixteen moribund newborn infants with respiratory failure were treated with extracorporeal membrane oxygenation (ECMO) for 1 to 8 days. Diagnoses and results were as follows: respiratory distress syndrome (2 improved, 1 survived); meconium aspiration syndrome (4 improved, 3 survived); persistent fetal circulation (some with diaphragmatic hernia)(3 improved, 2 survived). Intracranial bleeding occurred in 43 percent, accounting for most of the deaths. In a parallel series of 21 infants treated with conventional ventilator therapy, the mortality rate was 90 percent, and intracranial bleeding occurred in 57 percent. ECMO provided life support and gains time in newborn respiratory failure. In high mortality risk infants, the rate of survival is higher and intracranial bleeding lower with ECMO than with optimal ventilator management. Bartlett RH and colleagues. J Thorac Cardiovasc Surg. 1977 Dec; 74(6): 826 -33.

Extracorporeal circulation (ECMO) in neonatal respiratory failure. Bartlett RH, Gazzaniga AB, Huxtable RF, Schippers HC, O'Connor MJ, Jefferies MR. J Thorac Cardiovasc Surg. 1977 Dec; 74(6): 826 -33. What type of study is this? • • • Case series without formal controls Case control study Historical control study Non-randomized concurrent control study Randomized controlled study

Extracorporeal circulation (ECMO) in neonatal respiratory failure. Bartlett RH, Gazzaniga AB, Huxtable RF, Schippers HC, O'Connor MJ, Jefferies MR. J Thorac Cardiovasc Surg. 1977 Dec; 74(6): 826 -33. Does this study convince me to use ECMO in late preterm or term infants with severe respiratory failure? • Yes • No

Extracorporeal Circulation in Neonatal Respiratory Failure: A Prospective Randomized Controlled Study A prospective controlled randomized study of the use of extracorporeal membrane oxygenation to treat newborns with respiratory failure was carried out using the "randomized play-the-winner" statistical method. In this method the chance of randomly assigning an infant to one treatment or the other is influenced by the outcome of treatment of each patient in the study. If one treatment is more successful, more patients are randomly assigned to that treatment. A group of 12 infants with birth weight greater than 2 kg met objective criteria for high mortality risk. Bartlett RH and colleagues. Pediatrics. 1985 Oct; 76(4): 479 -87.

Play the winner Chow SC, Chang M. Adaptive design methods in clinical trials – a review. Orphanet J Rare Dis. 2008; 3: 11.

Extracorporeal Circulation in Neonatal Respiratory Failure: A Prospective Randomized Controlled Study One patient was randomly assigned to conventional treatment (that patient died); 11 patients were randomly chosen for extracorporeal membrane oxygenation (all survived). Intracerebral hemorrhage occurred in one of 11 surviving children. Extracorporeal membrane oxygenation allows lung rest and improves survival compared to conventional ventilator therapy in newborn infants with severe respiratory failure. Bartlett RH and colleagues. Pediatrics. 1985 Oct; 76(4): 479 -87.

Bartlett and colleagues. Extracorporeal Circulation in Neonatal Respiratory Failure: A Prospective Randomized Controlled Study. Pediatrics 1985; 76(4): 479 -87 CM: died ECMO: survived ECMO: 10 survived CM: 1 died Adapted from Truog “Ethical Conflicts in Randomized Controlled Trials”

Extracorporeal Circulation in Neonatal Respiratory Failure: A Prospective Randomized Controlled Study What type of study is this? • • • Case series without formal controls Case control study Historical control study Non-randomized concurrent control study Randomized controlled study Bartlett RH and colleagues. Pediatrics. 1985 Oct; 76(4): 479 -87.

Extracorporeal Circulation in Neonatal Respiratory Failure: A Prospective Randomized Controlled Study Does this study convince me to use ECMO in late preterm or term infants with severe respiratory failure? • Yes • No Bartlett RH and colleagues. Pediatrics. 1985 Oct; 76(4): 479 -87.

National experience with extracorporeal membrane oxygenation for newborn respiratory failure. Data from 715 cases. In a national registry, data were collected on 715 newborn patients with severe respiratory failure supported by extracorporeal membrane oxygenation (ECMO) in 18 neonatal centers. This represents almost all infants treated with ECMO between 1980 and 1987. Eighty-one percent of the patients survived. This result is statistically significantly better than any other treatment which produces less than 78. 4% survival. These observations were made on a large cohort that could not be accumulated at an individual center. These results indicate that ECMO and lung rest is appropriate and successful treatment for newborn respiratory failure unresponsive to other means of management and that almost all respiratory failure is reversible in near-term neonates. Toomasian JM and colleagues. ASAIO Trans. 1988 Apr-Jun; 34(2): 140 -7.

National experience with extracorporeal membrane oxygenation for newborn respiratory failure: Data from 715 cases. What type of study is this? • • • Case series without formal controls Case control study Historical control study Non-randomized concurrent control study Randomized controlled study Toomasian JM and colleagues. ASAIO Trans. 1988 Apr-Jun; 34(2): 140 -7.

National experience with extracorporeal membrane oxygenation for newborn respiratory failure. Data from 715 cases. Does this study convince me to use ECMO in late preterm or term infants with severe respiratory failure? • Yes • No Toomasian JM and colleagues. ASAIO Trans. 1988 Apr-Jun; 34(2): 140 -7.

Follow-up of infants treated with extracorporeal membrane oxygenation for newborn respiratory failure. Follow-up studies were conducted to assess the medical and developmental outcome of 92 infants treated with extracorporeal membrane oxygenation at the University of Michigan. Of 118 near-term (greater than 34 weeks' gestation) infants who received extracorporeal membrane oxygenation, 103 (87%) were surviving and available for follow-up at between 1 and 7 years of age. Medical outcome during year 1 found 31% of the children rehospitalized, primarily with respiratory illness. At last clinic visit 16% of the children exhibited moderate-to-severe neurologic abnormalities, and 8% had moderate-to-severe cognitive delay. Outcome post-extracorporeal membrane oxygenation appears similar to that seen in less ill cohorts of infants treated with more "conventional" therapy. Long-term follow-up of all such infants remains essential. Schumacher RE and colleagues. Pediatrics. 1991 Apr; 87(4): 451 -7.

Follow-up of infants treated with extracorporeal membrane oxygenation for newborn respiratory failure. What type of study is this? • • • Case series without formal controls Case control study Historical control study Non-randomized concurrent control study Randomized controlled study Schumacher RE and colleagues. Pediatrics. 1991 Apr; 87(4): 451 -7.

Follow-up of infants treated with extracorporeal membrane oxygenation for newborn respiratory failure. Does this study convince me to use ECMO in late preterm or term infants with severe respiratory failure? • Yes • No Schumacher RE and colleagues. Pediatrics. 1991 Apr; 87(4): 451 -7.

UK collaborative randomised trial of neonatal extracorporeal membrane oxygenation. A collaborative randomized controlled trial throughout the UK to assess whether a policy of referral for ECMO has a beneficial effect on survival to 1 year without severe disability in comparison with conventional management in newborn infants with severe respiratory failure. UK Collaborative ECMO Trail Group. Lancet 1996; 348(9020): 75– 82

UK collaborative randomised trial of neonatal extracorporeal membrane oxygenation. (Lancet 1996; 348(9020): 75– 82). Mortality ECMO 32% Death or Severe Neurodevelopmental Disability ECMO 33% 0. 55 (95% Cl 0. 39 to 0. 77) No ECMO 59% No ECMO 55. 4% 0. 56 (95% Cl 0. 40 to 0. 78) ECMO No ECMO

UK collaborative randomised trial of neonatal extracorporeal membrane oxygenation. What type of study is this? • • • Case series without formal controls Case control study Historical control study Non-randomized concurrent control study Randomized controlled study UK Collaborative ECMO Trail Group. Lancet 1996; 348(9020): 75– 82

UK collaborative randomised trial of neonatal extracorporeal membrane oxygenation. Does this study convince me to use ECMO in late preterm or term infants with severe respiratory failure? • Yes • No UK Collaborative ECMO Trail Group. Lancet 1996; 348(9020): 75– 82

Nitric oxide for respiratory failure in infants born at or near term. Effect on Death or use of ECMO Finer NN, Barrington KJ (2006) Nitric oxide for respiratory failure in infants born at or near term. Cochrane Database Syst Rev 4: CD 000399

Neonatal extracorporeal membrane oxygenation - a bridging technique. Soll RF. Lancet. 1996 Jul 13; 348(9020): 70 -1. “Although effective, ECMO is clearly a bridging technique, to be supplanted by any number of treatments that would be considered superior. ”

More on the ethics of research in critically ill newborns… Neonatal extracorporeal membrane oxygenation (ECMO): clinical trials and the ethics of evidence. V Mike, A N Krauss, and G S Ross J Med Ethics. 1993 Dec; 19(4): 212– 218. PMID: 8308876

The “best evidence” depends on the puzzle you are trying to solve!

Certain study designs are better suited for answering particular question types. Question Type Definition Appropriate Study Type(s) Therapy Questions about the effectiveness of treatment in order to achieve an outcome (prevention or treatment) Diagnosis Questions about the ability of a test or procedure to differentiate between those with and without a condition or disease RCT > Cohort Study Prognosis Questions about the progression of a disease or outcome of a patient with a particular condition Cohort Study > Case Control Studies > Case Series Etiology/Harm Questions about the harmful effect from an intervention or other procedure / exposure. Cohort Study > Case Control Studies > Case Series Randomized Controlled Trial (RCT)

Upcoming EBM Webinars Session 2: Understanding randomized controlled trials in neonatal-perinatal medicine April 15, 2019 @ 12: 00 pm EDT Session 3: Understanding systematic reviews and meta-analyses in neonatal-perinatal medicine September 9, 2019 @ 12: 00 pm EDT Session 4: Evidence in practice: Feeding preterm infants December 2, 2019 @ 12: 00 pm EST CMEs and Nursing Contact Hours: An email will be sent to all participants with links to post-webinar surveys. Upon completion of the survey, participants will receive their certificates. Surveys will be open for 2 weeks. Questions: colleen. ovelman@uvm. edu