Plan GRADE background two steps quality of evidence

Plan • GRADE background • two steps – quality of evidence – strength of recommendation • importance of values/preferences • an exercise in applying GRADE

Why Grade Recommendations? • strong recommendations – – – strong methods large precise effect few down sides of therapy • weak recommendations – – weak methods imprecise estimate small effect substantial down sides

Which grading system to use? • many available – – – Australian National and MRC Oxford Center for Evidence-based Medicine Scottish Intercollegiate Guidelines (SIGN) US Preventative Services Task Force American professional organizations • AHA/ACC, ACCP, AAP, Endocrine society, etc. . • cause of confusion, dismay

A common international grading system? • GRADE (Grades of recommendation, assessment, development and evaluation) • international methodologists, guideline developers – Australian NMRC, SIGN, USPSTF, WHO, NICE, Oxford CEBM, CDC, CC • ~ 20 meetings over last eight years • (~10 – 50 attendants) • BMJ 2004, six part series 2008

GRADE Uptake • • • • • • Up. To. Date World Health Organization British Medical Journal American Thoracic Society American College of Physicians Cochrane Collaboration BMJ Clinical Evidence KDIGO Polish Institute for EBM Guidelines Finland Society of Vascular Surgery Society of Pediatric Endocrinology European Respiratory Society American Endocrine Society of Critical Care Medicine Surviving sepsis campaign American College of Chest Physicians European Soc of Thoracic Surgeons Allergic Rhinitis in Asthma Guidelines Society of Vascular Surgery Infectious Disease Society of America National Institute for Clinical Excellence (NICE) Agency for Health Care Research and Quality (AHRQ) Swedish National Board of Health and Welfare Canadian Agency for Drugs and Technology in Health Ontario MOH Medical Advisory Secretariat Agencia sanitaria regionale, Bologna, Italia The German Agency for Quality in Medicine Evidence-based Nursing Sudtirol, Alta Adiga, Italy Norwegian Knowledge Centre for the Health Services University of Pennsylvania Health System Center for EB Practice Journal of Infection in Developing Countries – International Japanese Society of Oral and Maxilofacial Radiology

What are we grading? • two components • quality of body of evidence – extent to which confidence in estimate of effect adequate to support decision • high, moderate, low, very low • strength of recommendation • strong and weak

Determinants of quality • • RCTs start high observational studies start low • 5 limitations can lower quality • Bias – detailed design and execution • concealment, blinding, loss to follow-up – publication bias • Imprecision – wide confidence intervals

Consistency of results • consistency of results • if inconsistency, look for explanation – patients, intervention, outcome, methods • judgment of consistency – variation in size of effect – overlap in confidence intervals – statistical significance of heterogeneity – I 2

Relative Risk with 95% CI for Vitamin D Non-vertebral Fractures Favours Vitamin D Favours Control Chapuy et al, (1994) 0. 79 (0. 69, 0. 92) Lips et al, (1996) 1. 10 (0. 87, 1. 39) Dawson-Hughes et al, (1997) 0. 46 (0. 24, 0. 88) Pfeifer et al, (2000) 0. 48 (0. 13, 1. 78) Meyer et al, (2002) 0. 92 (0. 68, 1. 24) Chapuy et al, (2002) 0. 85 (0. 64, 1. 13) Trivedi et al, (2003) 0. 67 (0. 46, 0. 99) Pooled Random Effect Model 0. 82 (0. 69 to 0. 98) p= 0. 05 for heterogeneity, I 2=53% Relative Risk 95% CI

Relative Risk with 95% CI for Vitamin D (Non-Vertebral Fractures, Dose >400) Favours Vitamin D Favours Control Chapuy et al, (1994) 0. 70 (0. 69, 0. 92) Dawson-Hughes et al, (1997) 0. 46 (0. 24, 0. 88) Pfeifer et al, (2000). 48 (0. 13, 1. 78) Chapuy et al, (2002) 0. 85 (0. 64, 1. 13) Trivedi et al, (2003) 0. 67 (0. 46, 0. 99) Pooled Random Effect Mode 0. 75 (0. 63 to 0. 89) p= 0. 26 for heterogeneity, I 2=24% Relative Risk 95% CI

Relative Risk with 95% CI for Vitamin D (Non-Vertebral Fractures, Dose = 400) Favours Vitamin D Favours Control Lips et al (1996) 1. 10 (0. 87, 1. 39) Meyer et al (2002) 0. 92 (0. 68, 1. 24) Pooled Random Effect Mode 1. 03 (0. 86 to 1. 24) p = 0. 35 heterogeneity, I 2=0% Relative Risk 95% CI

Directness of Evidence • differences in patients – age, sex, ethnicity, condition – avian versus regular influenza • interventions – dose, class • outcomes – health-related quality of life, functional capacity, laboratory exercise

Directness interested in A versus B available data A vs C, B vs C Alendronate Risedronate Placebo

What can raise quality? • large magnitude can upgrade one level – very large two levels • common criteria – – – everyone used to do badly almost everyone does well quick action • hip replacement for severe osteoarthritis • dialysis vs no dialysis for prolonging life

Quality assessment criteria

Beta blockers in non-cardiac surgery Summary of Findings Quality Assessment Illustrative comparative risks Quality Relative Effect Relative (95% CI) or WMD No High 0. 71 (0. 57 to 0. 86) 5. 1% Imprecise No Moderate or low 1. 23 (0. 98 – 1. 55) 2. 3% Imprecise No Moderate 2. 21 (1. 37 – 3. 55) 0. 5% Outcome Number of participants (studies) Limitations Consistency Directness Precision Reporting Bias Myocardial infarction 10, 125 (9) No OK OK OK Mortality 10, 205 (7) No Possible ↓ OK Stroke 10, 889 (5) No OK OK 3. 6% (2. 9% to 4. 4%) 2. 8% (2. 2% to 3. 6%) 1. 1 (0. 69% to 1. 8%)

Strength of Recommendation • strong recommendation – benefits clearly outweigh risks/hassle/cost – risk/hassle/cost clearly outweighs benefit • what can downgrade strength? – low quality evidence – close balance between up and downsides

Risk/Benefit tradeoff • aspirin after myocardial infarction – 25% reduction in relative risk – side effects minimal, cost minimal – benefit obviously much greater than risk/cost • warfarin in low risk atrial fibrillation – warfarin reduces stroke vs ASA by 50% – but if risk only 1% per year, ARR 0. 5% – increased bleeds by 1% per year

Strength of Recommendations Aspirin after MI – do it Warfarin rather than ASA in Afib -- probably do it -- probably don’t do it

Significance of strong vs weak • variability in patient preference – strong, almost all same choice (> 90%) – weak, choice varies appreciably • interaction with patient – strong, just inform patient – weak, ensure choice reflects values • use of decision aid – strong, don’t bother – weak, use the aid • quality of care criterion – strong, consider – weak, don’t consider

Value and preference statements • underlying values and preferences always present • sometimes crucial • important to make explicit

Values and preferences Stroke guideline: patients with TIA clopidogrel over aspirin (Grade 2 B). Underlying values and preferences: This recommendation to use clopidogrel over aspirin places a relatively high value on a small absolute risk reduction in stroke rates, and a relatively low value on minimizing drug expenditures.

Values and preferences peripheral vascular disease: aspirin be used instead of clopidogrel (Grade 2 A). Underlying values and preferences: This recommendation places a relatively high value on avoiding large expenditures to achieve small reductions in vascular events.

Flavanoids for Hemorrhoids • venotonic agents – mechanism unclear, increase venous return • popularity – – – 90 venotonics commercialized in France none in Sweden and Norway France 70% of world market • possibilities – French misguided – rest of world missing out

Systematic Review • 14 trials, 1432 patients • key outcome – risk not improving/persistent symptoms – 11 studies, 1002 patients, 375 events – RR 0. 4, 95% CI 0. 29 to 0. 57 • minimal side effects • • is France right? what is the quality of evidence?

What can lower quality? • detailed design and execution – lack of detail re concealment – questionnaires not validated • rate down quality for study limitations? • indirectness – no problem • inconsistency, need to look at the results

Publication bias? • size of studies – 40 to 234 patients, most around 100 • all industry sponsored

Recommendation • for clinician – offer to patient – don’t offer to patient? • strength of recommendation – strong or weak • for the funding body – – – publicly funded not publicly funded strong or weak?

Conclusion • clinicians, policy makers need summaries – quality of evidence – strength of recommendations • explicit rules – transparent, informative • GRADE – simple, transparent, systematic – increasing wide adoption