CASE SERIES Critical appraisal Akbar Soltani Assistant Professor

CASE SERIES: Critical appraisal Akbar Soltani Assistant Professor of Medicine and Endocrinology Evidence Based Medicine Working Team (EBMWT) Tehran University of Medical Sciences (TUMS) Shariati Hospital EBMWT. EMRC A. SOLTANI TUMS

Checklist for the critical appraisal of case series. n n Reliable and validated search strategies for primary study designs other that RCTs and CCTs are not yet available, hence the impact of retrieval and publication bias in reviews of case series is completely unknown. It also possible that unbiased estimates of effect from case series may be contained within other study designs (e. g. each group within a comparative study may constitute a case series in its own right) and this should always be considered EBMWT. EMRC A. SOLTANI TUMS

Examples Gastrinoma and MEN n Tg and scan negative thyroid cancer n Hyperplasia and hyperparathyroidism n EBMWT. EMRC A. SOLTANI TUMS

1. Was the study conducted prospectively? n Can be difficult to assess, but if the outcomes are clearly measured before and after the intervention, and criteria are clearly defined for the measurement of outcomes a priori it is highly probable that this was the case. EBMWT. EMRC A. SOLTANI TUMS

Information required n n A description within the methods section describing the timing of the relevant events with respect to the initiation of the study, i. e. were cases selected for inclusion in the study before the results of the outcome of interest were known by the investigators. EX: Standard surrogates? (key outcomes) Tg, Gastrin… EBMWT. EMRC A. SOLTANI TUMS

Assessment Criteria n n Ideal - Study states that it is conducted prospectively Acceptable - Evidence that all key outcomes were measured before and after the intervention using clear criteria defined a priori. Unacceptable - Study states that is was conducted retrospectively or it clearly does not measure key outcomes before and after the intervention OR no information. Ex: Shrinkage of tumor Vs disappearance of Tg EBMWT. EMRC A. SOLTANI TUMS

2. Was the method of selection of cases identified and appropriate? n Again this is not always clear but if the case series has been selected from a wider population of cases treated it is important to assess whether this has been done in an unbiased way. EBMWT. EMRC A. SOLTANI TUMS

Information required n Detail within the methods or results section on the numbers treated and the numbers included in the case series and, if they are different, how cases were selected for inclusion and whether they were representative of the wider population. EBMWT. EMRC A. SOLTANI TUMS

Assessment Criteria n n n Ideal - Study states that a consecutive series of cases was included in the study. Acceptable - Evidence that cases were selected for inclusion in an unbiased way or evidence that the characteristics of the included cases were not significantly different form those of the treated population. Unacceptable - Clear evidence from the numbers that the included cases were a sample of those treated with no detail on the selection process or evidence that they were significantly different from the total population treated. OR no information. EBMWT. EMRC A. SOLTANI TUMS

3. Was the duration and completeness of follow-up reported and was it adequate? n n n Detail on losses to follow-up and deaths will usually be available. A particular problem in case series is that frequently only small subgroups of cases have reached given follow-up points which is potentially problematic if not handled carefully. EX: for thyroid cancer 5 years disease free may be useless EBMWT. EMRC A. SOLTANI TUMS

Information required n Numbers and characteristics of losses to follow-up and deaths. EBMWT. EMRC A. SOLTANI TUMS

Assessment Criteria n n n n n Ideal - Follow-up data on 80 - 100% of cases. Acceptable - Adequate management of deaths and losses to follow-up such as detailing their characteristics, performing sensitivity analyses and/or including them in the final analysis OR If losses to follow-up are cases who have not yet reached a given follow-up point, are those for whom data is available treated as a cohort, with results presented for the cohort only before and after the intervention? ? Unacceptable - Losses to follow up of over 20 - 25% particularly if they are unaccounted for. OR Follow-up data for a subgroup of patients followed up to a given point using baseline data for the whole series as a comparator OR no information. EBMWT. EMRC A. SOLTANI TUMS

Problems with Case Series n n Selection bias will always be present in case series a) Lack of a comparison group can make it appear as if there is an association between an intervention and an outcome when, in fact, there is not. b) There are usually no criteria for patient selection. c) Frequently cases are not consecutively selected. d) Clinicians usually report on those patients with the best outcomes. EBMWT. EMRC A. SOLTANI TUMS

Problems with Case Series n Observation bias will always be present in case series. a) Because there is no blinding, clinician beliefs in or hopes for an intervention can affect outcomes resulting in performance bias. b) Assessment bias often occurs because lack of consecutively selected patients can result in selective reporting favoring the intervention. c) Patients frequently improve after a medical visit, but outcomes might otherwise be the same whether a treatment is administered or not EBMWT. EMRC A. SOLTANI TUMS

. Problems with Case Series n n Observation bias will always be present in case series Therefore, without a comparison group, almost any intervention will appear to be beneficial but improvement may be due to i) The self-limited nature of the condition: Example: alternative medicine- subjective complain ii) Placebo effect Example: nut tablets and again alternative medicine iii) Regression-to-the-mean : again what EBMWT. EMRC A. SOLTANI TUMS

n n n Observation bias will always be present in case series Therefore, without a comparison group, almost any intervention will appear to be beneficial Example: every things TV serials and programs Vs wall Full time professors Vs holiday Every new interventions in our country! EBMWT. EMRC A. SOLTANI TUMS

There are reporting problems resulting from case series n a) Due to publication bias, negative results are almost never reported (the reporting of which would still present its own problems since a negative-finding case series would be highly prone to bias for the above reasons). n b) Authors of case series frequently compare their results to There is always the possibility that the authors will select case series for comparison that show their results in the best light. those of other case series. EBMWT. EMRC A. SOLTANI TUMS

Problems with Case Series n n n Case series can be useful in describing a clinical condition or to generate ideas for study. However, because of the above mentioned biases, case series can almost never be relied upon to draw conclusions between interventions and outcomes. Rarely, conditions where morbidity or mortality is nearly 100 percent and, with the intervention, is decreased dramatically, case series may be sufficient to draw conclusions about the effect of the intervention on outcomes – but it has to be emphasized that this is extremely rare. EBMWT. EMRC A. SOLTANI TUMS

Case-Series n Weakest research design n Can not be used to answer questions of n Intervention effectiveness n Usefulness of diagnostic tests n Causal factors of disease, recovery n Factors associated with outcome n Incidence n Prevalence n Design not capable of providing answers to above questions (but many published case-series try!) n Identify complications (but not rate or prevalence) n Identify questions that should be subjected to research n Hypothesis generating, but not hypothesis testing EBMWT. EMRC A. SOLTANI TUMS

Case Studies, cont. n n Weaker on both internal & external validity than the other designs, but Simply describing what you see or have Most useful for disease with a specific etiologic agent May describe patients at one point in time (like cross-sectional study) or over a period of time (like cohort study) EBMWT. EMRC A. SOLTANI TUMS

n SUMMARY EBMWT. EMRC A. SOLTANI TUMS

Case-Series n No sampling n Series of clinical cases( fracture in Tehran hospitals) n n n Convenience sample No concurrent control (Usually inadequate information about persons not developing the disease) No exposure n Description of cases (usually clinical cases) EBMWT. EMRC A. SOLTANI TUMS

Case-Series n n Usually retrospective, but may be prospective Measurement of disease, outcome n n generally “usual clinical practice” (Cases may not be representative; persons with disease are survivors of unknown population) Statistical analysis n Should not be performed (but often is!) EBMWT. EMRC A. SOLTANI TUMS

Case Reports/Series n Evaluation n Advantages n n n Disadvantages n n n EBMWT. EMRC Inexpensive Fast Lead to Hypotheses Personal Not statistically valid No comparison group Personal A. SOLTANI TUMS

Uses for case series designs n n Capture rare or clinically interesting events Identify, describe complications Alert health community to possible risks of procedures, risks of activities, links between factors and outcome n Intervention based on prognosis or effect size? n EBMWT. EMRC A. SOLTANI TUMS

THANK YOU EBMWT. EMRC A. SOLTANI TUMS