Introduction to Study Designs for EBM Michelle Howard

Introduction to Study Designs for EBM Michelle Howard, MSc, Ph. D (Candidate), Assistant Professor

Levels of Evidence 1: Systematic review of RCT or individual RCT 2: Cohort study, SR of cohort studies 3: Case-control study, SR of case-control study 4: Case-series 5: Expert opinion, physiology, bench research

Does HRT Prevent or Cause Heart Disease? RCT: subjects are assigned to intervention, do not get to choose Exposed to HRT % heart disease Not Exposed % no heart disease Cohort: Subjects choose intervention, exposure measured when exposed, outcomes measured PROSECTIVELY Cross Sectional: disease measured and exposure Case-Control: Subjects with disease identified (case), measured at same time RETROSPECTIVELY matched to those without disease (control), exposures measured RETROSPECTIVELY

Main Study Types: Experimental, Observational, Qualitative l The question dictates the type of study, eg: –EXPERIMENTAL Treatment effectiveness = RCT l Screening effectiveness = RCT l –OBSERVATIONAL (Case Series, Cross Sectional, Cohort or case-control ) Harmful exposure (e. g. smoking and cancer) l Etiology (e. g. gene and CF) l Prognosis (e. g. microalbuminuria and coronary event) l

Not all questions can be answered with RCT l l l Harm – “Antidepressant treatment and the risk of fatal and non-fatal self harm in first episode depression: nested case-control study” Etiology – “Dietary intakes of fruit, vegetables, and fiber, and risk of colorectal cancer in a prospective cohort of women (United States)” Diagnosis 1. Accuracy of physical examination in the diagnosis of hypothyroidism: a cross-sectional, double-blind study

For ALL studies ask yourself “Are the results VALID- or is there another explanation? ” VALID = TRUE

Case-Control Study on HARM Antidepressant treatment and the risk of fatal and non-fatal self harm in first episode depression: nested case-control study #1 Cases are depressed patients in emergency room who have attempted self-harm. They selfreport their anti-depressant use on a survey Controls are depressed patients from family practices who have not attempted self-harm. A chart audit is done on their anti-depressant prescribing #2 Cases identified as non-fatal self harm by using relevant medical terms in the EMR and review of the patient's free text notes. Controls were a random sample patients with no self-harm in their records, matched to cases on sex, year of birth within one year, and duration of cohort membership We derived the duration of prescriptions from the quantity of drug prescribed and the daily dose from the EMR

Prospective Cohort Study on Etiology Dietary intakes of fruit, vegetables, and fiber, and risk of colorectal cancer in a prospective cohort of women (United States) Diet assessed yearly by self-reported questionnaire (e. g. “never” to “more than 6 per day”) Exposure: Low, Medium, High Fiber 10 Years Outcome: Colorectal cancer determined by self-report questionnaire-cases followed up with pathology reports Results: No association between intakes of fruit, vegetables, and fiber, and colorectal cancer risk

How “valid” are the results? What if women with higher fiber diets are more health conscious and undergo more cancer screening tests compared to women with lower fiber diets? What if high fiber diets are more accurately reported than low fiber diets?

Diagnostic study validity l l l Did all patients, regardless of screening result have the “gold standard” test? Gold standard blinded? Test and gold standard reproducible?

Diagnostic Accuracy Study Fecal DNA versus fecal occult blood for colorectal-cancer screening in an average-risk population. All subjects first provided a fecal sample for DNA testing and then completed three Hemoccult II cards before undergoing screening colonoscopy. All tests were conducted in a blinded fashion. Stool samples were analyzed for DNA abnormalities without knowledge of Hemoccult II or colonoscopy results; colonoscopy was performed without knowledge of the results of fecal DNA testing. Since Hemoccult II testing was conducted at the study sites, the results were potentially available to the colonoscopists. A clinical research organization (Parexel) received the results of Hemoccult II tests and colonoscopy directly from the clinical sites and received the results of fecal DNA analyses from the clinical laboratory (Exact Sciences).

Subjects were given three Hemoccult II cards and instructions regarding dietary and medication modifications to comply with current recommendations Subjects could be evaluated only if the specimen for fecal DNA analysis was adequate, all six Hemoccult II panels had been completed, and colonoscopy was adequate. The colonoscopist documented the extent of the colon that was visualized and the quality of the bowel preparation. Adequate colonoscopy required visualization of the cecum and a minimum of 90 percent of the mucosa. The size and location of any lesions were recorded. Biopsy and surgical resection specimens were examined histopathologically at each site; no centralized pathological review was performed All samples analyzed for fecal DNA were processed in a single laboratory. Laboratory handling of all samples was fully automated.

Quality Criteria for RCT l l Randomization Allocation concealment Double-blinding Drop-outs

Randomised controlled trial of nurse practitioner versus general practitioner care for patients requesting "same day" consultations in primary care BMJ 2000; 320: 1043 -1048 ( 15 April ) In practices using randomisation by day, all patients consulting on a particular day saw the same type of practitioner. Practices were supplied with a calendar of their study period with the days allocated at random as nurse practitioner or general practitioner days by block randomisation. Truly Random? Blinded? Concealed?

Outcome measurement blinded? After the consultation, patients completed the consultation satisfaction questionnaire 8 and answered yes or no to questions on the information provided by the clinician during the consultation (the cause of the illness, what the patient could do to relieve symptoms, likely duration, how to reduce chances of recurrence, and what the patient should do if the problem didn't improve) Four weeks after the initial consultations, patients' medical records were checked for reattendance or hospital admission for the same problem. The results were recorded on an `audit sheet'.

Effects of remote, retroactive intercessory prayer on outcomes in patients with bloodstream infection: randomised controlled trial BMJ 2001; 323: 1450 -1451 ( 22 -29 December ) A random number generator was used to randomise the patients into two groups. A list of the first names of the patients in the intervention group was given to a person who said a prayer for the well being and full recovery of the group as a whole. There was no sham intervention. Chart audit of days in hospital was done by auditors that did not know the patients’ groups. Truly Random? Blinded? Concealed?

Threats to Validity: Bias Definition: “Any systematic error in the design, conduct or analysis of a study that results in a mistaken estimate of an exposure’s effect on the risk of disease” l Bias is always a problem with how data is measured or analyzed l It can over- or under-estimate an exposure’s effect on disease l Examples are: selection bias, misclassification bias, recall bias, “wish bias”.

Threats to Validity: Bias Example: In a case-control study of the risk of high cardiac output exercise on spontaneous abortion, women with abortion (cases) and those with healthy term pregnancies (controls) were asked about exercise history. l l l Women with abortion will seek to remember many more details than those without The problem is SYSTEMATIC, ie. It sorts out by disease group rather than randomly The problem is with measurement, not with a characteristic of the subject, so it is bias and not confounding.

Threats to Validity: Confounding Definition: “An observed association between an exposure and disease that is more likely caused by a third factor that is associated with both the exposure and disease” l Confounding is not a problem with measurement (as with bias) but is a problem with what factors are being considered l It can lead to false conclusions about whether the measured exposure is actually causing disease, or a third factor is causing disease.

Threats to Validity: Confounding Example: In Alabama, it was noted that with increases in ice cream sales came increases in death. Does ice cream cause death? Ice Cream Death HEAT

The End