Evidencebased Medicine EBM Basic Statistics Dr Kam Wong

Evidence-based Medicine (EBM) – Basic Statistics Dr Kam Wong Slide 1 Date: 17 th Jan 2014 www. gpteachingresources. weebly. com

Question 1 A new ‘Super test’ claims to have a superb capability to diagnose disease X. Its sensitivity is 99% and specificity is 90%. Which of the following statement is most correct? (A) The ‘Super test’ is definitely good as it has high sensitivity and specificity. (B) The ‘Super test’ is definitely better because its sensitivity is higher than specificity. Topic presented (C) Specificity is the probability for a test to show a negative results in those without a particular disease. (D) The ‘Super test’ is not good as its specificity is below 95% Slide 2 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Question 2 Patients with bacteraemia (Blood culture) Positive WBC count = or > 15 K? Yes No Negative 80 30 20 70 Topic presented The sensitivity of “WBC” in detecting bacteraemia is: (A) 70 / (70+30) (B) 70 / (20+70) (C) 80 / (80+30) (D) 80 / (80+20) Slide 3 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Question 3 Renal artery stenosis Positive “New Scan” Negative 60 25 40 75 The specificity of the “New Scan” is: (A) 75 / (75+25) (B) 75 / (75+40) (C) 60 / (60+40) (D) 60 / (60+25) Slide 4 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com Topic presented

Question 4 A study randomised diabetic patients into treatment X (placebo) and treatment Y. It reported that leg ulcer occurred in 22% of the patients in the placebo group (treatment X) and 2% of the patients given treatment Y. What is the ‘number needed to treat’ to prevent one leg ulcer? (A) 20 (B) 10 (C) 5 (D) 15 Slide 5 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com Topic presented

Question 5 A researcher reported that there was significant difference of waist circumference between male and female. The male’s waist circumference is 3 cm larger than the female’s and the p-value was 0. 12. Which of the following statements is most correct? (A) The study was not significant because the absolute difference in waist circumference was only 3 cm. (B) The study was significant because the p-value is greater than 0. 05 Topic presented (C) P value is the probability that an observed difference occurred by chance. (D) The study was not significant because the p-value must be smaller than 0. 001 Slide 6 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Sensitivity Specificity Positive Predictive Value Negative Predictive Value Topic presented Slide 7 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Definition Sensitivity: The probability of a positive test in those with the disease Specificity: The probability of a negative test in those without the disease Topic presented Slide 8 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Sensitivity Patients with the condition Yes No Positive a b Negative c d Test Result Sensitivity = a / [a + c] Slide 9 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com Topic presented

Specificity Patients with the condition Yes No Positive a b Negative c d Test Result Specificity = d / [b + d] Slide 10 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com Topic presented

Positive Predictive Value (PPV) PPV is the probability that a positive test is a true positive. Patients with the condition Yes No Positive a b Negative c d Test Result Topic presented Positive predictive value= a / [a + b] Slide 11 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Negative Predictive Value (NPV) NPV is the probability that a negative test is a true negative. Patients with the condition Yes No Positive a b Negative c d Test Result Topic presented Negative predictive value= d / [c + d] Slide 12 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Example 1: Sensitivity Patients with bacteraemia (blood culture) Positive Negative WBC count = or > 15 K? Yes No 55 275 5 177 Sensitivity = 55/ [55+5] = 0. 91 Topic presented Reference: Bass JW, et al. Antimicrobial treatment of occult bacteremia: a multicenter cooperative study. Pediatric Infect Dis J 1993; 12: 466 – 73. Slide 13 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Example 1: Specificity Patients with bacteraemia (blood culture) Positive Negative WBC count = or > 15 K? Yes No 55 275 5 177 Specificity = 177/ [275+177] = 0. 39 Topic presented Reference: Bass JW, et al. Antimicrobial treatment of occult bacteremia: a multicenter cooperative study. Pediatric Infect Dis J 1993; 12: 466 – 73. Slide 14 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Example 1 (continued) Positive Predictive Value (PPV) PPV is the probability that a positive test is a true positive. Patients with bacteraemia (Blood culture) Positive WBC count = or > 15 K Negative Yes 55 275 No 5 177 Topic presented Positive Predictive Value: 55 / [55 + 275] = 0. 17 Slide 15 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Example 1 (continued) Negative Predictive Value (NPV) NPV is the probability that a negative test is a true negative. Patients with bacteraemia (Blood culture) Positive WBC count = or > 15 K Negative Yes 55 275 No 5 177 Topic presented Negative Predictive Value: 177 / [5 + 177] = 0. 97 Slide 16 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Example 2 • Consider HIV screening in the general population • Imagine Western blot has a sensitivity of 99. 99% but a specificity of 95% • Say in a city of 100, 000 has an HIV incidence of 1/10, 000 • What will be the true & false positives as a result of screening the city? Topic presented Slide 17 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Example 2 (continue) • With that HIV incidence of 1/10, 000 we may expect 10 people have HIV in that city of 100, 000 people. Number of estimated HIV patient= 100 000 x 1/10 000 = 10. HIV Yes Positive HIV Screening Negative Total No TP FP FN TN 10 99, 990 Slide 18 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com TP= True Positive FP= False Positive TN= True Negative FN= False Negative Topic presented

Example 2 (continue) With a sensitivity of 99. 99%, the True Positive= 99. 99/100 x 10 =9. 999 which is rounded to 10. HIV Yes Positive HIV Screening Negative Total No 10 FP 0 TN 10 99, 990 Slide 19 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com TP= True Positive FP= False Positive TN= True Negative FN= False Negative Topic presented

Example 2 (continue) With a specificity of 95%, the True Negative= 95/100 * (100 000 -10) = 95/100 * 99990 = 94990 HIV Yes Positive HIV Screening Negative Total No 10 5, 000 0 94, 990 10 99, 990 Slide 20 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com TP= True Positive FP= False Positive TN= True Negative FN= False Negative Topic presented

Example 2 (continue) If the specificity of Western Blot improves to 99%, the True Negative will be = 99/100 * 99990 = 98990; and the False Positive = 99990 - 98990= 1000 HIV Yes Positive HIV Screening Negative Total No 10 1, 000 0 98, 990 10 99, 990 Slide 21 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com TP= True Positive FP= False Positive TN= True Negative FN= False Negative Topic presented

Example 2(continued) In the above example, the false positive reduces from 5000 to 1000 by improving the specificity of the test! Higher specificity will decrease the number of false positive especially in a disease of low incidence. Topic presented Slide 22 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Revisit Question 1 A new ‘Super test’ claims to have a superb capability to diagnose disease X. Its sensitivity is 99% and specificity is 90%. Which of the following statement is most correct? (A) The ‘Super test’ is definitely good as it has high sensitivity and specificity. (B) The ‘Super test’ is definitely better because its sensitivity is higher than specificity. Topic presented (C) Specificity is the probability for a test to show a negative results in those without a particular disease. (D) The ‘Super test’ is not good as its specificity is below 95% Slide 23 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Revisit Question 2 Patients with bacteraemia (Blood culture) Positive Negative WBC count = or > 15 K? Yes 80 30 20 70 No Topic presented The sensitivity of “WBC” in detecting bacteraemia is: (A) 70 / (70+30) (B) 70 / (20+70) (C) 80 / (80+30) (D) 80 / (80+20) Slide 24 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Revisit Question 3 Renal artery stenosis Positive Negative Positive “New Scan” 60 25 40 75 Negative Topic The specificity of the “New Scan” is: (A) 75 / (75+25) (B) 75 / (75+40) (C) 60 / (60+40) (D) 60 / (60+25) Slide 25 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com presented

Relative Risk Absolute Risk Reduction Number needed to treat Topic presented Slide 26 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Relative Risk (RR) RR: ratio of the risk of the outcome in the treatment group compared with the control group. RR is usually reported in a prospective study. The equivocal value (cut-off value) for RR is “ 1”. Example 3: The Diabetes Control and Complication Trial investigated the effect of intensive diabetes therapy on the development and progression of neuropathy. The study reported that neuropathy occurred in 9. 6% of the patients randomized to control-group and 2. 8% of the patients randomized to intensive therapy. Topic presented RR= [0. 028 / 0. 096] = 0. 29 The RR is less than 1 which means that treatment reduces the risk of neuropathy. Slide 27 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Example 3 (continued) Absolute Risk Reduction (ARR): The absolute difference in the rates between the treatment and control groups. Number needed to treat (NNT): The number of people who need to be treated to achieve one good outcome or prevent one adverse event. ARR= 0. 096 – 0. 028 = 0. 068 NNT= 1/ 0. 068 = 15 Topic presented 15 diabetic patients need to be treated by the ‘intensive therapy’ to prevent one neuropathy. Reference: The effect of intensive diabetes therapy on the development and progression of neuropathy. Annals of Internal Medicine 1995; 122: 561 -8 Slide 28 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Revisit Question 4 A study randomised diabetic patients into treatment X (placebo) and treatment Y. It reported that leg ulcer occurred in 22% of the patients in the placebo group (treatment X) and 2% of the patients given treatment Y. What is the ‘number needed to treat’ to prevent one leg ulcer? (A) 20 (B) 10 (C) 5 (D) 15 Slide 29 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com Topic presented

Example 4 The absolute risk for DVT in women aged 50 -60 yr is 1 in 10, 000. Studies show Oral Combined HRT for 5 years has a relative risk (RR) of 2. In a cohort of 10, 000 women aged 50 -60 yr, how many additional cases would be found in the HRT group? Which of the following is most correct? (A) (B) (C) (D) (E) 1 2 4 6 8 Slide 30 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com Topic presented

Example 4 The absolute risk for DVT in women aged 50 -60 yr is 1 in 10, 000. Studies show Oral Combined HRT for 5 years has a relative risk (RR) of 2. => hrt/ [1/10000] =2 => hrt = 2 x [1/10000] Topic presented ==> hrt = 2/ 10000 Hence, in a cohort of 10, 000 women aged 50 -60 yr who take oral combined HRT for 5 years, 2 may have the risk of DVT. Additional case: 2 -1 = 1. Slide 31 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Example 4 The absolute risk for DVT in women aged 50 -60 yr is 1 in 10, 000 [or 0. 1 in 1000] Studies show Oral Combined HRT for 5 years has a relative risk (RR) of 2. In a cohort of 10, 000 women aged 50 -60 yr who take oral combined HRT for 5 years (RR=2), additional case of DVT is 2 - 1= 1 So The most correct answer: (A) 1 (B) 2 (C) 4 (D) 6 Topic presented (E) 8 Slide 32 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

P-value Confidence interval Topic presented Slide 33 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

An analogy for confidence interval (CI): • Passing mark for an examination is set at 65 marks • Student A scores 85 marks (95% CI 75 marks – 95 marks) 65 75 85 95 • Student B scores 78 marks (95%CI 60 marks – 96 marks) Topic presented 60 65 78 96 • Student A performs better than student B • When we are making comparison, look at the confidence intervals. Slide 34 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

P value, confidence interval Example 5: In a case control study to investigate the potential association between lung cancer and smoking Group 1: people with lung cancer Group 2: people without lung cancer The smoking history of both groups are compared. The Null Hypothesis: there is no significant difference between the Topic presented groups and their smoking history The Alternative Hypothesis: there is significant difference between the groups and their smoking history Slide 35 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Example 5 (continued) The odds ratio O. R. = [Smoking/Non-smoking]cancer/ [Smoking/Non-smoking]noncancer = [12/5]/ [4/9] = 5. 4 Topic presented Odds ratio is usually reported in a retrospective study or case-control study. The equivocal value(cut-off value) for odds ratio is “ 1”. Slide 36 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Example 5 (continued) The odds ratio =5. 4, p=0. 03, 95% confidence interval 1. 2 -- 26 What is your interpretation of the above results? (A) People who have lung cancer are 5. 4 times more likely to have been smokers Topic presented (B) The above finding is significant as the confidence interval does not include the value of 1. 0 (C) P value is the probability that an observed difference occurred by chance. (D) All of the above Slide 37 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Confidence Interval: Another example If the odds ratio equal 1: equivocal i. e. there is no significant association between smoking and lung cancer. • Study A: Odds Ratio for smoker to have lung cancer is 6 and the 95% confidence interval for the odds ratio is between 3 and 10 • Study B: Odds Ratio for smoker to have lung presented cancer is 2 Topic and the 95% confidence interval for the odds ratio is between 0. 5 and 3. 5 Slide 38 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Confidence Interval: Another example • Study A Odds Ratio: 6 (95% CI 3 – 10) 1 3 6 10 • Study B Odds Ratio: 2 (95%CI 0. 5 – 3. 5) Topic presented 0. 5 1 2 3. 5 • Results from Study A is more significant • Hence, when we read a research report, we should not just read the single value of odds ratio but we should read the confidence interval for the odds ratio too. Slide 39 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Revisit Question 5 A researcher reported that there was significant difference of waist circumference between male and female. The male’s waist circumference is 3 cm larger than the female’s and the p-value was 0. 12. Which of the following statements is most correct? (A) The study was not significant because the absolute difference in waist circumference was only 3 cm. (B) The study was significant because the p-value is greater than 0. 05 Topic presented (C) P value is the probability that an observed difference occurred by chance. (D) The study was not significant because the p-value must be smaller than 0. 001 Slide 40 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

How did you go in the Quiz? • Questions? Topic presented Slide 41 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

An example of exam question A new screening blood test for Beeper Phobia Disorder (BPD) was developed which detects the presence of the Beeper Phobia gene. The surgical residents at Bedpan University Hospital were screened for BPD with this new test to determine how well it identifies early cases of this dreaded disease which, if left untreated, can result in permanent mental impairment. The goal is to be able treat BPD while the residents are still asymptomatic before they are in danger of mutilating their beepers. The results were obtained and are displayed in next slide. Topic presented Slide 42 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

An example of exam question (continue) (+) BPD (-) BPD Total (+) Test 90 30 120 (-) Test 10 70 80 Total 100 200 Topic presented What is the sensitivity of the screening test for BPD? A. 88% B. 70% C. 50% D. 90% E. 75% Slide 43 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

EBM in OSCE • You may have an OSCE station in which you are given a research paper for critical appraisal. EBM is usually integrated in the study. • Read the abstract (objectives/ hypothesis, methods, results, conclusion) • Is the objective/hypothesis clinically relevant / valid? • Methods: • Randomised controlled trial (RCT)? Retrospective? Prospective? • . Sampling method (randomised, systematic, convenience. . . )? Sample size? • Representative samples? Bias? • Double-blinded? Single-blinded? • Pilot run? • Ethics approval? Will the researchers publish the paper if the results are not supportive of their hypothesis? • Drop out rate? Reasons for drop out? Adverse events? Topic presented Slide 44 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

EBM in OSCE - continue Continue. . • Results, Analysis & Discussion: • Apply your knowledge of ‘p value’, ‘confidence interval’, ‘NNT’. . • Are the results supportive of their hypothesis? Applicable to your patients? (compare the profile of the sample/population with the profile of your patient) • The approach/finding of the study: educational to you +/- your patients? • Sponsor for the study: conflict of interest? Topic presented Reference: RACGP Curriculum “Critical Thinking and Research”, Dr Andrew Moreton’s Mock OSCE DVD. Slide 45 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com

Thank You Topic presented Slide 46 17 th Jan 14 Dr Kam Wong www. gpteachingresources. weebly. com