How to Teach Statistics in EBM Rafael Perera

How to Teach Statistics in EBM Rafael Perera

Basic teaching advice • • Know your audience! Create a knowledge gap Give a map of the main concepts Decide which ones to focus on Use plenty of examples Let them do the work/thinking

Main Concepts Bias and Measurement error P values and Confidence Intervals Which Statistical tests are needed and when Correlation and Association Models / Regression and alternatives for Adjustment • Survival Analysis • Meta Analysis • Statistics for Diagnostic Studies • • •

There is a time and place…

Fundamental Equation of Error Use large numbers Use good study design Researcher • Measure = Truth + Bias + Random Error Confidence Intervals and P-values Critically Appraise Design Reader

Bias versus Random error Bias high Random error low high true result

Bias and Measurement error Groups of 3 -4 people 1 – subject 2 – measurers Measurers – measure (twice) and record the head size of the subject. Keep measurements hidden.

Bias and Measurement error Intra-Observer variability Measurement error • Same answer • Varied by < 0. 5 cm • Varied by < 1 cm • Varied by < 2 cm • Varied by >2 cm

Bias and Measurement error Inter-Observer variability Measurement error • Same answer • Varied by < 0. 5 cm • Varied by < 1 cm • Varied by < 2 cm • Varied by >2 cm

Bias and Measurement error Bias Included ears? Included nose? Which part of the head? Other?

Does it matter? In paediatric practice following meningitis, a head circumference that increases by 7 mm in a day will result in urgent head imaging In obstetrics measurements of the fundal height can vary by up to 5 cm (the difference between having a baby delivered early due to IUGR or not when opposite occur) The question is can you reproduce the test in your setting and will it perform as well in your setting

Measuring Random error Most things don’t work!

Two methods of assessing the role of random error • P-values • Confidence Intervals • (Hypothesis Testing) – use statistical test to examine the ‘null’ hypothesis – if p<0. 05 then result is statistically significant (Estimation) – estimates the range of values that is likely to include the true value Relationship between p-values and confidence intervals If the ‘no effect’ value falls outside the CI then the result is statistically significant

The Steps in Testing a Hypothesis State the null hypothesis H 0 Choose the test statistic summarizes Based on H 0 calculate the probability of the data Interpret the getting the P-value that value of the test statistic

Some Statistical tests • Comparing groups – T-tests (1 or 2 groups, normally distributed) – Chi-squared (2 or more groups, categorical or binary data) – Mann-Whitney U (2 groups, non-normal data) – Log-rank test (2 groups, survival data) – ANOVA (multiple groups, normally distributed) –… • Tips: – Understand what the hypothesis being tested is – Use the p-value to assess the level of evidence against it – (Experienced) Assess if the test was adequate for the question and data analysed

Hand outs 1. 2. 3. 4. 5. Incidence/ Prevalence and CI Survival analysis Regression models / Adjustment Linear association / Correlation Confounding / Odds Ratios / Logistic Regression 6. Diagnostic Tests 7. Meta-analysis

Reading confidence intervals

Clinically significant Vitamin X shortens a 5 day cold Would you take it twice per day if it shortened the cold by:

Clinically significant Vitamin X shortens a 5 day cold Would you take it twice per day if it shortened the cold by: 50%

Clinically significant Vitamin X shortens a 5 day cold Would you take it twice per day if it shortened the cold by: 50% 20%

Clinically significant Vitamin X shortens a 5 day cold Would you take it twice per day if it shortened the cold by: 50% 20% 10%

Clinically significant Vitamin X shortens a 5 day cold Would you take it twice per day if it shortened the cold by: 50% 20% 10% 5%

Clinically significant Vitamin X shortens a 5 day cold Would you take it twice per day if it shortened the cold by: 50% 20% 10% 5% 1%

Which are clinically significant? 20 (a) No difference 0 10 Minimum clinical Important difference (b) (c) (d)

Thank you

EXTRAS

Different types of measurements use different types of statistics • Dichotomous: �� – Male, female OR infected, non-infected • Categorical: ��� – Red, green, blue OR • Ordinal: � � � – Nil, +, ++ of glucose • Interval: – temperature STATISTICS Proportion, Risk Mode, Proportions Mode, Median? Mean, Median

Flowchart of Statistical Tests for Hypothesis Testing Between X one observed 2 test for goodness of fit variable and a theoretical distribution Between distributions X Independence between two or more 2 test for independence variables Mc. Nemar’s test for related groups Parametric samples Two samples Hypothesis testing and a ssessing d ifferences T test for independent T test difference for Between means for related samples continuous data Non parametric Rank sum test for independent samples Sign test for related samples Parametric ANOVA > t wo samples Kruskal Wallis Non parametric One sample vs. H 0 Z score Between proportions for categorical data Z score equal proportions

Flowchart of Statistical Tests for Hypothesis Testing Between one observed variable and a theoretical distribution Between distributions Independence between two or more variables c 2 test for goodness of fit c 2 test for independence Mc. Nemar’s test for related groups

Flowchart of Statistical Tests for Hypothesis Testing Parametric t-test independent samples t-test difference for related samples Two samples Rank sum test for independent samples Between means for continuous data Non Parametric Sign test for related samples Parametric ANOVA Non Parametric Kruskal – Wallis > two samples

Flowchart of Statistical Tests for Hypothesis Testing One sample vs. H 0 Z-score Between proportions for categorical data Two samples Z-score equal proportions Summarising proportions One sample: Risk, Odds Two samples: Relative risk, Odds ratios, Risk differences