OUTLINE OF PRESENTATION 1 Systematicbiaserror in epidemiological studies

OUTLINE OF PRESENTATION 1. Systematic(bias)error in epidemiological studies 2. Validity and testing for validity 3. Reliability and testing for reliability 4. Group task for today

1. Bias is a systematic error in the design, conduct, or analysis of a study that results in errors when calculating measures of association between risk factors and outcomes 2. It is thus always one-sided, either underestimating or over-estimating risk 3. It is not usually a subjective error, and is often unavoidable, but must be recognised as a limitation of the study to allow for correct interpretation

Types of bias in epidemiological studies Selection bias: In selection bias study participants are selected in one locality of convenience and are therefore not representative of the target population e. g. Selection of a sample for high prevalence of alcoholic foetal syndrome in western cape is not representative of prevalence of alcoholic foetal syndrome in SA. Information bias /Recall bias: Recall bias result because those affected by a disease often recall their exposures better than those who are unaffected e. g. Those who were exposed to polio virus will remember better than those who were unexposed Non-response bias: leads to a lack of information on non-responders. Thus if the non-response rate is high, the study may be severely biased. Sometimes called volunteer bias. 4. Misclassification bias:

Validity of a test The validity of a test is defined as its ability to distinguish between those who have the disease(Sensitivity )and those who do not have the disease(Specificity). Factors affecting validity: 1. Incorrect calibration and or uncalibration of the measuring device e. g. Sphygmomanometer(internal validity) 2. Unstructured interviews or questionnaires in a study (internal validity 3. Unrepresentativeness of the sample from the population(external validity. Components of validity Validity of test has therefore two components namely sensitivity and specificity. Sensitivity of a test is defined as the ability of a test to identify correctly those who have the disease Specificity of a test is defined as the ability of a test to identify correctly those who do not have the disease.

True characteristic in the population Test Result Have the disease Positive True positive(TP) Have the disease and test positive Negative Do not have the disease False Positive(FP) Do not have the disease but test positive False negative(FN) Have the but True Negative(TN) Test negative and test negative Sensitivity = TP TP +FN Do not have the Specificity = TN TN+FP

Types of validity tests Gold standard test Screening test Gold standard test refers to a diagnostic test or benchmark that is the best available under reasonable conditions. It is referred to as the most accurate test possible without restrictions A hypothetical ideal of gold standard test has a 1. sensitivity of 100% with respect to the presence of the disease (it identifies all individuals who have the disease; it does not have any false-negative results) Sensitivity = a a +c OR TP TP +FN

Also a hypothetical ideal of gold standard test has a 2. specificity of 100% with respect to the absence of disease (it identifies all individuals who do not have the disease. it does not have any false-positive results). Specificity = d d +b OR TN TP +FP

Example: Suppose we have a hypothetical population of 1000 people, the gold standard will yield a sensitivity of 500 with respect to the presence of disease with no false negative test results and 500 with respect to the absence of disease with no false positive test results. See below as shown.

True characteristics in the population Test Results Have the disease Positive 500 Negative 0 500 Total Sensitivity= 500+0 = 100% Do not have the disease 0 500 Specificity= 500+0 = 100% In practice, there are sometimes no true "gold standard" tests.

SCREENING VALIDITY TEST Screenings tests are tests that look for diseases before you have symptoms. Screening tests can detect diseases early and thus start treatment Detectable Preclinical phase Clinical outcome No disease Symptoms first appear Biologic onset of disease Disease detectable by screening disease diagnostic test Therapy given

• Hypothetical example: Suppose we have a hypothetical population of 1000 people of whom 100 have a certain disease and 900 do not. In this instance we use a screening test to distinguish persons with the disease from those who do not. The results obtain by applying the test to this population of 1000 people are shown below

True characteristics in the population Test Results Have the disease Positive 80 100 Negative 20 800 Total 100 900 Sensitivity= 80 80 +20 = 80% Do not have the disease Specificity= 800 +100 = 89%

• The question we now ask is how good is the screening test compared to the gold standard. First, the test indicates that of the 100 people with the disease, 80 were correctly identified as positive and a positive identification was in 20 and thus the sensitivity of the test which is defined as the proportion of diseased people were correctly identified as positive by the test is 80/100 or 80% Secondly, the test indicates that of the 900 people who did not have the , the test correctly identified 800 people as negative and thus the specificity f the test, which is defined as the proportion of non diseased people who are correctly identified as negative by the test is therefore 800/900 or 89%

Why is the issue of false positive important? The issue of false positive is important because: 1. All people who screened positive are brought back for sophisticated and more expensive test which becomes a burden on the health care system. 2. Another is the anxiety and worry induced in persons who have been told that they tested positive and this labelling is not completely erased even if the results of subsequent evaluation are negative.

Why is the issue of false negative important? The issue of false positive is important because: 1. If for example, the disease is a type of cancer that is curable only in its early stages, a false negative result could represent a virtual death sentence. 2. Thus the result depends on the nature of the severity of the disease being screened for effectiveness of available intervention measure and whether the effeteness is greater if the intervention is administered early in the natural history of the disease

Predictive value of a test We have so far asked the question "How good is the test(gold std) at identifying people with the disease and people without the disease”. Put in another way "If we screen a population, what proportion of people who have the disease will be correctly identified? ” These question is clearly for public health consideration. Positive predictive value of a test In the clinical setting, a different question for the clinician would be "If the test results are positive in this patient, what is the probability that this patient has the disease "This is called the positive predictive value(PPV) In other words the question is “what proportion of patients who test positive actually have the disease in question?

Negative predictive value of a test Also a parallel question can be asked about negative test results as follows “If the test results is negative , what is the probability that this patient does not have the disease "This is called negative predictive value(NPV) of the test.

Calculation of both PPV and NPV PPV of a test is calculated by dividing the number of true positive(TP) by the total number of people who tested positive, i. e. TP divide by TP plus FP NPV of a test is calculated by dividing the number of true negatives(TN) by the total number of people who tested negative i. e. TN divide by TN plus FN

True characteristics in the population Test Results Have the disease Positive 80 Negative 20 Do not have the disease 100 800 Totals 180 PPV=80/180=44% 820 NPV=800/820=90% A Positive predictive value of 44% means that of the 180 tested only 80 have the disease A Negative predictive of 90% means that of the 820 tested, only 800 do not have the disease

Reliability(Repeatability) of a diagnostic or screening tests Reliability is the ability of a measure(both the subject and the observer to produce a repeatable result. This does not mean that the result is accurate, it just means that if the test is repeated, a similar result will be obtained. Clearly , regardless of the sensitivity and specificity of test, if the test results cannot be reproduced, the value of and usefulness of the test are minimal The factors contributing to the variability or non reliability of a test are listed. Intra-subject variation( variation within individual subject 0 Intra-observer variation (variation in reading of test result by same person Inter-subject variation(variation of test results between or among individual Intra-observer variation(variation of test results between observers

Number of patients 1 2 3 4 5 6 7 8 9 10 n=10 Observer 1 Observer 2 0 0 1 0 1 1 0 0 0 0 1 0=Negative 1=Positive

Positive for test 1 and 2 Patients 7 and 10= 2 positive’s Negative for 1 and 2 Patient 1, 4, 6, 8 =4 negative’s Positive for 1 and negative for 2 Patient 3, 5, and 9=3 positives Negative for 1 and positive for 2 Patient 2, =1 negative Observer 1 Positive Negative Total Observer 2 Positive 2 1 3 Negative 3 4 7 Total 5 5 10 Calculate the expected the total expected proportion of agreement (Pe) Calculate the expected the total expected proportion of agreement(Po) The (kappa) statistic

• Use the 2 x 2 table on the next slide for calculating the kappa statistic (strength of agreement): ü The total expected proportion of agreement, Pe, is given by : Pe = [ (a+c) * (a+b)] + [(b+d) *(c+d)] N N ü The total observed proportion of agreement, Po is given by: Po= (a+d) N ü The (kappa) statistic is given by: Po- Pe 1 - Pe where Po-Pe represents the proportion of the observed agreement in excess of chance, and 1 -Pe represent the maximum proportion of agreement in excess of chance.

• • • Value of kappa <0. 2 Poor 0. 21 -0. 41 -0. 61 -0. 81 -1. 0 Strength of agreement Poor Fair Moderate Good Very good

Read chapters ……………. Elect 2 students to act as Doctor X and Doctor Y The rest of the students will be patients, half of which will go to Dr X, the other half to Dr Y • The doctors will interview the patients, and decide whether they are eating a healthy diet or should be referred to a nutritionist • The recommendation for each patient will be recorded on the data collection sheet by each doctor. • Then the doctors will swop patients and repeat the interviews. • No contamination must take place, i. e. the doctors and patients should not share with each other anything that was said by their counterpart in the previous interview. • The 2 doctors will make 10 copies of their data collection sheets and distribute these to the 10 class groups. • The groups will summarise the data using the data summary sheet. • Each group will calculate the kappa statistic, and the results will be presented to the class. • • •