VALIDITY AND RELIABILITY The two most important terms

VALIDITY AND RELIABILITY The two most important terms used in scientific research to refer to the quality of trustworthiness of data are validity and reliability Investigators attempt to maximize the reliability and validity of the data they collect by using properly constructed tools, appropriate data collection procedures and sampling techniques that not only target the right population, but also yield accurate data Any measurement has two components: the true value or score and an error component The error component of the data reflects the limitations of the instruments and the data collection procedures 10/18/2021 owinojoseph@gmail. com 1

VALIDITY AND RELIABILITY (Cont’d) For each measurement, the error component can further be partitioned into random error and systematic error Random error affects the reliability while systematic error affects validity Reliability is therefore, concerned with the internal properties of a measure, whereas validity refers to the relationship between the data and the variable being measured 10/18/2021 owinojoseph@gmail. com 2

Reliability is the consistency of measurement Reliability is a measure of the degree to which a research instrument would yield the same results or data after repeated trials It is a measure of the degree to which a research instrument would yield the same results or data after repeated trials Data obtained from behavioural research studies are influenced by random errors of measurement Reliability in research is influenced by random error. As random error increases, reliability of the data decreases 10/18/2021 owinojoseph@gmail. com 3

Reliability (Cont’d) Errors may arise from inaccurate coding, ambiguous instructions to the subjects, interviewers’ fatigue, interviewees’ fatigue, interviewer’s bias etc. Random error will always exist regardless of the procedures used in a study The smaller the deviations of the observations from the true measurement, the more reliable the data will be There are five different aspects of reliability of data that are commonly assessed depending on the nature and measurement of each variable construct. Techniques for assessing reliability are discussed hereunder: 10/18/2021 owinojoseph@gmail. com 4

Types of Reliability i) Inter-rater reliability is applicable in observational studies and estimates the degree of agreement between or among two or more people observing and then rating the same activity The reliability estimate is given in terms of the correlation between the two sets of score given by two different observers 10/18/2021 owinojoseph@gmail. com 5

Types of Reliability (Cont’d) ii) Test-retest technique The test-retest method of assessing reliability of data involves administering the same instrument twice to the same group of subjects There is usually a time lapse between the first testing period and the second testing period The main disadvantage of the test-retest technique is that subjects may be sensitized by the first testing so they may easily remember their responses during the second testing. This may lead to assessing memory instead of reliability 10/18/2021 owinojoseph@gmail. com 6

iii) Equivalent or alternate-form technique One way of overcoming the problem associated with the test-retest technique is to administer two equivalent scales For each variable, two scales comprising different items but designed to measure exactly the same attribute or trait are developed. Such tests are called parallel tests The estimate of reliability is obtained by computing the correlation between scores obtained by the same individuals on the two scales for each variable iv) Split-half technique The split-half technique develops one scale for each variable then dividing the scale into two halves, which are taken simultaneously then scored separately for each subject 10/18/2021 owinojoseph@gmail. com 7

VALIDITY Validity establishes the relationship between the data and the variable or construct of interest Validity is defined as the accuracy, truthfulness and meaningfulness of inferences that are based on the data obtained from the use of a tool or a scale for each construct or variable in the study Validity therefore, estimates how accurately the data obtained in the study represents a given variable or construct in the study Validity is concerned with the meaningfulness of research components Presence or absence of systematic error in data largely determines validity of data for each construct or variable 10/18/2021 owinojoseph@gmail. com 8

Threats to Internal Validity 1. History refers to situations where a study extends over a long period of time. During such an extended period, events occur that ultimately influence the subjects or objects being studied. 2. Maturation is a threat to internal validity of a study where biological or psychological processes, which may occur among the subjects in a relatively short time, influence research findings 3. Instrumentation compromises internal validity of a study when the measurement tools and procedures are flawed 10/18/2021 owinojoseph@gmail. com 9

Threats to Internal Validity (Cont’d) 4. Pre-testing Pre-test sensitizes participants and therefore, they may tend to perform better on the post-test not because of the treatment alone but, because they have been sensitized to the test 5. Statistical regression is a problem in studies where the selection of subjects is based on their performance on the pre-test. This situation occurs because extreme scores tend to have the highest errors of measurement 10/18/2021 owinojoseph@gmail. com 10

Threats to Internal Validity (Cont’d) 6. Attrition occurs when many subjects or participants in a study drop out before the study is completed. Attrition interferes with the original randomness of the sample which leads to bias 7. Differential selection occurs when subjects are systematically selected into a treatment group 10/18/2021 owinojoseph@gmail. com 11