Concepts to be included Measurement validity External invalidity

Concepts to be included § Measurement validity § External invalidity § Internal invalidity § Statistical conclusion validity

VALIDITY IN CAUSAL RESEARCH HENK VAN DER KOLK

AIM Four types of validity in the context of causal inference: § Statistical conclusion validity § Internal validity § Measurement and sampling validity § External validity

EXAMPLE ‘Does comedy (jokes, fun, laughther) improve the outcomes of academic learning among students at this university? ’ Studying a sample of IBA students

ASSESSING RESEARCH DESIGNS FOUR CRITERIA - Statistical conclusion validity - Internal validity • Correct conclusions drawn in the study itself - Measurement and sampling validity - External validity • Generalization & inference to theory/population/other cases

A WARNING The four types only help to find specific problems in causal research, they are not a classification scheme of problems. Mistakes in a study can often be interpreted in several ways.

EXAMPLE Suppose we do a classical experiment to study the relationship between ‘fun’ and ‘learning’. § Control group gets ‘normal lecture’ § Experimental groups gets the same lecture, but more funny

INTERNAL VALIDITY & STATISTICAL CONCLUSION VALIDITY Are the conclusions about the association, time order between the measured variables correct? Is the relationship unaffected by a third variable (no spurious relationship).

STATISTICAL CONCLUSION VALIDITY Correct handling of data and using the appropriate statistical tests. Mainly about correctly establishing associations between variables.

INTERNAL VALIDITY Is the time order between the measured variables correctly established? Is the relationship unaffected by third variables? (no spurious relationship).

ASSESSING A SPECIFIC STUDY The study: Statistical conclusion validity Units Treatments Observations (= independent variables) (= dependent variables) Internal validity

MEASUREMENT & SAMPLING VALIDITY Measuring the variables and units appropriately. Referring to theoretical constructs intended. Examples: § Is making some jokes in the lecture a good indication if ‘comedy’ in teaching? (measurement validity) § Is studying only IBA students adequate? (sampling validity)

ASSESSING RESEARCH DESIGNS The theory: Units Theoretical construct Setting Sampling validity Measurement validity The study: Treatments Units (= independent variables) Observations (= Dependent variables) Study setting

EXTERNAL VALIDITY The theory: Units Theoretical construct Setting External validity Sampling validity Measurement validity The study: Treatments Units (= independent variables) Observations (= Dependent variables) Study setting

THIS MICROLECTURE Four criteria: • Statistical conclusion validity • Internal validity • Measurement and sampling validity • External validity

IMAGES USED: Slide 4: https: //pixabay. com/en/students-outdoor-people-young-703003/ Slide 7: https: //pixabay. com/en/speakers-speaker-training-lecture-414553/ Slide 9: https: //pixabay. com/en/analysis-statistics-chart-graphic-810025/