Experimental Research Design Causality Validity Threats to Validity
- Slides: 20
Experimental Research Design Causality & Validity • Threats to Validity – Construct (particular to experiments) – Internal – External – already discussed
Threats to Construct Validity • Mono Method bias – Remove common method variance – Establish convergent & discriminant validity • Hypothesis guessing • Inappropriate level of construct – Usually lower than what occurs in natural setting
Threats to Internal Validity • Time – History, Maturation • Measurement – Testing, Instrumentation • Selection – Randomization, Statistical regression, Mortality • Contamination – Diffusion, Compensation • Interaction among threats
Internal Validity • Establish that cause led to the effect – Identify non-experimental (uncontrolled) factors that affect the dependent variable or the relation b/w the independent and dependent variables
10 Threats to Internal Validity Time • History – Factors occurring along with the independent variable • An event occurring at the time of the intervention that overshadows (or reduces the effect of iv) and causes changes in dv – E. g. , change in manufacturing material along with training influences performance quality
Copyright © 2003 John Wiley & Sons, Inc. Sekaran/RESEARCH 4 E FIGURE 7. 1
10 Threats to Internal Validity Time • Maturation – Biological or psychological processes or factors that occur w/the passage of time & affect the dependent variable • E. g. , job experience, socialization etc. affect performance
Copyright © 2003 John Wiley & Sons, Inc. Sekaran/RESEARCH 4 E FIGURE 7. 2
10 Threats to Internal Validity Measurement • Testing – Studies w/pre-post measures or multiple measures • Increased or decreased sensitization to posttest measures • Changes in participants due to measurement itself • Increased fatigue or resentment • Self-consistency effects
10 Threats to Internal Validity • Instrumentation – Change in the measures from pre to post test • Change in observers’ frames of reference • Changes in participants’ frames of reference (e. g. , beta & gamma changes p. 43 Saks) • Change in instrumentation quality (e. g. , performance quality measurement)
10 Threats to Internal Validity • Instrumentation – Use of different measurement techniques • Establish expected convergent & discriminant validity • Importance of using different measures to overcome percept-percept problem
Threats to Internal Validity Selection • Randomization – Randomly assign to experimental conditions • Only in experimental designs • Matching on key variable if random assignment not possible
Threats to Internal Validity Selection • Randomization – Randomly select from population • Calculate response rate • Establish comparability of respondents to population on key features • Stratified (v. cluster? ) sampling
10 Threats to Internal Validity Selection • Statistical Regression – Upon repeat measurement, scores usually “regress” or “move toward” the mean • Usually when extreme scorers are selected into the study – Can be classified as ‘error’ variation
10 Threats to Internal Validity Selection • Mortality – Features of participants who do not complete the study affect the results – Reduced power in testing effects
10 Threats to Internal Validity Contamination • Diffusion/imitation of treatments – When people in control groups inadvertently know about or are affected by intervention and that changes their behavior
10 Threats to Internal Validity Contamination • Compensatory Effects – Knowledge about intervention and changes in control group members’ behavior is more purposeful
10 Threats to Internal Validity • Interaction among threats – When multiple threats operate at the same time obscuring the effect of the IV on the DV
Threats to Internal Validity • Time-related – History, maturation • Measurement – Testing, Instrumentation • Selection – Randomization, Statistical regression, mortality • Contamination – Diffusion, Compensation • Interaction among threats
- Disadvantages of experimental research
- Experimental vs nonexperimental
- Internally valid definition
- Criterion validity vs predictive validity
- Research paradigm example
- Research instrument in experimental research
- Experimental vs non experimental
- What are the 5 types of non experimental research design
- Experimental vs non experimental
- Characteristics of experimental research design
- Non experimental research topics
- Disadvantage of quasi experimental design
- Non experimental research correlational design
- Experimental research design
- Experimental research design types
- Classic experimental design
- Extraneous variable example
- Research design
- 4 hurdles of causality
- Association vs causality
- Kausal