Research Designs Nonexperimental designs 1 Causal modelling 2

Research Designs Nonexperimental designs 1 Causal modelling 2 tests of the models college titel en nummer

Characteristics • No manipulation • No random assignment • In order to test causal hypotheses all confounding variables have to be included in the study in order to control for their effects 11/30/2020 college titel en nummer 2

The Problem • We are interested in the effect of a X (seeing a movie) on a dependent variable Y (Opinion). • X -------> Y • X= 0 if people have not seen a movie • X= 1 if people have seen a movie. • One can not simply use the difference of means between the two groups because these two groups can differ on several variables X 1 to Xk and not only X. 11/30/2020 college titel en nummer 3

Similar and Different • This is the same problem as in quasi experimental designs • The difference is that one can not use a pretest score • So the only solution is to control for all so called confounded variables. • Looking for these variables is called “Causal modeling” 11/30/2020 college titel en nummer 4

Causal Modelling • All variables which can give an alternative explanation for the relationship between X and Y have to be introduced in the study in order to control for their effect : • X • • X 1 11/30/2020 Y . . . Xk college titel en nummer 5

Research in Russia • Some years ago somebody suggested: Russia represents a real life experiment for social change. • It is clear this is not a real experiment: We can only observe changes. • The problem to study was: • What change occurs ? • What is the effect of change on satisfaction with some aspects of life ? 11/30/2020 college titel en nummer 6

Three causal hypotheses • We expect that satisfaction is determined by an objective and a subjective characteristic: • the situation as it is and the preferred ideal situation • So we expect the following DIRECT EFFECTS • 1. the satisfaction (S) will remain the same as before unless • 2. A change in the objective situation(DS) will change satisfaction • 3. A change in the preferred ideal situation (DI) 11/30/2020 college titel en nummer 7

Path diagram • This leads to the following causal model: DS • • S(t) S(t-1) • DI • The direct effects are represented by directed arrows. 11/30/2020 college titel en nummer 8

Indirect effects and Spurious relationships • S (t-1) has also indirect effect on S(t) because • hypothesis 1: • Some people who are dissatisfied will try to change their situation • hypothesis 2: • Some people who are dissatisfied and do not expect any change will adjust their ideals downwards. 11/30/2020 college titel en nummer 9

Adjustment of the model • Two more direct effects of S(t-1) on DS and DI: • DS • S(t) S(t-1) • DI • A consequence of this extention of theory with two direct effects is that S(t-1) has two indirect effects on S(t) and that • part of the relationship between S(t) and DS and between S(t) and DI is spurious. Check this. 11/30/2020 college titel en nummer 10

OTHER SPURIOUS RELATIONS • Some of the relationships of S(t-1) on the other variables can also be spurious because of effects of the Difference between the objective situation and the ideal at time t-1 called DI(t-1). • We hypothesize: • DI(t-1) has an effect on S(t-1) and DI 11/30/2020 college titel en nummer 11

More • We also hypothesize also that there are effects of the expectations concerning the possibilities to Change the situation will cause spurious relations. We call this EPD. • We hypothesize : • EPD has an effect on S(t-1) and DS. 11/30/2020 college titel en nummer 12

Second Adjustment of the model • This leads to the following model DS • • S(t) EPD S(t-1) • DI DI(t-1) • Check where spurious relationships arize due to this extention of the causal model. 11/30/2020 college titel en nummer 13

WHEN IS A MODEL COMPLETE ? • As was mentioned before: • all variables which cause spurious relationships have to be introduced. • Variables which only influence the cause or only the effect variable can be omitted • Intervening variables can also be omitted. 11/30/2020 college titel en nummer 14

More variables • Part of the relationship between S(t-1) and DI(t 1) and EPD are certainly spurious due to satisfaction at a previous point in time , S(t-2), effecting S(t-1) and DI(t-1) and EPD. 11/30/2020 college titel en nummer 15

Solution • But this variable S(t-2) does not directly affect S(t) and DS and DI. • So ignoring this variable will only disturb the relationships between the variables S(t-1) , DI(t -1) and EPD. • This problem can be solved by introduction of a correlations between S(t-1), EPD and DI(t-1) representing the effect of S(t-2). 11/30/2020 college titel en nummer 16

The final model • The leads to the following final model • DS EPD • S(t) S(t-1) • DI DI(t-1) • The double headed arrows are correlations • The effects of the predetermined variables on each other can not be studied. 11/30/2020 college titel en nummer 17

Testing Causal models • In nonexperimental research testing of the model is essential. • Without the test one does not know if the estimated values are correct • A two steps procedure will be discussed to test these causal hypotheses: • 1 estimation of the effects • 2 testing the model 11/30/2020 college titel en nummer 18

Data: Correlations • • s(t) Ds Di s(t-1) ep. D di(t-1) 11/30/2020 s(t) Ds Di s(t-1) ep. D di(t-1) -------- -------1. 00 -0. 04 1. 00 -0. 40 -0. 14 1. 00 0. 34 -0. 12 0. 29 1. 00 0. 02 -0. 09 0. 02 1. 00 0. 31 -0. 06 0. 58 0. 61 -0. 02 1. 00 college titel en nummer 19

Correlation is not effect • Correlation = direct effect + indirect effects + spurious relationships + joint effects • So the correlation is not the proper coefficient to use for tests • The direct effects are the correlation - indirect effects - spurious relationships - joint effects • These effects are fundamental for causal analysis 11/30/2020 college titel en nummer 20

Estimation of the direct effects • Regression equations: • S(t) = a 1+ b 12 DS + b 13 DI +b 14 S(t-1) • DS = a 2 + b 24 S(t-1) + b 25 EPD • DI = a 3 + b 34 S(t-1) + b 35 DI(t-1) 11/30/2020 college titel en nummer 21

Estimation of equation 1 • Equation Number 1 Dependent Variable. . • S(t) How satisfied are you • --------- Variables in the Equation --------- • Variable • • DI -. 451272. 027172 -. 536938 -16. 608 DS. 061065. 275531. 006916. 222 S(t-1). 476791. 031331. 495480 15. 218 (Constant) 2. 918733. 189812 15. 377 11/30/2020 B SE B Beta T college titel en nummer Sig T. 0000. 8247. 0000 22

Estimation of equation 2 • Equation Number 2 Dependent Variable. . • DS Change in situation • --------- Variables in the Equation --------- • Variable • • • S(t-1) EPD (Constant) 11/30/2020 B -. 012817 -. 027567. 232339 SE B Beta T . 004240 -. 117603 -3. 022. 011930 -. 089906 -2. 311. 039426 5. 893 college titel en nummer Sig T. 0026. 0212. 0000 23

Estimation of equation 3 • Equation Number 3 Dependent Variable. . • DI change in their ideal • --------- Variables in the Equation --------- • Variable • • • S(t-1) DI(t-1) (Constant) 11/30/2020 B SE B Beta T Sig T -. 127727. 045763 -. 111556 -2. 791. 0054. 769304. 047097. 652877 16. 335. 0000 -2. 948347. 217671 -13. 545. 0000 college titel en nummer 24

The result • Before drawing conclusions one has to know if these results can be trusted • In nonexperimental research model testing is essential 11/30/2020 college titel en nummer 25

The expected correlations • sums of direct, indirect effects , spurious relations and joint effects assuming that the model is correct • s(t) Ds Di s(t-1) ep. D di(t-1) • -------- ------- • s(t) 1. 01 • Ds -0. 10 1. 00 • Di -0. 40 -0. 03 1. 00 • s(t-1) 0. 34 -0. 12 0. 29 1. 00 • ep. D 0. 02 -0. 09 -0. 01 0. 02 1. 00 • di(t-1) -0. 02 -0. 07 0. 59 0. 61 -0. 02 1. 00 11/30/2020 college titel en nummer 26

Residuals • Residual = observed correlation - expected correlations assuming that the model is correct • s(t) Ds Di s(t-1) ep. D di(t-1) • -------- ------- • s(t) -0. 01 • Ds 0. 06 0. 00 • Di 0. 01 -0. 10 0. 00 • s(t-1) 0. 00 • ep. D 0. 00 0. 03 0. 00 • di(t-1) 0. 33 0. 01 0. 00 11/30/2020 college titel en nummer 27

The adjusted model 11/30/2020 college titel en nummer 28

The new estimates 11/30/2020 college titel en nummer 29

Does the model fit ? • The residuals are significantly different from zero • So the model still has to be rejected but is much better than before • However we will show that this test is too simple and an alternative will be introduced. 11/30/2020 college titel en nummer 30

The new estimates 11/30/2020 college titel en nummer 31

Some substantive conclusions • Changes in the living conditions (DS) has hardly any effect • The difference between the reality and the ideals (DI(t-1)) is the most determining factor for • S(T-1) and DI • The strongest effect on satisfaction (S(t) ) comes from Changes in ideals DI : • People who adjust their ideals downwards become more satisfied !!! 11/30/2020 college titel en nummer 32

Methodological Conclusion • Also in case of nonexperimental research causal hypotheses can be tested • But it requires a lot of preparation in advance : causal modeling • because there is no randomization and • also a pretest is normally not available • So one has to control for all variables which can cause a spurious relationship between the variables of interest. 11/30/2020 college titel en nummer 33

Methodological conclusions • Most of the time regression analysis is used • Regression coefficients estimate indeed the effect of an independent variable on a dependent variable • but only if • the causal model is well developed so that all relevant variables are introduced • and the model fits to the data 11/30/2020 college titel en nummer 34