Quasi Experiments NonExperimental Research Methods Statistics Fall 2011

Quasi Experiments Non-Experimental Research Methods & Statistics Fall 2011

Quasi Experiments • “Research procedure in which the scientist must select subjects for different conditions from preexisting groups” – Research Methods, Mc. Burney & White • “An empirical study used to estimate the causal impact of an intervention on its target population. Quasiexperimental research designs share many similarities with the traditional experimental design or randomized controlled trial, but they specifically lack the element of random assignment to treatment or control. Instead, quasiexperimental designs typically allow the researcher to control the assignment to the treatment condition, but using some criterion other than random assignment. – Wikipedia, Nov. 17, 2011

It’s a matter of control True Experiment Quasi Experiment • Random assignment of subjects to condition • Manipulate the IV • Selection of subjects for the conditions • Observe categories of subjects – If the subject variable is the IV, it’s a quasi experiment • Control allows ruling out of alternative hypotheses • Don’t know whether differences are caused by the IV or differences in the subjects

Other features • In some instances cannot completely control the what, when, where, and how – Need to collect data at a certain time or not at all – Practical limitations to data collection, experimental protocol

Validity • Internal validity is reduced due to the presence of controlled/confounded variables – But not necessarily invalid • It’s important for the researcher to evaluate the likelihood that there alternative hypotheses for observed differences – Need to convince self and audience of the validity

External validity • If the experimental setting more closely replicates the setting of interest, external validity can be higher than a true experiment run in a controlled lab setting • Often comes down to what is most important for the research question – Control or ecological validity?

Nonequivalent Control Group Designs • 2 groups, non-random allocation of subjects and groups, pre-test, treatment (Y/N), post-test • Desired pattern: – Dependent variables have equal pre-test values, difference seen between experimental and control groups on post-test (Pre: LL, Post: HL) – Want to show that any differences that exist did not impact the value of the variable of interest

Exercise • Interpret the following graphs (experimental, control): – Pre: (L, H) Post: (M, H) – Pre: (L, M) Post: (H, ML) – Pre: (L, M) Post: (H, M)

Mixed Factorial Design with One Nonmanipulated Variable • Example: experiment on pain perception (effect of caffeine, expected differences between men and women) • Protocol: – 25 men/25 women, each takes part in two sessions, one week apart – One session: drink coffee (decaf) and put hand in ice-water until feel pain – Other session: drink coffee (caffeinated) and put hand in ice-water until feel pain • Between subjects variable (male/female) • Within subjects variable (caffeine intake)

Non-Experimental Research Read and understand: • Gabriella Belli - Nonexperimental Quantitative Resarch • “Any quantitative study without manipulation of treatments or random assignment is a nonexperimental study” • Experimental research shows cause and effect • Non-experimental research studies variables as they exist

Purpose • Descriptive: – primary focus for the research is to describe some phenomenon or to document its characteristics. • Predictive: – primary focus for the research is to predict some variable of interest (criterion) using information from other variables (predictors). • Explanatory : – the primary focus for the research is to explain how some phenomenon works or why it operates.

• Cross-sectional: Time Frame – data are collected at one point in time, often in order to make comparisons across different types of respondents or participants. • Prospective (longitudinal): – data are collected on multiple occasions starting with the present and going into the future for comparisons across time. • Retrospective: – look back in time using existing or available data to explain or explore an existing occurrence.

Techniques • Surveys – http: //www. socialresearchmethods. net/kb/survey. php • Interviews – http: //www. socialresearchmethods. net/kb/intrview. php • Observations – http: //interactionarchitect. com/knowledge/article 19991212 shd. htm – http: //www. edu. plymouth. ac. uk/resined/observation/obshome. htm • Unobtrusive methods: – http: //www. socialresearchmethods. net/kb/unobtrus. php