Laboratory v field experiments The tradeoff between internal

Laboratory v. field experiments The tradeoff between internal and external validity

Natural sciences • In natural sciences, most experiments have been undertaken in carefully controlled laboratory conditions – Seeking laws of physics, etc. • Inanimate objects are unlikely to react differently in controlled and uncontrolled conditions • The main goal is to avoid contaminating the experiment via the introduction of third variables • Far more exacting and finer measurement is possible under controlled conditions

Laboratory experiments in social science • Closely controlled social science experiments do have the advantage of limiting the impact of third variables, but the unnatural situation presents problems for applying the findings outside the laboratory – “Artificiality” of laboratory setting

• Human behavior is sensitive to the environment within which it occurs – People act differently in a laboratory than in the natural world • Several characteristics of laboratories are thought to influence behavior • The very third variables controlled in the lab may be the ones that determine behavior in the real world • So, findings from laboratory experiments may only be valid for laboratory environments

An example • New commercials are tested in controlled conditions – Eye tracking – Liking for commercials – Influence on purchase interest • May try to provide less artificial conditions for study – Simulated living room • Commercials that test high in lab experiments often do not work very well when used in real marketing campaigns

So, experiments move out of the lab: • Researchers want to retain some of the advantages of the experiment: – Ability to manipulate/introduce the independent variable and to control how much of it is presented – Time order—which comes first • While sacrificing some of their ability to control third variables • The goal is to improve our ability to generalize our findings to the real world

The field experiment • One way to do so is to carry out a ‘field experiment’ – The researcher still manipulates the independent variable, but she does so within the natural world – For example, police try randomly stopping motorists and testing for alcohol to see if alcoholrelated crashes decline

Information Resources, Inc. • Behavior. Scan system

What problems do we encounter? • Greatly reduced ability to prevent third variable contamination of results – The crusading doctor in MHHP – History effects • Cycles • What can you do? – – – Measure/monitor likely alternative explanatory variables Question subjects about sources of influence Multiple manipulations of the independent variable over time Multiple measurement waves Multiple dependent measures • None will be perfect

• Expense – Tradeoff between extensive and intensive study – Budget constraints on number of sites, etc. • Access/permission – Some research may present concerns to authorities, citizens, etc. • Gain authorization/support prior to entering the field • Maintain good relationships with community leaders, etc. throughout the intervention/research

Capturing ‘natural experiments’ • Sometimes unusual or unique events occur – ALAR scare – Service disruptions – Political campaigns – School shooting – Institution of Wi-Fi in an area

“Natural experiments” • Because most such events are unplanned, the ability to prepare for them is limited – May keep a research group, materials and resources ready for certain types of events • Must engage in ‘firehouse research’ gathering as much data as possible in a short time – Inefficient, and may miss important data – However, real-world events, etc. may provide very valuable data—may have both internal and external validity

• Use statistics, research analysis models that are appropriate to experimental studies