Experimental Quasi Experimental Design Outline Basics of Experimental

Experimental & Quasi. Experimental Design

Outline Basics of Experimental Design Pre-Experimental Design Classic Experimental Design Quasi-Experimental Design Advantages and Limitations of Experimental Designs

Basics of Experimental Design An experiment is a scientific investigation that tests cause-and-effect relationships while controlling for the influence of other factors. A key feature of the design is that measures are taken on two or more groups both before and after an experimental intervention.

The rationale for Experimental Design Nurse researchers use experimental designs to measure if a treatment (intervention) has a particular effect on some outcome. It is important to ensure that other possible reasons for variations in the outcome have been ruled out. In short, we must be confident that it is the treatment and not something else that is effecting the outcome.

Example : If you were studying the effect of listening to instrumental music on a patient’s stress level on the day of surgery, one would want to make certain that factors such as the severity of the illness, gender, and age of the patient have been taken into account.

Experimental researchers use the term internal validity to refer to the extent to which one can demonstrate that one’s treatment is having an impact on a given outcome and that other sources of influence have been controlled. External validity, on the other hand, refers to the extent to which one can make extrapolations from the particular study to other groups in general.

Key Element in Experimental Designs Time : A key feature of an experiment is that measures are taken at different points in time (i. e. , before and after the treatment is administered). Variables in Experiments : Treatment variable, Control Variables, confounding Variables, Random Variables. Treatment Levels : Often, researchers examine two or three different treatment levels. These refer to the different treatments to which subjects are exposed.

Variables in Experiments A dependent variable is the effect in a cause-and-effect relation. In an experiment, the dependent variable is the phenomenon measured in order to determine if any change has taken place as a result of some experimental treatment. The independent variables include all the variables taken into account, or manipulated, by the researcher that may influence the dependent variable.

Treatment Variable In an experiment, the independent variable whose effect is being studied is known as the treatment variable. An experiment attempts to detect the direct effect of the treatment on the dependent variable. Some experiments include more than one treatment variable. The experimenter must determine the effects of each treatment used on the dependent variable.

Control Variables Control variables are those specifically taken into account in designing a study. These variables need to be controlled, or taken into account, in designing the study to ensure that the study findings accurately reflect reality.

Confound Variables Experimentalists refer to confounding variables as known factors that may unintentionally obscure or enhance a relationship. (Survey researchers use the term source of spuriousness to refer to the same idea. )

Random variables are unknown factors that may have an impact on our dependent variable. It is important to control for these variables; otherwise, we may falsely conclude that it was our treatment that influenced the dependent variable.

Pre-Experimental Designs Pre-experimental designs are considered to be of limited scientific merit because they cannot rule out alternative explanations for observed relationships. A pre-experimental design is one that does not permit clear causal inferences about the impact of a treatment on the dependent variable. Although these designs share some of the elements of the experiment, their designs inhibit clear causal interpretations.

Same Group : Pretest and Posttest Design Suppose you decided that you would need a measure of the students’ predis- positions toward a university nursing education before seeing the CD-ROM. Having completed this measure, the CD-ROM could then be shown, and the difference between the pre- and posttest scores could be used to measure the change in the desire to attend a university nursing program.

History Maturation Testing Instrument Decay Statistical Regression

Exposed or Comparison Group Design In the above design, one group is exposed to the CD-ROM promoting university nursing education, and the other is not. Can we legitimately conclude that the difference in attitude between the exposed group and the unexposed group represents the impact of the CD-ROM?

Two additional confounding factors are noted by Campbell and Stanley (1966): Selection & Mortality Selection thus refers to subjects selecting them selves into a study. Mortality thus refers to subjects selecting themselves out of a study. Therefore, mortality may systematically distort the results of the study.

Classic Experimental Design To solve the problems in causal inference faced by preexperimental designs, classic experimental designs may be used. Classic Experimental Design : Between-Subjects Design Within-Subjects Design Between-and-within-subject Designs Compared

Between Subject Design A between-subjects design, also known as the pretest, posttest control group design, contains three crucial elements of the classical experiment: control over extraneous variables, methods of dealing with pretreatment similarity of groups, and manipulation of a treatment. A between-subjects design involves a control and an experimental group. Measures are taken from members in both groups before treatment and repeated after the treatment has been experienced. Whereas the control group is exposed to a neutral treatment (seeing the nature CD-ROM in the illustration we have been using), the experimental treatment is the focus of the study (the university CDROM). Data analysis reveals whethere are differences in the dependent variable as a result of the intervention, while taking into account other factors.

The experimental group is the one that is exposed to the treatment intervention. The researcher attempts to ensure that each subject experiences the intervention in as similar a fashion as possible to deal with extraneous factors that may be influencing the outcome of the study. The control group is established so that comparisons can be made between the experimental group and the control group. Normally, the control group is exposed to a placebo or some neutral treatment that is similar in the time it takes and the kind of experience it represents. In the case of the CD-ROM example, the control group is exposed to a CD-ROM having a nature theme.

Achieving Equivalence of Control and Experimental Groups in a Between-Subjects design The experimental group is the one that is exposed to the treatment intervention. The control group is established so that comparisons can be made between the experimental group and the control group.

There are three ways of achieving equivalence: 1. Through randomization (in which individuals are randomly assigned to either the treatment or to the control group) 2. Through precision matching (e. g. , matching pairs according to gender, socioeconomic status, and grades, and then assigning one person from each of the matched pairs to the control group and the other person to the experimental group) 3. Through a combination of the above two: using precision matching to match pairs and then randomly assign one member of each pair to the experimental group and the other one to the control group

Analyzing the Data To estimate the impact of the CD-ROM, we could do the following computation on the percent wanting to attend a university nursing program before and after seeing the CD-ROM (Table 4. 3):

The seven factors Campbell and Stanley (1966) identified as confounding interpretations are dealt with in between subjects designs through: • Attempting to make certain that the groups are similar to begin with, • Noting that the various factors should influence the treatment and the control groups equally. Thus, between-subjects designs rely on establishing pretreatment similarity of control and experimental groups to minimize the effects of history, maturation, testing, instrument decay, statistical regression, selection, and mortality.

Demonstrating a Causal Relationship in a Between-Subjects Design To show a causal relationship in any research, three conditions must be demonstrated: 1. Changes in the treatment variable must occur before changes in the dependent variable. 2. The variables must be associated; as values on the treatment variable are increased, the dependent measures vary correspondingly. 3. Nothing but the treatment variable can have influenced the dependent variable.

Nothing but the treatment variable can have influenced the dependent variable. Step 1: Ensure that the context in which the experiment is carried out is the same for all subjects. Step 2: Balance the background characteristics of the subjects. Step 3: Neutralize any confounding variables. Step 4: Deal with random variables.

Within-Subject Design An alternative to the between-subjects design is a withinsubject design (also known as repeated measures designs). Sometimes these designs use a single subject, sometimes just a few subjects, and sometimes large samples. What is the idea behind this kind of design? The key advantage of a within-subject design is that it provides convincing evidence for the impact of the treatment variable on the dependent variable.

Between and Within-Subjects Design Compared What is the efficacy of therapeutic touch (a specialized type of healing touch often used in nursing practice) on situationally induced stress in healthy adult women? Student nurses comparison: a between-subjects design. Experienced nurse practitioners: a between-subjects design. Expert nurse practitioners: a withinsubject design.

Quasi-Experimental Designs A quasi-experimental design is one in which it has not been possible to meet all of the following requirements: • Randomly assign subjects to a treatment or control group. • Control the timing or nature of the treatment. An important feature of the quasi-experimental design is its attempt to compensate for the absence of either the random assignment or the control group component by introducing other controls. Example : Nonequivalent Control Group Design, Panel Studies.

Advantages & Limitation of Experimental Designs Pre-experimental Design Limitations : The pre-experimental design involving a same group: pretest and posttest discussed earlier in this chapter had problems in that in this design all kinds of other random factors may have influenced the change in outcome measures.

Quasi-Experimental Designs Advantages Quasi-experimental designs are typically done when the topic dictates that an experiment is not possible. Thus, they are often the best that can be done given certain ethical or practical constraints. Sometimes quasi-experimental designs involve representative samples. In such studies, a group of individuals (usually called a panel) is monitored over time. Limitation Given limited control over the context in which the study is conducted, quasi experimental studies lack the causal inference power of the classic experiment.

Classic Experimental Designs Advantages The advantage in both these designs is that they can provide powerful, convincing evidence for the effect of a treatment variable. Limitations : External Validity, Limit to Predictability in Experimental Studies, Artificiality Issue, Limitations on Number of Variables, Experiments Cannot Study All Topics.

Artificiality Issue The third major concern is that there is an element of artificiality in laboratory experiments that is difficult to detect, interpret, or control. In the nursing world, the natural environment in which the client is situated bears heavily on health outcomes. Controlling all factors, as you do in a classic experiment, may not yield results that are applicable in the real world, where you are unable to control the same set of factors.

External Validity Few experiments are conducted using samples that can be viewed as representative of a larger population; therefore, one cannot make extrapolations to the general population. Clearly one would need to repeat the study in many different locales to determine the extent to which valid generalizations may be drawn from a single study.

Limit to Predictability in Experimental Studies The fact that experimental studies do a better job than surveys in predicting outcomes should not be misinterpreted. Experiments should produce higher predictability than that achieved by non experimental designs using the same variables. A robust variable in an experiment, however, may be relatively impotent outside the laboratory setting.

Limitations on Number of Variables. For practical reasons, experimental designs can only deal with a few variables simultaneously. (Multiple-variable designs are possible, but with several treatment and control levels, one would soon need far too many subjects to run the experiment.

Experiments Cannot Study All Topics. It is neither practical nor ethical to experiment on all aspects of human behavior; therefore, other designs remain important tools for nurse researchers. For example, in a study exploring the effectiveness of a new birth control method, it would be impractical and entirely unethical to randomly assign half the clients at a well women’s clinic to use no birth control while the other half would be assigned to use the new method.

Exercise 1

Exercise 2

Exercise 3