Nonequivalent Control Group Langer and Rodin 1976 Lack

Nonequivalent Control Group • Langer and Rodin (1976) Lack of opportunity to make personal decision contributes to psychological debilitation amongst nursing home residents. • Comparison Groups • (1) Treatment group: • communication stressing independent decision making • were also given a small plant as a gift (if they decided to accept it) • living on one floor of the nursing home • (2) Nonequivalent control group: • communication stressing staff responsibility for them • also received a plant as a gift (whether they chose to have one or not) • living on another floor of the nursing home • Questionnaires about control and happiness • One week before treatment • Three weeks after treatment

Nonequivalent Control Group • Suppose group differences are observed at a posttest. • Rule out threats to internal validity: – By adding a comparison group, researchers can rule out threats due to history, maturation, testing, instrumentation, and regression. – We assume that these threats happen the same to both groups, therefore, these threats can’t be used to explain posttest differences.

Nonequivalent Control Group • What threats are not ruled out? – Selection: • Because individuals are not randomly assigned to conditions, the two groups are not likely to be equivalent before the intervention (hence, “nonequivalent control”). • These preexisting differences may account for group differences in the outcome at the end of the experiment.

Nonequivalent Control Group Additive Effects with Selection • selection- history effect: may have different experiences • selection- maturation effect: may mature at different rates • selection-instrumentation effect: be measured more or less sensitively by the instruments • differential subject attrition: may drop out of the study at different rates • differential regression: may differ in terms of regression to the mean

Nonequivalent Control Group • Langer and Rodin (1976) – Residents in the two groups did not differ significantly on the pretest measures – selection-maturation effect • When treatment group is selected not randomly assigned • The natural growth rate “rate of change” of two groups from different populations might be different see figure 10. 4 – Early ages and older ages most vulnerable – When two groups are from different populations • residents on different floors of a nursing facility – Assigned to rooms randomly – selection- history effect • event affecting the residents’ happiness and alertness occurred on one floor of the nursing home but not on the other

Nonequivalent Control Group • Langer and Rodin (1976) – selection-instrumentation effect • changes in a measuring instrument across floors • Floor or ceiling effects differ across floors – differential regression • regression toward the mean is to be expected when individuals are selected on the basis of extreme scores • pretest scores were more extreme across floors – Expectancy Effects, Contamination, and Novelty Effects • Expectancy: all the observers were kept unaware of the research hypothesis • Contamination: “there was not a great deal of communication between floors” • Novelty or Hawthorne effect: Individuals living on each floor were given equal attention and staff expressed caring about them

The issue of external validity • How much do the findings from Langer and Rodin generalized to other elderly residents – Characteristics of the nursing home where the research was completed – Quality of the nursing home – Characteristics of the residence within the nursing home • Amount of independence • Socioeconomic status • Replication of a study can help demonstrate external validity

Quasi-Experimental Designs • Simple Interrupted Time-Series Design – Observe a dependent variable for some time before and after a treatment is introduced – Could be following individuals – Could be from populations using archival data – Maybe difficult to get continuous baseline data O 1 O 2 O 3 O 4 X O 5 O 6 O 7 O 8 – Look for clear discontinuity in the time-series graph for evidence of treatment effectiveness. – Study Natural treatments such as a terrorist attack of September 11, 2001 • Self-reported character strengths see figure 10. 5

– obvious discontinuity in the time series around the 9/11 event indicated an effect on self-reported character strengths

Quasi-Experimental Designs • Simple Interrupted Time-Series Design – Discontinuity in the time series is the major evidence of effective treatment • Usually is a gradual change • Impact of clean-air ordinance to ban smoking in Bowling Green • Hospital admissions due to smoking related diseases over a six-year period • Three years before and three years after the smoking ban • See figure 10. 6

Hospitalizations decreased gradually after implementation of the smoking ban

Simple Interrupted Time-Series Design • Suppose a discontinuity is observed when treatment (X) is introduced. • Rule out threats to internal validity: – history threats are the most troublesome in this design, – Seasonal variation is also a possible problem for interpretation – Can compare to another year at the same season – instrumentation threats also are likely in some studies.

Simple Interrupted Time-Series Design • What threats are more easily ruled out? – Maturation: We assume maturational changes are gradual, not abrupt discontinuities. – Testing: If testing influences responses, these effects are likely to show up in the initial observations (i. e. , before the intervention). Also, testing effects are less likely with archival data. – Regression: If scores regress to the mean, they will do so in the initial observations.

Quasi-Experimental Designs • Time Series with Nonequivalent Control Group Design – Add a comparison group to the simple interrupted time series design: O 1 O 2 O 3 O 4 X O 5 O 6 O 7 O 8 --------------------------------- O 1 O 2 O 3 O 4 O 5 O 6 O 7 O 8

• Include data from hospitalization records from Kent Ohio similar population based on age and gender distribution but without a smoking ban

Example: Study Habits • Suppose a nonequivalent control group is added — these students don’t participate in the study habits course. • Who should be in the comparison group? • What threats are you able to rule out?

The Impact of Television: A Natural Experiment in Three Communities • • Notel – no television Unitel – one TV station Multitel – four TV stations They studied the towns twice: – once before TV came to Notel (Phase 1) – again after it had TV for two years (Phase 2). • Changes in sex roles, cognitive scores, aggressive behavior, leisure activities all larger in Notel then in other towns.

Program Evaluation • Goal: To provide feedback to administrators of human service organizations in order to help them decide what services and to whom they will provide, and how to provide them most effectively and efficiently. • This is a big growth area — particularly in the field of mental health (managed health care). • Program evaluators assess needs, process, outcomes, and efficiency of social services.

Four Questions of Program Evaluation • Needs: Is an agency or organization meeting the needs of the people it serves? (survey designs) • Process: How is a program being implemented (is it going as planned)? (observational designs) • Outcome: Has a program been effective in meeting its stated goals? (experimental, quasiexperimental designs; archival data) • Efficiency: Is a program cost-efficient relative to alternative programs? (experimental, quasiexperimental designs; archival data)

Basic Research and Applied Research • Program evaluation is the most extreme case of applied research — the goal of program evaluation is practical, not theoretical. • The relationship between basic research and applied research is reciprocal: – Basic research provides scientifically based principles about behavior and mental processes. – These principles are applied in the complex, real world. – New complexities are recognized (e. g. , the scientific principles may not always apply in real-world settings) and new hypotheses must be tested in the lab using basic research.

The social and political realities of society’s experiments • Which health care programs work • Medicare has health insurance for people over 65 • Health quality partners program evaluated – Sending a trained nurse for regular home visits • Check patient’s health and medications, answered questions – Reduced hospitalizations by 33% – Cut Medicare costs by 22 % – Medicare decided to drop the program • Decided that the program was not scalable • Did not fit with current model of providing care – because of political and economic realities