Independent Groups 1 Random groups design Methods in

Independent Groups 1: Random groups design Methods in Psychological Science Spring 2020 Dr Chapman

Why Psychologists Conduct Experiments • Multimethod approach – Compliments observational and survey research – Seek convergent validity for research findings across methods • Third goal of psychological research – Explanation • Examine the causes of behavior • Test – Hypotheses derived from theories – Effectiveness of treatments and programs

Experimental Research • A “true” experiment must include – Independent variable (IV) – Dependent variable (DV) • Independent Variable – Manipulated (controlled) by experimenter – At least 2 conditions (levels) • “Treatment” and “Control” • Dependent Variables – Measured by experimenter – Used to determine effect of IV • Typically researchers measure several dependent variables to assess effect of IV.

Experimental Control and Internal Validity • Internal validity – are able to confidently state that the independent variable caused differences between groups on the dependent variable (i. e. , a causal inference). – must be able to rule out alternative explanations – For example: Pennebaker (1989) “inhibition theory” • students who write about emotional experiences – are healthier – more academically successful • compared to students who write about superficial experiences • How do we know the emotional writing caused them to be healthier and academically successful (i. e. , a causal inference? ) • What are some alternative explanations? – How does this differ from external validity?

Internal Validity • Differences in performance (DV) can be attributed unambiguously to effect of independent variable (IV) • Three conditions to make a causal inference: – Covariation: • must observe a relationship between the independent and dependent variables. • participants who write about emotional events have better health and academic outcomes than participants who write about superficial events – Time-order relationship: • cause precedes the effect • writing about emotional events comes before the beneficial health and academic outcomes – Elimination of plausible alternative causes: • Using control techniques to rule out other possible causes for the outcome.

Causal Inferences : Time-Order Relationship – Manipulation of IV can establish the time-order relationship between the IV and the DV – Establishing a time-order relationship can be tricky. – If participants in the emotional writing condition • were healthier before writing • more academically successful before writing – Then the effect precedes the cause – Need to assume the groups are equal before experimental manipulation is given – How can groups be made equal prior to manipulation?

Causal Inferences : Eliminating Alternatives • Elimination of plausible Reasonable Alternatives – Only alternatives that fit the research hypothesis • Confounding variables: – When the independent variable of interest and a different, potential independent variable are allowed to covary (go together), a confounding is present. • For example students who write about emotional experiences are also involved in an exercise program – Confounding variables represent alternative explanations for a study’s findings. – An experiment that has a confounding is not internally valid. – An experiment that is free of confoundings has internal validity.

Control Techniques • To eliminate alternative explanations – Holding conditions constant • • all other possible experiences to be constant across the groups differ only in terms of the independent variable same as not having any confoundings in the experiment for example, what did Pennebaker and Francis do? – Balancing • characteristics of participants cannot be held constant • for example, some participants are healthier or smarter • on average, the participants in each condition are essentially the same before the experiment begins • the groups in the experiment are equally healthy, smart, motivated, conscientious, etc. prior to the independent variable manipulation • Proper use of control techniques increases internal validity

Independent Groups Designs(Between-Subjects) • Different individuals participate in each condition of the experiment. – No overlap of participants across conditions • Three types based on Techniques for Balancing – Random groups design – Matched groups design – Natural groups design

Random Groups Designs • Individuals are randomly assigned to conditions “levels” of the IV. – Need at least two conditions but there can be many more. • Logic of causal inference – When groups are equivalent at the beginning of an experiment (through balancing) – conditions are held constant – any differences among groups on dependent variable are caused by the manipulated independent variable.

Example: Body Image Among Young Girls (Dittmar, Halliwell, & Ives, 2006) • Research prediction – Young girls exposed to a very thin body image will experience greater body dissatisfaction than young girls who are exposed to realistic or neutral body images. • IV: version of picture book with 3 levels • Barbie (very thin body image) • Emme (realistic body image) • Neutral (no body images)

Body Image Among Young Girls, cont. • Dependent Variables – Several measures of body image and body dissatisfaction – Child Figure Rating Scale • Rate perceived actual body shape • Rate ideal body shape • Obtain difference score – Negative score: desire to be thinner

Control Techniques • Manipulation – IV: participants in the three conditions have different experiences • Three groups of participants (hence between-groups) – Group 1 Barbie – Group 2 Emme – Group 3 neutral images • Holding conditions constant – IV is only factor that differs systematically across groups – Dittmar et al. (2006) held constant • All girls listened to same instructions and story • All completed the same questions after the story

Control Techniques, continued • Balancing – Random assignment of participants to conditions balances subject characteristics, on average. • Assign 20 girls to each of the conditions • So a total of 60 girls in the study – Groups are equivalent prior to IV manipulation. – All subject variables are balanced. • Body weight, number of Barbie dolls, preexisting levels of body dissatisfaction, etc.

Block Randomization • Block Randomization is a technique used to assign participants randomly to conditions. – A “block” is a random order of all conditions in the experiment. • For example, three conditions (A, B, and C) might be randomly ordered as B C A. – The first participant would be in condition B, the second in condition C, and the third in Condition A. – Generate another random order of the block (e. g. A C B). – Assign the next 3 participants to conditions A, C, and B, respectively. – Continue until the goal for number of participants in each condition are met (e. g. , 20 participants in each condition).

Block Randomization with five levels “conditions” of an independent variable

Random Groups Designs, continued • Block randomization • Advantages of block randomization: – Creates groups of equal size for each condition – Controls for time-related events that occur during the course of an experiment: • Natural changes in experimental conditions, experimenters, and participants that occur over time are balanced across the conditions of the experiment. • Some experiments may take several months to complete – As with all random assignments, block randomization balances subject characteristics across the conditions of the experiment.

Threats to Internal Validity • The internal validity of an experiment – the ability to make a causal inference about the effect of an independent variable on a dependent variable – is threatened when: • intact groups are used, • extraneous variables are not controlled, • selective subject loss occurs, and • demand characteristics and experimenter effects are not controlled.

Threats to Internal Validity • Intact Groups – Intact groups are formed prior to the start of an experiment • Examples: – Different sections of introductory psychology » why are students in the 8: 30 vs 10: 30 section – children in different classrooms – different departments within an organization – Individuals are not randomly assigned to intact groups. As a result, individual differences among groups threaten the validity of the experiment. – When groups, rather than individuals, are randomly assigned to conditions, subject characteristics are not balanced.

Threats to Internal Validity • Extraneous Variables – Practical considerations when conducting an experiment may confound an experiment — these are referred to a extraneous variables. • Examples: number of participants in each session, different experimenters, different rooms in which an experiment is conducted – Extraneous variables are controlled using balancing or holding conditions constant. • Balancing occurs when the two experimenters conduct each condition but are randomly assigned to administer a condition at any particular time. • Holding conditions constant would involve using only one of the experimenters. This experimenter would test participants in both the treatment and control conditions.

Threats to Internal Validity • Subject Loss (Attrition) – Subject loss occurs when participants fail to complete an experiment. – Selective subject loss occurs when (1) Participants are lost differentially across the conditions of the experiment (e. g. , more participants are lost in the treatment condition compared to the control condition), for example exercise to reduce weight (2) When some characteristic of the participant is responsible for the loss (e. g. , subject characteristics such as personality, intelligence, physical characteristics), and (3) The subject characteristic is related to the dependent variable in the experiment. – This challenges the internal validity of an experiment if the equivalent groups formed at the beginning of an experiment are no longer equivalent at the end of the experiment because of attrition.

Threats to Internal Validity – Subject Loss (Attrition) – Example: Suppose a treatment for depression is compared to a control condition (no treatment). • Selective subject loss might occur if participants are more likely to drop out of the control condition than the treatment condition (or vice versa). • Any differences in depression symptoms (the dependent variable) at the end of the experiment might be due to the treatment or to selective subject loss in the experiment. – Mechanical subject loss occurs when equipment failure or experimenter error results in a participant’s inability to complete the experiment. • Often, mechanical subject loss occurs due to chance factors and is likely to occur equally across the conditions of an experiment.

Threats to Internal Validity • Demand characteristics are the cues and other information that participants use to guide their behavior in a psychological study. Example: In a drug-treatment study, demand characteristics suggest to participants that they will improve as a result of the drug. • Thus, participants may expect to improve when given the treatment. • These expectancies (rather than the drug) may cause participants to improve. • A placebo control group is used to assess whether participants’ expectancies contribute to the outcome of an experiment. – Participants who receive a placebo may believe they are receiving an effective drug treatment; this controls for their expectations regarding the effectiveness of treatment. – If participants who receive the actual drug improve to a greater extent than participants who receive a placebo, we can be more confident that the drug produced the beneficial outcomes rather than participants’ expectancies.

Threats to Internal Validity • Experimenter effects – refer to potential biases that occur when experimenters’ expectancies regarding the outcome of the experiment influence their behavior toward participants in different conditions. • can be controlled by keeping experimenters and observers unaware of the hypotheses or expected results • this is referred to as keeping experimenters “blind. ” • A double-blind experiment – refers to procedures in which both the participants and the experimenters/observers are unaware of which condition is being administered. • control for both demand characteristics and experimenter effects. • allow researchers to rule out participants’ and experimenters’ expectancies as alternative explanations for a study’s outcome.

Establishing the External Validity of Experiments • Conclusions from an experiment – Can they be generalized beyond a specific experiment to other • Individuals • Settings • Conditions – Any individual experiment has limited external validity because • Control from holding constant increases internal validity but can reduce the external validity • However reduced external validity is OK when there is good internal validity

External Validity of Experiments • For Example: Research with college student on cognitive mechanisms of decision making – Would this produce low external validity? – Are college students all that different from other people? – The answer to this question likely depends on the research question.

Establishing external validity of experiments – Generalize beyond the scope of a specific experiment – Example: Do insults provoke aggression • Across several experiments include characteristics of the – Situations: in private, in public – Settings: in classroom, at a bar – Populations: young children, adolescents, adults – Compare findings from a series of experiments • Do different situations, settings or population produce the similar results? • The goals is not to have external validity but to identify the extend of external validity • We do not expect some behaviors such as aggression to be similar across all situations, settings or populations. • This the partial replication approach

Establishing External Validity of Experiments • External validity of findings are established when the findings are replicated • Field experiments are a way to increase the external validity of laboratory findings by replicating an experiment in a real-world setting. • Partial replications are common: Research findings generalize when a similar result occurs when slightly different experimental procedures are used in a subsequent experiment. • Psychologists also make use of conceptual replications. – We are more interested in relationships among variables considered at the conceptual level rather than in specific conditions, settings, and participants. – Different operational definitions for concepts may be used in replications to fit the particular population or setting.

Additional Independent Groups Designs • Random assignment requires large samples (10 -15 per group) to balance participants’ characteristics across the conditions of the experiment. • When only small samples are available, a researcher may choose the matched groups design. – select the one or two individual differences variables related to the research question for matching. – participants should be matched on • dependent variables • or variables very similar to the dependent variable • matching variables are the characteristics that definitely should be equivalent before the experiment • matched pairs of participants are randomly assigned to conditions

Matched Groups Design • Example: Effects of a relaxation treatment for high blood pressure – Independent Variable: Treatment vs. Placebo – Dependent Variable: Blood pressure – Suppose you don’t have enough participants for random assignment to create equivalent groups (only 16 participants). – What would you want to match? That is, what feature or characteristic of participants in the two groups should be equivalent before the experiment begins? • Blood pressure. – Why blood pressure? • Suppose that participants in the treatment group had lower blood pressure before they start the experiment. • Any differences in blood pressure (DV) between the treatment and the placebo group at the end of the experiment might be due to the different blood pressures at the beginning of the experiment.

Matching Participants: Example Name Beginning Blood Pressure Susan Janet Philip Barbara John Benjamin Catherine William 160 / 110 150 / 120 160 / 110 180 / 130 170 / 120 150 / 120 170 / 120 Assignment to conditions: Treatment matched to Philip Janet Barbara William Control Susan Catherine John Benjamin Average Blood Pressure: Treatment Control 165 / 120

Matched Groups Design • Important points about matching: – participants are matched only on some variables – the groups may differ on other important variables. • These differences may be alternative explanations for the study’s results. – Can use more then one variable for matching – However, the more characteristics you try to match on, the harder it will be to match participants.

Natural Groups Designs • Psychologists’ research questions often ask how individuals differ, and how these individual differences are related to important outcomes. • Individual differences variables (subject variables) refer to characteristics or traits that vary across individuals. Examples: – Physical Characteristics: sex, race – Social (Demographic) Characteristics: ethnicity, religious affiliation, marital status – Personality Characteristics: extraversion, emotional stability, intelligence – Mental Health Characteristics: depression, anxiety, substance abuse, disordered thinking

Natural Groups Designs • For example, – do men and women differ in what they seek in intimate relationships? • Researchers can not randomly assign participants to these groups (male/female). • When a researcher investigates an independent variable in which the groups (conditions) are formed naturally, we say a “natural groups design” is used.

Natural Groups Designs • For example, Suppose we want to compare occupational functioning of schizophrenics and normal controls (non-schizophrenics). • What is the independent variable? Natural groups variable: schizophrenic vs. non-schizophrenic participants • What is the dependent variable? Occupational functioning

Natural Groups Designs • Causal inferences and natural groups designs: – Researchers can’t make a causal inference when a natural groups design is used. • For example, suppose schizophrenics have poorer occupational functioning than the normal participants. – Can we say that schizophrenia caused this poorer occupational functioning? – What part of causal inference is missing ?