Lecture Outline o Introduction to experimental research o

Lecture Outline o Introduction to experimental research o Introduction to correlational research o Three types of research validity o Threats to research validity

Experimental Research Overview and Major Features o o o An independent variable (IV) is manipulated A dependent variable(s) (DV) is measured Many basic experiments consist of two levels of the independent variable o o o Control over extraneous variables o o o experimental group control group holding constant randomizing effects A causal relationship between the independent and dependent variables can be established

Basic Goal of Experimental Design o Manipulate possible values of the IV o Observe behavioral changes as indexed by values of the DV

Basic Characteristics of Experimental Designs o As part of randomizing effects of extraneous variables, subjects are randomly assigned to research groups n A given participant must be equally likely to be assigned to any of the given groups o o o Random number table Flip a coin In experimental designs, a causal relationship between the independent and dependent variables can be established

Independent Variable o o Differing values of the IV are called “levels” May vary quantitatively n n 10 mg day 20 mg day e. g. here are 2 levels of variable dosage o May vary qualitatively Zoloft n Imipramine n Prozac E. g. here are 3 levels of the variable medication type n

Operational Definitions • An operational definition is a clearly defined set of procedures for obtaining a measure of the construct of interest. • The key to an acceptable operational definition is that the procedure is specified precisely enough to allow replication by others. Examples: quality of memory -- accuracy of recall in a certain task depression -- Beck Depression Inventory (survey) score arousal -- galvanic skin response (conductivity of the surface of the skin)

Operational Definitions: Examples He is a good car salesperson Verbal Statement He: 1. Sells many cars 2. Points out + features 3. Is nice 4. Sells only good cars Operational Definition

Operational Definitions: Examples Stress Verbal Statement 1. Environment/Number of stressors a. Number negative Life events b. Number of Examinations 2. Self-report Measurable Nervous mood 3. Behavior a. number of fidgets/minute b. Psychophysiological responses elevated Operational Definition

Identify Key Experimental Features o o Social loafing refers to people’s tendency to slack on group tasks. In one study on social loafing (Latane, Williams and Harkins) college participants cheered alone, and in groups of 2, 4, or 6 ppl. They were told to cheer as loud as possible at specific times so researchers could determine how much noise people made in social settings. The amount of noise made by each participant dropped as group size increased. 1) What is the IV? 2) What is the DV? 3) Why is random assignment important here?

Correlational Research: Major Features o o o No independent variables are manipulated Two or more dependent variables are measured and a relationship established Correlational research does not show causality

When To Use Correlational Research l. Gathering data in early stages of research l. Manipulating an independent variable is: ä Impossible ä Unethical l. Relating > 2 naturally occurring variables

Correlational Research o 2 Variables Positive and Linear Negative and Linear

Example of Non-Linear Relationship Yerkes-Dodson Law Better Performance Worse Low Arousal High

Correlational Research: Major Points o Don’t confuse statistics with research design o o Correlational research simply denotes: o o o Correlation coefficients ( a statistic) can be used in correlational or experimental research designs (although they are more commonly used in correlational designs) an ASSOCIATION or a RELATIONSHIP between variables Correlational research is “silent” about causality

Correlation Does Not Imply Causation Causality – must demonstrate that variance in one variable can only be due to influence of the other variable o Directionality of Effect Problem o Third Variable Problem

Directionality of Effect Problem X Y X Y

Directionality of Effect Problem X Class Y Higher Attendance Higher Grades X Y Higher Class Attendance Higher Grades

Directionality of Effect Problem X Aggressive Behavior Y Viewing Violent TV X Y Viewing Violent TV Aggressive Behavior Viewing Violent TV Aggressive children may prefer violent programs or Violent programs may promote aggressive behavior

Third Variable Problem Y X Z

Class Exercise Identify the third variable that influences both X and Y

Third Variable Problem Class Attendance + GPA

Third Variable Problem Number of Churches + Crime Rate

Third Variable Problem Ice Cream Consumed + Number of Drownings

Measurement Considerations o o Variables should be measured accurately and consistently. Accuracy n Validity o o Bathroom scale gives correct weight Consistency n Reliability o Bathroom scale gives same weight consistently

Evaluating Research: 3 Validities o Validity n o Are we measuring/manipulating what we say we are? Three validities n n n Construct validity Internal validity External validity

Construct Validity o The degree to which the construct can be inferred from the operational definition of that construct n Never one perfect operationalization o Schacter and Singer (1962) n Two-factor theory of emotion: o n Emotions require arousal and label Factor 1: AROUSAL Participants received epinephrine shots and either informed them of side-effects or not o n Factor 2: LABEL Participants around "euphoric" or "angry“ confederate o

How Valid are These Manipulations? o Euphoria n o Confederate shoots rubber bands, plays with hula hoops and crumples up paper and practices hook shots into wastebasket Anger n Confederate complains about the injection, adds negative comments about the questionnaire they are filing out and ends up ripping up the questionnaire.

Simplified Results Informed of epinephrine effects Uninformed of epinephrine effects Euphoria Condition Anger Condition

Internal and External Validity o Internal validity n The extent to which the observed effect is caused only by the experimental treatment condition The ability to draw conclusions about a causal relationship from our data o Experiments usually high in this o o External validity n The extent to which the results can be applied to and across different persons, settings and times AKA generalizability o Experiments often low in this o

Generalizing From the Lab Setting o Mundane realism n o Psychological realism n o The extent to which an experiment is similar to real-life situations The extent to which an experiment triggers relevant psychological and group processes Key point n n Lab experiments not automatically artificial When an experiment lacks mundane realism it may be very realistic in terms of psychological realism

Assess Internal & External Validity o Does lighting affect productivity? o Study 1: n Give people a production task in a laboratory. The lab room is either bright or dim. Measure and compare their productivity (how quickly and accurately they perform the task). Study 2: n Give employees in a well-lit office and those in a dim office a production task. Measure and compare their productivity (how quickly and accurately they perform the task). o

The Tradeoff o Real life situations n n High external validity Hard to rule out other explanations o o Low internal validity Lab settings n n High internal validity Often artificial and findings can’t be generalized o Low external validity

Overview: Threats to Validity o o 1) People are Different 2) People Change 3) Process of Studying People Changes Them 4) Variables with Treatment Change People

1) People are Different: Individual Differences o o o People are different Can’t determine if outcome is due to IV or to individual differences Pitfall of one group experiments n Pseudo-experiments o No control group o Threat to internal validity

1) People are Different: Selection Bias/ Nonresponse Bias o Selection n Choosing participants from a nonrepresentative sample o Threat to external validity o President Alf Landon? o Nonresponse bias n When a substantial proportion of those invited to participate refuse to do so o Threat to external validity

2) People Change History o Events occurring between first and second measurements but not part of the manipulation. DV: Pretest Subjects Smoking Measure w. Pitfall of one group pretest-posttest design w. Threat to internal validity Independent Variable Training Program DV: Posttest Smoking Measure

2) People Change Maturation o People change over time n Grow older wiser, stronger , healthier, more tired, more bored… Third graders Math Exam Training Program Math Exam w. Pitfall of one group pretest-posttest design w. Threat to internal validity

3) People Change Regression Toward The Mean o o AKA statistical regression Extreme scores (high or low) change over time (regress to the mean) n n High scores become lower Low scores become higher Hex of the Sports Illustrated cover o Pitfall of one group pretest-posttest design o

Does Studying Something Change it? o o "The more precisely the POSITION is determined, the less precisely the MOMENTUM is known" The Hawthorne effect n An increase in worker productivity produced by the obtrusive observation of that process

3) Studying People Changes People Testing Effects o Simply taking the pretest changes the participant! n n May cue people in on what is being studied Make them more adept at a skill Students Science Exam Science Program w. Pitfall of one group pretest-posttest design w. Threat to internal validity Science Exam

3) Studying People Changes People Mortality (Attrition) o o No, this isn’t that people in your study are dying Participants drop out of the study before completing it Couples Relationship Status Marriage Counseling Relationship Status w. Homogeneous attrition Attrition rates equal across experimental conditions n. Threat to external validity n w. Heterogeneous attrition Attrition rates different across experimental conditions n

3) Studying People Changes People Participant Reaction Bias o Research participants realize they are being studied and behave in a way they normally would not n n n o Good subject role Participant reactance Evaluation apprehension Threats to internal validity

Applicant Hiring Experiment o INSTRUCTIONS n n o This study assesses job applicants capacity to handle stress You will be interviewing an unemployed person Your task is to read questions to the job applicant and then give negative responses After the first eight questions you should read the 'Stress Remarks' SCENARIO n n n The applicant appears well-qualified and experienced The answers to the first eight questions are intelligent, lucid and to the point. Stress Remark: o "This job is much too difficult for you according to the test".

Jane o Jane remembered the mix of male and female participants in the waiting area and noted the fact that the researcher was male. She decided that the experimenter had predicted that female subjects will be less likely to make the stress remarks and she so simply ignored them and said "well done you are doing well, aren't you!" in order to help the researcher's project.

Good-Subject Role o Participants consciously/unconsciously try to behave consistent with experimenters hypothesis o Demand characteristics n Feature of an experiment that may inform participants of the purpose of the study

Su o Su remembered the mix of male and female participants in the waiting area and noted the fact that the researcher was male. She also decided that the experimenter has predicted that female subjects will be less likely to make the stress remarks and made a point of not only making the stress remarks, but saying it in an aggressive manner - in order to disabuse the researcher of female stereotypes

Participant Reactance o Participants try to disconfirm perceived hypotheses

John o John looked at the highly professional instruction cards and the expensive video recording equipment and decided this was a test of management skills. So, he attempted to stay calm and read out the questions in a controlled, professional manner in order to show the experimenter that he was made of stern stuff and competent at handling difficult interpersonal situations. However, he was so nervous about doing the task well that he mixed up the order of questions and read them out too quickly.

Evaluation Apprehension o Participants try to behave in a way that will portray them most favorably.

Controlling for Demand Characteristics o Keep participants in the dark n Cover stories o Schacter and Singer’s emotion study n Unobtrusive observations o One-way mirror o Hot sauce o Radio stations o Ensure anonymity

4) Treatment Changes People Experimenter Bias o o o AKA expectancy effects Experimenter expectations biasing results Two sources of experimenter bias n n o o Unintentionally treat participants differently Experimenters record the behaviors differently Threat to internal validity The story of Clever Hans

Expectancy Effects o Rosenthal and Jacobson (1966) n n n Students IQ tested Told teachers some students were going to be “intellectual bloomers” These students actually chosen at random At end of school year their IQ tested again Results o n “Ready to bloomers” saw marked improvements Self-fulfilling prophecy Teacher expectations are communicated to students who in turn respond to those cues by adjusting their behavior. o The result: the original expectation becomes true. o

Solution to Expectancy Effects o o 1) Practice makes perfect (or nearly so) 2) Run all participants in one fell swoop 3) Go high tech! 4) Ignorance can be bliss o Unaware of hypotheses o Unaware of condition n n Single blind Double blind

4) Treatment Changes People Confounds o Some additional variable varies systematically with the independent variable o Really, all threats we have discussed the problem was that something else was accompanying the independent variable.