Research Design Designing Your Experiment Setting the Stage

Research Design

Designing Your Experiment Setting the Stage Type of Manipulation Straightforward Manipulation Staged or Event Manipulation 1. to create a psychological state in the participants 2. to simulate some situation that occurs in real life frequently employs confederates Strength of the Manipulation Cost of the Manipulation

Experimental Group Design There are many ways to design an experiment. Here are three of the most common group designs: Independent or Between-Subjects Groups Design Repeated Measures or Within-Subjects Groups Design Mixed-Subjects Group Design

Independent or Between-Subjects Requirements: Each level of each independent variable has different subjects. participants are randomly assigned to the different groups random assignment prevents any systematic biases Advantages: minimizes order effects minimizes demand characteristics is the easiest to statistically analyze. Drawbacks: requires many subjects less statistical power takes more time Control Experimental S 01 S 02 S 03 S 04 S 05 S 06 S 07 S 08 S 09 S 10 S 11 S 12

Order Effects order of presenting the treatments affects the dependent variable Practice Effect improvement in performance with repetition Fatigue Effect decrease in performance with repetition Contrast Effect second response is altered due to the influence of the first Counterbalancing is used to correct order effects:

Counterbalancing helps to minimize order effects • Complete Counterbalancing • Randomized Block Design • Matched Pairs Design • Latin Squares

Repeated Measures or Within-Subjects Requirements: Each subject participates in all levels of all independent variables Advantages: uses fewer subjects than the Between-Subjects Design greater statistical power/ lower score variability takes less time minimizes subject effects Drawbacks: is difficult to control order effects increases demand characteristics Control Experimental S 01 S 02 S 03 S 04 S 05 S 06

Mixed-Subjects Requirements: must be at least two independent variables Each subject participates in all levels of just one independent variable Advantages: uses fewer subjects than the Between-Subjects controls for order effects controls for subject effects Drawbacks: is the most difficult to analyze statistically Control Experimental Control S 01 S 02 S 03 S 04 S 05 S 06 Experimental S 07 S 08 S 09 S 10 S 11 S 12

Factorial Designs Factorial designs have more than one independent variable or factor e. g. , 2 x 2, 2 x 3, 2 x 2, IV by participant variables - IV x PV designs Main Effects: the effect of each independent variable by itself Interaction and/or Moderator Variables: Interaction: when one IV level affects the DV differently than another level Moderator Variable: a particular situation or participant characteristic Outcome of a 2 X 2 Factorial Design: 1. Main Effect A significance/non-significance post-hoc analysis (across individual levels) 2. Main Effect B significance/non-significance post-hoc analysis (across individual levels) 3. Interaction significance/non-significance if interaction is significant you must reevaluate the main effects

Interaction The interaction is the effect of the combination of the two independent variables on the dependent variable. It is best seen by graphing the means of all levels of both factors. If the two lines are parallel then the interaction is typically nonsignificant. If the lines are not parallel as in the figure below, it means that there is an interaction between the two independent variables that is having an effect on the dependent variable.

Life Span Development Designs

Life Span Development Designs Weaknesses and Strengths

Non-Experimental Research Methods • Surveys • Descriptive Research Methods • Correlation

Surveys • are a common and important method of studying behavior • provide us with a tool for asking people about themselves • their attitudes and beliefs • Past or intended future behaviors • demographics or other facts • provide a snapshot of how people think at some point in time • are used to determine relationships between variables • can measure how attitudes and behaviors change over time • can serve as a compliment to experimental research

Survey Problems • Response Set • tendency to respond to all questions from one perspective • can affect the usefulness of survey data • social desirability - "faking good" • answers in the most socially acceptable way • big problem with sensitive questions • False Answers • respondents will lie if they do not trust the experimenter • must give full disclosure and maintain confidentiality

Research Objectives • Attitudes and Beliefs • The way people evaluate and think about issues • Facts and Demographics • Things they know about themselves and their situation • Behaviors • Focus on past behaviors or future intended behaviors

Question Wording • Simplicity • Avoid jargon and technical terminology • Double-Barreled Questions • Avoid questions that ask two things at once • Loaded Questions • No leading or emotionally charged questions • Negative Wording • Avoid phrasing questions with negatives • “Yea-Saying” and “Nay-Saying” • Don’t’ ask similar questions that might get similar answers

Question Types • Closed- Versus Open- Ended Questions • Number of Response Alternatives • Rating Scales • Labeling Response Alternatives

Closed- Vs. Open- Ended • Closed-Ended Questions • • Fixed number of response alternatives More structured approach Easier to code Used when the dimensions of the variable are well-defined • Open-Ended Questions • • Respondents are free to answer any way they like Requires time to categorize and code More costly Can yield valuable insights into what people are thinking

Number of Response Alternatives • 5 -7 point scale is preferable On a scale of 1 (low) – 7 (high), how would you rate your happiness? ______ strongly agree ___ ___ strongly disagree

Rating Scales • Graphing Rating Scale strongly agree ______________ strongly disagree • Semantic Differential Scale evaluation: good ___ ___ bad activity: strong ___ ___ weak potency: active ___ ___ passive • Non-Verbal Scale for Children

Labeling Response Alternatives Hi-Frequency Lo-Frequency

Formatting Your survey should: 1. appear attractive and professional 2. be neatly typed 3. be free of spelling and grammatical errors 4. have easy to identify questions and answers 5. leave space between questions 6. keep the scale format consistent 7. order the sequence of the questions carefully 8. ask the most interesting question first 9. capture the respondents attention 10. motivate the respondent to answer all of the questions 11. group similar theme questions together 12. be tested on a small sample first

Questionnaires • Personal Administration to Groups or Individuals • Have a captive audience • Researcher is available for questions • Mail Surveys • Fairly inexpensive • Has a potential for low responders • Internet Surveys • Inexpensive and growing • Problem confirming who is responding • Other Technologies • Cell phones, pagers, PDA’s

Interviews • Face-to-Face • Expensive and time consuming • Telephone • Less expensive • Data is collected quickly • Focus Group • • Group interview with 6 -10 for 2 -3 hours Open ended questions with group interaction Must facilitate communication and quell problems Costly and time consuming • Panel Study • Studies changes over time • Same people surveyed at two or more points in time

Descriptive Research Methods Behavior is observed and measured as it occurs naturally • Case Study • Naturalistic Observation • Systematic Observation • Archival Research

Limitations 1. Direction of cause and effect can not determine cause and effect 2. The Third-Variable Problem some other variable may be driving the relationship confounding variables

Case Studies provides a description of an individual: • person • business • school • neighborhood “Anna O”

Psychobiography • Case study research that applies psychological theory to explain the life of an individual • Usually investigates some important historical figure: Political figures Literary/Artistic figures Sports/Hollywood Notables • Uses library research and phone interviews • No direct observation at all

Naturalistic Observations are made: in a particular setting and, over an extended period of time A variety of techniques to collect information are used Final report includes: the observations and, the researcher's interpretations of the findings Helps us to understand how people in a social/cultural setting: live, work, and, experience the setting

Data Collection Naturalistic observation researchers must immerse themselves in the situation and observe: • the setting • the patterns of personal relationships • people's reactions to events • other environmental factors must keep detailed field notes must write or dictate at least once a day everything that has happened • Goal 1: describe setting, events, and persons observed • Goal 2: analyze what was observed analysis needs to be supported by multiple confirmations must provide a complete and accurate picture no hypotheses testing

Participation and Concealment 1. Non-Participant Observer is an outsider who does not become an active part of the setting could miss key interactions 2. Participant Observer does become a part of the setting could lose objectivity could affect behavior (reactivity) 3. Experimenter Bias must leave biases at home carefully assess the observers role

Defining the Scope of the Observation Naturalistic observation researchers may have to limit the scope of study to focus on central issues Too large a scope: risk being overwhelmed could miss more subtle effects Too small a scope: risk missing important data could miss larger more important interactions

Limitations to Naturalistic Observation 1. Do not start with a well-defined hypothesis can affect the direction of data collection 2. Very difficult to conduct requires a great amount of energy ever changing pattern of events - some important, some not must record them all must be flexible to adjust recording as research progresses 3. Identify inconsistencies additional research may be needed negative case analysis an observation that does not fit the explanation

Systematic Observations • Careful observation of: one or more specific behaviors in a particular setting • Interest is in only a few specific behaviors • Observations are quantifiable • Researcher begin with prior hypotheses about the behaviors

Coding Systems • Are used to quantify behavior • Should be simple as possible • Should allow observers to easily categorize behaviors • Should use coding systems devised by others system has already been proven useful training materials are usually available

Methodological Issues 1. Equipment videotaping is becoming very common 2. Reactivity the possibility that the presence of the observer will affect people's behavior 3. Reliability the degree to which a measurement reflects a true score rather than measurement error reliable measures are stable, consistent, and precise 4. Sampling samples of behavior taken over a long period provide more accurate measure of behavior

Archival Research Uses previously compiled information systematic analysis of existing documents requires research to devise coding systems that raters can use to quantify the information in the documents Researcher does not collect any original data There are three types: 1. Statistical Records 2. Survey Archives 3. Written and Mass Communication Records

Statistical Records Archival researchers use existing statistical records that are on file There are many sources: US Census APA MLB Public Records marriage license applications

Survey Archives Data from surveys stored in large databases Available to any researcher who wants them Internet has allowed more availability

Written and Mass Communication Written Communication diaries historical letters public documents speeches Mass Communication books magazine articles movies television programs newspapers

Limitations of Archival Research Desired records may be difficult to obtain Can never be completely sure of the accuracy of the data collected by someone else.

Correlation • A Correlation: • is a statistical test. • demonstrates the relationship between two variables within the same participant. • can determine significant relationships between variables. • does NOT show a causal relationship between the variables. • is best represented by a Scatterplot. • is determined by calculating the correlational coefficient.

Scatterplot • Graphical representation of the two sets of data you are comparing. • Shows the strength and direction of the relationship.

Correlational Coefficient

Pearson Product Moment Correlation n = number of pairs Sigma X = the sum of all the X variable values Sigma Y = the sum of all the Y variable values Sigma XY = the sum of all the X times Y values Sigma X 2 = the sum of all the X 2 values Sigma Y 2 = the sum of all the Y 2 values

Calculating a Correlation X Y X 2 Y 2 XY 10 104 100 10816 1040 12 100 144 10000 1200 19 98 361 9604 1862 4 150 16 22500 600 25 75 625 5625 1875 15 105 225 11025 1575 21 82 441 6724 1722 7 133 49 17689 931 113 847 1961 93983 10805 n=8

Neutral Correlation

Positive Correlation

Correlation: Quiz/Exam

Negative Correlation

Correlation: Days Absent/Exam I

Correlation is Not Causation