Statistical Analysis of Factorial Designs z Review of

Statistical Analysis of Factorial Designs z Review of Interactions z Kinds of Factorial Designs z Causal Interpretability of Factorial Designs z The F-tests of a Factorial ANOVA z Using LSD to describe the pattern of an interaction

Review #1 -- interaction Task Presentation Paper Computer Hard 90 40 > 70 > Task Difficulty Easy 60 1) Put in a <, > or = to indicate the pattern of the simple effect of Task Presentation for Easy Tasks 2) Put in a <, > or = to indicate the pattern of the simple effect of Task Presentation for Hard Tasks 3) Is there an interaction? Is the simple effect of Task Presentation the same for Easy and for Hard Tasks ? ? ? 4) Describe the pattern of the interaction. . .

Review #1 -- 1 st main effects Task Presentation Paper Computer Hard > 70 40 > Task Difficulty Easy 90 60 65 = 65 1) Compute the marginal means for Task Presentation. 2) Put in a <, > or = to indicate the pattern of the main effect of Task Presentation 3) Is there a main effect of Task Presentation ? ? ? 4) Is the pattern of the main effect for Task Presentation descriptive or potentially misleading (is the pattern of the main effect or Task Presentation the same as the patterns of the simple main effects of Task Presentations for both Easy and Hard Tasks) ? 5) Describe the main effect

Review #1 -- 2 nd main effects 1) Compute the marginal means for Task Difficulty Task Presentation Paper Computer 80 > > Hard 70 > Task Difficulty Easy 90 40 60 50 2) Put in a <, > or = to indicate the pattern of the main effect of Task Difficulty 3) Is there a main effect of Task Difficulty ? ? ? 4) Is the pattern of the main effect of Task Difficulty descriptive or potentially misleading (is the pattern of the main effect or Task Difficulty the same as the patterns of the simple main effects of Task Difficulty for both Computer and Paper Presentations) ? 5) Describe the main effect

“Kinds” of 2 -factor Designs BETWEEN GROUPS FACTORIAL DESIGN: • each IV uses a between groups comparison • each participant completes only one condition of the design WITHIN-GROUPS FACTORIAL DESIGN: • each IV uses a within-groups comparison • each participant completes all conditions of the design MIXED FACTORIAL DESIGN: • one IVs uses a between groups comparison and the other IV uses a within-groups comparison. • each participant completes both conditions of the withingroups IV, but completes only one condition of the between groups IV. • it is important to specify which IV is “BG” and which is “WG”

Between groups factorial design experimental or natural grps designs used to study “differences” Each participant is in only one condition, having a particular combination of Initial Diagnosis and Type of Treatment Initial Diagnosis Individual Therapy Group Therapy Depression Clients diagnosed as depressed who are treated with individual therapy Clients diagnosed as depressed who are treated with group therapy Social Anxiety Clients diagnosed with social anxiety who are treated with individual therapy Clients diagnosed with social anxiety who are treated with group therapy

Mixed group factorial design natural groups designs used to study “different changes” or “changing differences” Species was a between groups IV (a turtle can only be a member of one species). Each turtle participated in both the mid-morning & dusk conditions of the Time of Day IV. Species of Turtle Time of Day Snapping Turtle Painted Turtle Mid-morning Each snapping turtle completed a trial during mid-morning Each painted turtle completed a trial during mid-morning Evening Each snapping turtle completed a trial during the evening Each painted turtle completed a trial during the evening

Mixed group factorial design experimental designs used to increase data collection efficiency or statistical power Type of Evidence was a between groups IV -- people can’t read the “same study” twice & give independent ratings. Each participant rated the guilt of both Defendants -- a within-groups IV --as they would in this type of case. Type of Evidence Defendant DNA Eye Witness Major Actor DNA evidence was presented against the major actor An eye witness testified to seeing the major commit the crime Conspirator DNA evidence was presented against the conspirator An eye witness testified to seeing the conspirator commit the crime

Within-groups factorial design – experimental designs used to increase data collection efficiency or statistical power Each participant completed four trials, one of each combination of Retention Interval and Word Type. Retention Interval Word Type Immediate Test Delayed Test Familiar The test was given immediately after the study of a list of 40 familiar words. The test was given 5 minutes after the study of a list of 40 familiar words. Unfamiliar The test was given immediately after the study of a list of 40 unfamiliar words. The test was given 5 minutes after the study of a list of 40 unfamiliar words.

Within-groups factorial design – natural designs to study “changing changes” Each participant was observed in both School & Home (WG) settings both when they were 12 &16 (WG) Age Setting 12 years old 16 years old School Participants were observed in a school setting at age 12. Participants were observed in a school setting at age 16. Home Participants were observed in a home setting at age 12. Participants were observed in a home setting at age 16.

Practice Identifying Types of Factorial Designs - answers next page The purpose of the study was to examine the possible influence of two variables upon maze-learning by rats, length of the maze (either 10 feet or 30 feet) and the size of the reward (either 1 sugar pellet or 5 sugar pellets). Here are three “versions” of the study tell which is BG, WG & MG a. Each rat completed one trial. Each was assigned to either the longer or the shorter maze, and also assigned to receive either 1 or 5 sugar pellets upon completing the maze. b. Each rat completed two trials in either the longer or the shorter maze. Following one trial in the assigned maze, each received 1 pellet reward, after the other trial they received the 5 pellets. c. Each rat completed four trials, two in the shorter maze and two in the longer maze. Each received 1 pellet after one of the short-maze trials and 5 pellets after the other, and also 1 pellet after one of the long-maze trials and 5 pellets after the BG MG WG

Another Example -- 3 versions of the same study The researcher wanted to investigate infant's startle responses to loud sounds. The two variables of interest were the Position of the Sound (in front of versus behind the infant) and the Type of Sound (a hand-clap versus deep male voice saying "Hey"). Here are three “versions” of the study tell which is BG, WG & MG a. Each infant completed trials all involving a hand-clap or MG all involving the voice saying "Hey". During some of the trials, the appropriate type of sound was made in front of the infant. During other trials, the appropriate type of sound was made behind the infant. WG b. Each infant had some trials during which the sound was made in front of then and some during which the sound was made behind them. Some of the sounds were the hand-clap and the others were the voice saying "Hey". c. Each infant always heard either the hand-clap or the “Hey”, and whatever sound they heard was always played either in front of them or behind them. BG

Remember about the causal interpretation of effects of a factorial design Start by assessing the causal interpretability of each main effect Remember, in order to causally interpret an interaction, you must be able to casually interpret BOTH main effects. For each of the following: Tell the IVs and tell what effects could be causally interpreted (assuming proper RA, IV manip. and confound control were used): 1. Male and female participants who were African American, Mexican American, or European American were asked to complete a questionnaire about satisfaction with their Senators. zilcho-causo 2. Children played with either a toy gun, a toy car or a puzzle, some while their parents were in the room and some not. The DV was the amount of aggressive behavior they exhibited. 3. Participants played with either a simple puzzle or a complex puzzle in pairs made up of two boys, two girls or one boy & one girl. All three ! Puzzle type only.

Statistical Analysis of 2 x 2 Factorial Designs Like a description of the results based upon inspection of the means, formal statistical analyses of factorial designs has five basic steps: 1. Tell IVs and DV 2. Present data in table or figure 3. Determine if the interaction is significant • if it is, describe it in terms of one of the sets of simple effects. 4. Determine whether or not the first main effect is significant • if it is, describe it • determine if that main effect is descriptive or misleading 5. Determine whether or not the second main effect is significant • if it is, describe it • determine if that main effect is descriptive or misleading

Interpreting Factorial Effects Important things to remember: • main effects and the interaction are 3 separate effects each must be separately interpreted -- three parts to the “story” • most common error -- “interaction is different main effects” • best thing -- be sure to carefully separate three parts of the story and tell each completely • Be careful of “causal words” when interpreting main effects and interactions (only use when really appropriate). • caused, effected influenced, produced, changed …. • Consider more than the “significance” • consider effect sizes, confidence intervals, etc. when describing the results

Statistical Analysis of a 2 x 2 Design Task Presentation (a) Paper Computer SE of Presentation for Easy Tasks Task Difficulty (b) Easy 90 70 80 Hard 40 60 50 65 65 Presentation Main Effect Difficulty Main Effect SSPresentation SSDificulty 65 vs. 65 80 vs. 50 SE for Presentation for Hard Tasks Interaction Effect SSInteraction SEEasy vs. SEHard

Constructing F-tests for a 2 x 2 Factorial FPresentation = ( SSPresentation / df. Presentation ) ( SSError / df. Error) FDifficulty = ( SSDifficulty / df. Difficulty ) ( SSError / df. Error ) FInteraction = ( SSInteraction / df. Interaction ) ( SSError / df. Error)

Statistical Analyses Necessary to Describe Main Effects of a 2 x 2 Design In a 2 x 2 Design, the Main effects F-tests are sufficient to tell us about the relationship of each IV to the DV… • since each main effect involves the comparison of two marginal means -- the corresponding significance test tells us what we need to know … • whether or not those two marginal means are “significantly different” • Don’t forget to examine the means to see if a significant difference is in the hypothesized direction !!!

Statistical Analyses Necessary to Describe the Interaction of a 2 x 2 Design However, the F-test of the interaction only tells us whether or not there is a “statistically significant” interaction… • it does not tell use the pattern of that interaction • to determine the pattern of the interaction we have to compare the simple effects • to describe each simple effect, we must be able to compare the cell means we need to know how much of a cell mean difference is “statistically significant”

Using LSD to Compare cell means to describe the simple effects of a 2 x 2 Factorial design • LSD can be used to determine how large of a cell mean difference is required to treat it as a “statistically significant mean difference” • Will need to know three values to use the computator • dferror -- look on the printout or use N – 4 • MSerror – look on the printout • n =N/4 -- use the decimal value – do not round to the nearest whole number! Remember – only use the lsdmmd to compare cell means. Marginal means are compared using the man effect F-tests.

Applying lsdmmd to 2 x 2 BG ANOVA Task Presentation Paper Computer Task Difficulty Easy Hard 60 90 60 70 Is there an Interaction? Based on what? for the interaction F(1, 56) = 6. 5, Mse= 300, p =. 023 Yes! F-test of Int What info do we need to compute the LSDmmd? k = 4 groups n = (df + k) / k = (56 + 4) / 4 = 15 MSe = 6. 5 df error = 56 (round down to 50)

With an LSDmmd = 12. 7 Simple effect of Task Presentation SE of Task Presentation for Easy Tasks SE of Task Presentation for Hard Tasks > = 0 = 20 > Simple effects of Task Difficulty SE of Task Difficulty for Paper Pres. SE of Task Difficulty for Comp. Pres. 30 10

Applying lsdmmd to 2 x 2 BG ANOVA Task Difficulty Easy Hard Task Presentation Paper Computer 60 90 75 60 70 65 for Difficulty ME F(1, 56) = 4. 5, p =. 041 lsdmmd = 12. 7 Is there a Task Difficulty main effect? Based on what? Yes! F-test of ME Is main effect descriptive (unconditional) or potentially misleading (conditional)? Simple effects of Task Difficulty 0 = 20 > SE of Task Difficulty for Paper Pres. SE of Task Difficulty for Comp. Pres. Descriptive only for Computer presentation; misleading for Paper presentations.

Applying lsdmmd to 2 x 2 BG ANOVA Task Difficulty Easy Hard Task Presentation Paper Computer 60 90 60 70 60 80 for Presentation ME F(1, 56) = 7. 2, p =. 011 lsdmmd = 12. 7 Is there a Task Difficulty main effect? Based on what? Yes! F-test of ME Is main effect descriptive (unconditional) or potentially misleading (conditional)? Simple effects of Task Difficulty 30 < SE of Task Presentation for Easy Tasks SE of Task Presentation for Hard Tasks 10 Descriptive only for Easy tasks; misleading for Difficult tasks. =

Effect Sizes for 2 x 2 BG Factorial designs For Main Effects & Interaction (each w/ df=1) r = [ F / (F + dferror)] For Main Effects & Simple Effects d = (M 1 - M 2 ) / Mserror r = d² -----d² + 4 (This is an “approximation formula”)