Factorial design I Fully between and withinsubjects design
Factorial design I: Fully between- and withinsubjects design Week 6 1
Clarification: Discriminant validity • If you find yourself convincing your friend “…but this is not exactly the same as…”, then you are trying to discriminate between two similar constructs. • E. g. , fighting spirit <-> rebelliousness • E. g. , disgust sensitivity <-> fear sensitivity • Why is discriminant validity important? • Scientists do not want to reinvent the wheel! 2
Complexities of reality 3
Recall: Factors vs. levels • “Factor” = independent variable. Note: Factor ≠ levels • Example: One factor, two levels • Gender (M vs. F) • Direction (L vs. R) • Example: One factor, multiple levels • Drug Type (Drug A, Drug B, placebo) • Nationality(Indian, Dutch, British) • Month (Jan, Apr, Jul, Oct) 4
Complexity of questioning • Often, we want to go beyond saying “Effect of X on Y”. We often use terms (qualifiers) like “but”, “except when…”, “unless…” to qualify our statements. 80 60 40 20 0 Weight (kg) Two examples: • (L) Males are heavier than females, except during childhood • (R) Males are heavier than females, especially after puberty Males Adults Females Children 80 60 40 20 0 Males Before Females After 5
How many factors and levels could the following have? 1. The effect of gender and age on religiosity 2. The effect of conscientiousness, workload, and teacher likability on student punctuality • The above are the obvious ones. What about the following? 6
How many factors and levels could the following have? “Modern men increasingly value brains over beauty when choosing long-term mates, say researchers. While the common view is that our mate choices are evolutionarily hardwired in our brains and therefore minimally responsive to changing conditions, some evolutionary scientists now argue that humans are programmed to respond with great flexibility to changing environments. ” • An important skill in psychology: When you read research, you need to train yourself to identify factors and levels. • Sometimes these are implicit 7 http: //www. sciencedaily. com/releases/2016/02/160210170618. htm
Factorial design What is it, why do we do it, and how do we interpret it? 8
What is factorial design? • A factorial design has more than one factors. • Examples: • “ 2 (Gender: Male vs. females) × 2 (Measurement: Pre vs. post)” • “ 2 (Gender: Male vs. females) × 3 (Nationality: Indians, Koreans, Germans)” • “ 2 (Gender: Male vs. females) × 2 (Measurement: Pre vs. post) × 2 (Treatment: Drug vs. placebo)” 9
Terminology • When a design only has between-subject factors, we call it “fully between-subjects design” • When a design only has within-subject factors, we call it “______________ design” • When a design has at least one-within subject factor and at least one between-subject factor, we call it a “mixed design” 10
Follow conventions • When describing your design, include the following details: 1. The number of levels 2. The verbal label of the levels 3. Specify whether the design is fully between, fully within, or mixed 4. In the case of mixed, specify which is the betweensubjects factor and which is the within-subjects factor 11
Follow conventions Example: • “I employed a 2 (Gender: Male vs. females) × 3 (Nationality: Indians, Koreans, Germans) fully between-subjects design. ” • When you sufficiently master factorial ANOVAs 2 years later, you can be more flexible, as long as your description is still clear, complete, and unambiguous. But at your level, follow the rules. • Don‘t panic: We will practice how to write results properly a few weeks later. 12
Why use factorial designs? 7 6 5 4 3 2 1 0 Productivity • Suppose you conduct a study on background music and productivity. Western Hindustani classical Music played 7 6 5 4 3 2 1 0 Music preference Western classical Hindustani classical Western Hindustani classical Music played • Simplicity is good, but sometimes reality is inherently complex; simplicity sometimes obscures reality 13
Example: Chess expertise and memory # correct reconstructions • Suppose chess players (experts vs. novices) are shown 32 chess boards, each for 10 seconds. 25 20 15 10 5 0 20 8 Expert Novice Expertise • After board, they are asked to reconstruct the positions of the pieces. The number of reconstructions is the DV. 14
Interpretation • An expert chess player has good memory. • Implication: • To become an expert chess player, one needs to have a good memory • Without good memory, one cannot be an expert chess player • But is that the reality? • Could it be that experts and novices have equally good memory, just that experts have better schemas (mental structures) of chess? • How can you find out? 15
Challenging reality: The role of schemas in expertise • Hypothesis A: IF experts make use of schemas to remember chess positions, THEN disrupting the use of schemas would cause a decline in memory of chess pieces • Hypothesis B: IF experts don’t rely on schemas on remembering chess positions, THEN disrupting the use of schemas would not cause a decline in memory of chess pieces. • Operationalizing “disrupting the use of schemas” • Including real vs. fake positions • Logic: Real chess positions are the result of logical gameplays, and thus the relationship between each chess piece is meaningful; relationships between pieces in fake chess positions are meaningless 16
Examples of real and fake positions Meaningful chess position (real) Meaningless chess position (fake) 17
Results Real • When given fake chess positions, experts perform no better than novices. Fake # correct reconstructions 25 20 20 • This implies that 15 10 8 • the ability to use schemas comes with expertise. • a superior memory is not needed in order to become an expert chess player Expert Novice Expertise • We wouldn’t have known that if not for the second factor: position type (real vs. fake) 8 8 5 0 18
Tabulating the data Factor A: Expertise Factor B: Position type Level A 1: Expert Level A 2: Novice Level B 1: Real 20 8 14 Level B 2: Fake 8 8 8 14 8 11 Factor A: Expertise Factor B: Position type Level A 1: Expert Level A 2: Novice Level B 1: Real A 1 B 1 A 2 B 1 Mean B 1 Level B 2: Fake A 1 B 2 A 2 B 2 Mean A 1 Mean A 2 Grand mean 19
Key concept: Main effects • Suppose this was the data: Factor A: Expertise Factor B: Position type Level A 1: Expert Level A 2: Novice Level B 1: Real 20 8 14 Level B 2: Fake 20 8 14 # correct reconstructions Real 25 20 15 10 5 0 Fake 20 20 8 Expertise 8 1. We say that there is a main effect of expertise; 2. No main effect of position type. Novice 20
Key concept: Main effects • Suppose this was the data: Factor A: Expertise Factor B: Position type Level A 1: Expert Level A 2: Novice Level B 1: Real 20 20 20 Level B 2: Fake 8 8 8 14 14 14 # correct reconstructions Real 25 20 15 10 5 0 Fake 20 20 8 8 Expert Novice Expertise 1. We say that there is a main effect of position type; 2. No main effect of expertise 21
Key concept: Main effects • Suppose this was the data: Factor A: Expertise # correct reconstructions Factor B: Position type 30 20 Level A 1: Expert Level A 2: Novice Level B 1: Real 26 14 20 Level B 2: Fake 20 8 14 23 11 17 Real Fake 26 20 10 14 8 1. We say that there is a main effect of expertise; 2. A main effect of position type. 0 Expertise Novice 22
Key concept: Interaction • In actual fact, this was the data: Factor A: Expertise Factor B: Position type Level A 1: Expert Level A 2: Novice Level B 1: Real 20 8 14 Level B 2: Fake 8 8 8 14 8 11 # correct reconstructions Real 25 20 15 10 5 0 Fake 20 8 Expertise 8 Novice 1. The # correct reconstructions (i. e. , memory) depends on both the expertise and position type. 2. That is, expertise and position type have interacting influences on memory. 23
Let’s take a breather… Q: How do I know if there is an interaction visually? No interaction Interaction • Do the lines behave similarly (are parallel) or not? • Does the effect of one factor depend on the level of the other factor? 24
Let’s take a breather… • Name an example of a one-factor design (IV & DV) • Q: For a one-factor design, can you have a main effect? • You “could”, but it doesn’t quite make sense. • Plain English: “main” suggests that there are some other effects, just like the word “primary” implies the existence of something “secondary”, “big” implies the existence of “small”, etc. 25
Let’s take a breather… • Let’s assume we have two factors, A and B, each with two levels, A 1/A 2 and B 1/B 2. • Work in a pair or trio. • Graph the following, with Factor A always on the x-axis: 1. 2. 3. 4. Only Factor A has an effect; no interaction Only Factor B has an effect; no interaction Both A and B have an effect; no interaction Neither A nor B has an effect; no interaction 26
Let’s take a breather… • The following statements all imply the existence of an interaction. Graph the following, with Factor A always on the x-axis: 1. Perfect crossover interaction (figure out what this means from the sound of it) 2. One cell is different from the rest 3. One main effect + interaction 4. Two main effects + interaction • There are many patterns of interaction possible from just a 2 x 2 design • Misconception alert: Do not think that a 2 x 2 design just has four possible interaction patterns! 27
The simple underlying mathematics behind an interaction (don’t need to know) • 28
Many concepts in statistics revolves around the same formula! (don’t need to know) • Main Interaction effect of X 2 of X 1 of X 2 29
OK, let’s carry on What happens after an interaction is found? 30
Key concept: Simple main effects (simple effects) # correct reconstructions Position type (B) Real (B 1) Fake (B 2) 25 20 15 10 5 0 20 8 8 8 Expert (A 1) Novice (A 2) Expertise (A) • Now that we know that there is an interaction, what do we do? • We then test whether memory differs: • • Within experts Within novices Within real positions Within fake positions 31
Key concept: Simple main effects (simple effects) # correct reconstructions Position type (B) Real (B 1) Fake (B 2) 25 20 15 10 5 0 20 8 8 8 Expert (A 1) Novice (A 2) Expertise (A) Factor A: Expertise Factor B: Position type Level A 1: Expert Level A 2: Novice Level B 1: Real A 1 B 1 A 2 B 1 Mean B 1 Level B 2: Fake A 1 B 2 A 2 B 2 Mean A 1 Mean A 2 Grand mean • In other words, we test Factor A at levels of Factor B; • And Factor B at levels of Factor A 32
Key concept: Simple main effects (simple effects) # correct reconstructions Position type (B) Real (B 1) Fake (B 2) 25 20 15 10 5 0 20 8 8 8 Expert (A 1) Novice (A 2) Expertise (A) • A 2 (Expertise: Experts vs. Novices) × 2 (Position type: Real vs. Fake) thus has 4 simple main effects: • • Experts vs. Novices within Real (i. e. , B 1 at A 1 & A 2) Experts vs. Novices within Fake (i. e. , B 2 at A 1 & A 2) Real vs. Fake within Experts (i. e. , A 1 at B 1 & B 2) Real vs. Fake within Novices (i. e. , A 2 at B 1 & B 2) 33
Three-way factorial designs • Technically, you could have n-way factorial designs. • For a three-way factorial design, it means you have ______ factors • Q: Can you tell the number of levels simply from knowing how many factors there are? Answer: _______ 34
Three-way factorial designs • A two-way ANOVA can have: • Main effect of Factor A • Main effect of Factor B • Interaction between A*B • A three-way ANOVA can have: • • Main effect of Factor _____ Interaction between _____ That is seriously a lot! (And that’s not even counting the number of simple effects!) 35
Four-way factorial designs # Do not even think about it # 36
Tips on learning about factorial designs • Understanding conceptually what a factorial design is will not come easy. • If you think you can just read through the slides and “understand” what a factorial design is, you are greatly mistaken. • Here we only demonstrated with one example. You must spend time thinking through, coming up with personal examples. • If you don’t understand what factorial design is, you will be seriously lost when you have to analyze factorial ANOVA 37
Take home messages • Occam’s Razor: Simpler explanations are preferred over more complicated ones. • However, reality is sometimes complex. Simplicity can be misleading. 38
- Slides: 38