Planned Contrasts and Data Management Class 20 Topics

Planned Contrasts and Data Management Class 20

Topics Covered Today 1. Complete Two Way ANOVA 2. Planned Contrasts 2. Post-hoc tests 3. Analysis of Residual Variance 4. Data Management a. Setting up data files b. Cleaning data

Interaction: Gender * Birth Order Total Mea n (4. 3 2)

Deviation of an Individual Score in Two Way ANOVA ABSijk – T = (Ai – T) + (Bj – T) + (ABij – Aij – Bij + T) + (ABSijk – ABij) Ind. score Total Mean Factor A Effect Factor B Interaction AXB Effect (Birth Order) (Gender (Birth * ) Gender) Error (w’n Effect)

Degrees of Freedom in 2 -Way ANOVA Between Groups Factor A (Birth Order) Factor B (Gender) Interaction Effect Factor A X Factor B (Birth X Gender) df A = a - 1 2 – 1 = 1 df B = b – 1 2 – 1 = 1 df. A X B = (a – 1) (b – (2 -1) x (2 -1) = 1) 1 Error Effect Subject Variance df s/AB = n - ab 10 – (2 x 2) = 6 Total Effect Variance for All Factors df Total = n – 1 10 – 1 = 9

Analysis of Variance Summary Table: Two Factor (Two Way) ANOVA Source of Variation Square F Ratio Sum of Squares df (MS) A SSA a - 1 B SSB b - 1 A X B (SS) SSA X B (a - 1)(b - 1) Within (S/AB) SSS/A Total SST ab (s- 1) Mean MSA = SSA df. A MSS/AB MSB = SSb dfb MSB MSS/AB MSAB SSAB A X B = df MSA X B MSS/AB SSS/AB df. S/AB = abs - 1

F Ratios for 2 -Way ANOVA Birth Order Gender Birth Order X Gender

ONEWAY ANOVA AND GENDER MAIN EFFECT Source Gender Error Sum of Square 3. 42 s df Mean Square F Sig. 1 3. 42 1. 22 . 34 22. 45 8 2. 81 TWO-WAY ANOVA AND GENDER MAIN EFFECT Source Sum of Squares df Mean Square F Sig. Gender 3. 42 1 3. 42 4. 03 . 09 Birth Order Interactio n Error 16. 02 18. 87 . 005 3. 75 1 3. 75 4. 42 . 08 5. 09 6 0. 85 9 Total Oneway F: 3. 42 = 4. 42 1. 22 2. 81 Twoway F: 3. 42 =

"Pop" Culture: Gun Support as a Function of Political Party and Gender A Planned Contrast Study

Support of Gun Control: Which Party? Which Gender? How much do you support handgun instruction in school? 1 2 3 We predict: GOP Men? 5 5 GOP Women? 5 Dem Men? 2 4 5

Planned Contrast: Function 1. Factorial ANOVA tests for orthogonal (perpendicular) interactions. 2. Some studies predict non-orthogonal interactions. 3. Planned contrast provides more predictive power to confirm non-orthogonal contrasts of any particular shape (“wedge”, “arrow” [like above] or other).

Planned Contrast: Execution (Conceptual) 1. Must predict pattern of interaction before gathering data. Predict that Democratic women will be most opposed to gun instruction in school, compared to Democratic men, Republican men, and Republican women. Note: Preregistering prediction will help legitimize planned contrast.

Convert Separate Factors into Single Factor 1. Two separate factors pol. party 1) GOP 2) Democrat gender 1) Male 2) Female 2. Convert the two separate factors into a single factor genparty 1) Male Republican 2) Male Democrat 3) Female Republican 4) Female Democrat

Convert Separate Factors into Single Factor SPSS Syntax (commands) genparty 1 = Male Republican 2 = Male Democrat 3 = Female Republican 4 = Female Democrat

Converting Multi-factors into Single Factor for Planned Contrast Gender Political Party Male Female Republican 5. 00 4. 75 Democrat 4. 50 2. 75 Converted into single factor with four levels GENPARTY 1 5. 00 2 4. 50 = Male/Republican = Male/Democrat

Planned Contrast: Execution (Conceptual) 3. Conduct one-way ANOVA, with new single variable as predictor. 4. Assign weights to the four levels, as follows: This formula also ok. 1) Male Republican -1 1 2) Male Democrat -1 1 3) Female Republican-1 1 4) Female Democrat 3 -3 * Weights indicate which sub-groups are to be compared. 5. Planned contrast then limits comparison to the * Weights must add up to zero indicated groups, but “counts” all subjects in terms of * Positive and negative weights are arbitrary; only degrees of freedom and computation of error. This contrast matters provides greater predictive power. This is even true if weight for some group(s) set at zero.

Graph of Gender X Political Party and Opposition to Gun Instruction in School

Univariate Analysis of Variance [Data. Set 1] Orthogonal Interaction

Planned Contrast, Page 1 Note: This is ANOVA p value, NOT contrast p value

Planned Contrast, Page 2 Contrast Tests Std. Contrast Contras Error value t Assumes eq. var. -6. 000 1. 768 -3. 394 12 . 005 Doesn’t assume -3. 539 5. 50 1 . 014 -6. 000 1. 696 t df Sig. (2 –tailed)

Post Hoc Tests Do female democrats differ from other groups? 1 = Male/Republican 5. 00 2 = Male/Democrat 4. 50 3 = Female/Republican 4. 75 Conduct six t tests? NO. Why Will capitalizes on 4 = Female/Democrat 2. 75 not? chance. . 05 +. 05 =. 30 Solution: Post hoc tests of multiple comparisons. Post hoc tests correct for the inflated likelihood of Type I error Kent's favorite—Tukey test of multiple comparisons, which is the most generous. NOTE: Post hoc tests can be done on any multiple set

Conducting Post Hoc Tests 1. Recode data from multiple factors into single factor, as per planned contrast. 2. Run oneway ANOVA statistic 3. Select "posthoc tests" option. ONEWAY gunctrl BY genparty /CONTRAST= -1 -1 -1 3 /STATISTICS DESCRIPTIVES /MISSING ANALYSIS /POSTHOC = TUKEY ALPHA(. 05). Note: Not necessary to conduct planned contrast to conduct post-hoc test Selected post -hoc test

Post hoc Tests, Page 1

Post Hoc Tests, Page 2 HSD = Honest Significant Difference

“Quality Control” for Planned Contrast Issue: Planned contrast can be a very “liberal” test, confirming patterns that don’t closely fit with actual predictions. Predicted this Obtained this Result of this – 1, -1, + 3 planned contrast might still be significant How to assess the “quality” of a significant planned

Analysis of Residual Variance Logic of test: Did (Between groups effect – Contrast effect) leave significant amount of systematic (non-random) variance unexplained? If so, then the contrast did not do a good job. It did not explain the outcome fully. However, if “what’s left over” (i. e. , between effect – contrast) is not significant, then the contrast accounts for most of the treatment. In this case, the contrast did do a good job.

Contrast Should “Absorb” Most of Between-Groups Effect ═ ─ Between. Groups Effect Contrast Effect Residual Effect

Analysis of Residual Variance Test BETWEEN GROUPS EFFECT - CONTRAST EFFECT = “RESIDUALS EFFECT” BETWEEN GROUPS EFFECT - CONTRAST EFFECT RANDOM ERROR = F Residuals If F Residuals is large then most of planned contrast results is really between groups effect. NOT GOOD If F Residuals is small, then planned contrast truly explains outcome. GOOD!

Understanding Test of Residual Variance ─ Between. Groups Effect SS Between ═ ─ Contrast Effect ═ ─ SS Contrast ═ Residual Effect SS Residuals How much greater is residual than random error? We want the difference to be small. In other words, how much “meaningful effect” remains, after contrast contribution removed? We want there to be very little. Need F ratio: Mean Residuals ÷ Random error = F Residuals X If F Residuals is significant: Contrast is ______ Good ______ Not Good

1. Steps in Analysis of Residual Variance Test Get SPSS printout of planned contrast 2. Get t of contrast, square it to get contrast F [t = F ; F = t 2 ] 3. Compute SS contrast (SSc): Multiply contrast F by mean sq. w/n (MSw) of oneway ANOVA. This results in SS contrast (SSc). 4. Compute SS residuals (SSr): Get SS between (SSb) from oneway, and subtract SSc. (SSb – SSc) = SS residuals (SSr) 5. Compute MS residuals (MSr): Divide SSr by Contrast degrees of freedom (df). Contrast df = (Oneway Between. Groups df – 1). 6. Compute F residuals: Divide MSr by MSw. MSr/MSw = F residuals 7. Compute df for F resid: numerator df = (df oneway – df

Residuals Analysis Test 1. Get SPSS printout of planned contrast 2. F of contrast (Fcont) = t 2; t = -3. 39 t 2 = 11. 49 3. SScontrast (SScont) = F cont X MSw = 11. 49 X 1. 04 = 11. 95 4. SSresiduals (SSres) = SSbetween (SSb) – SScont = 12. 50 – 11. 95 =. 55 5 a. Residuals df = df oneway – df contrast = 3 -1 = 2 5 b. MSresiduals (MSres) = SSres / residuals df =. 55/2 = 0. 275 6. F residuals (Fresid): Divide MSres by MSw = 0. 275/1. 04 =. 26 7. DF for Fresid: Numerator = df residuals (see 5 a, above) = 2, Denominator = df within=12; df Fresid = (2, 12)

Data Management Issues Setting up data file Checking accuracy of data Disposition of data Why obsess on these details? Murphy's Law If something can go wrong, it will go wrong, and at the worst possible time. Errars Happin!

Creating a Coding Master 1. Get survey copy 2. Assign variable names 3. Assign variable values 4. Assign missing values 5. Proof master for accuracy 6. Make spare copy, keep in file drawer

Coding Master variable values variable names Note: Var. values not needed for scales

Cleaning Data Set 1. Exercise in delay of gratification 2. Purpose: Reduce random error 3. Improve power of inferential stats.

Complete Data Set Note: Are any cases missing data?

Checking Descriptives Are any “Minimums” too low? Are any “Maximums” too high? Do Ns indicate missing data? Do SDs indicate extreme outliers?

Checking Correlations Between Variables Do variables correlate in the expected manner?

Using Cross Tabs to Check for Missing or Erroneous Data Entry Case A: Expect equal cell sizes Gender Oldest Youngest Only Child Males 10 10 20 Females 5 15 20 TOTAL 15 25 40 Case B: Impossible outcome Number of Oldest Youngest Only Child Siblings None 4 3 6 One 3 4 0 More than one 3 4 2 TOTAL 10 10 8

Storing Data Raw Data 1. Hold raw data in secure place 2. File raw data by ID # 3. Hold raw date for at least 5 years post publication, per APA Automated Data 1. One pristine source, one working file, one syntax file 2. Back up, Back up

File Raw Data Records By ID Number 01 -20 21 -40 41 -60 61 -80 81 -100 101 -120

COMMENT SYNTAX FILE GUN CONTROL STUDY SPRING 2007 COMMENT DATA MANAGEMENT IF (gender = 1 & party = 1) genparty = 1. EXECUTE. IF (gender = 1 & party = 2) genparty = 2. EXECUTE. IF (gender = 2 & party = 1) genparty = 3. EXECUTE. IF (gender = 2 & party = 2) genparty = 4. EXECUTE. COMMENT ANALYSES UNIANOVA gunctrl BY gender party /METHOD = SSTYPE(3) /INTERCEPT = INCLUDE /PRINT = DESCRIPTIVE /CRITERIA = ALPHA(. 05) /DESIGN = gender party gender*party. ONEWAY gunctrl BY genparty /CONTRAST= -1 -1 -1 3 /STATISTICS DESCRIPTIVES /MISSING ANALYSIS /POSTHOC = TUKEY ALPHA(. 05). Save Syntax File!!!

Research Project Notebook Purpose: All-in-one handy summary of research project Content: staff, etc. ) 1. Administrative (timeline, list of 2. Overview of Research 3. Experiment Materials * Surveys * Consents, debriefings * Manipulations * Procedures summary/instructions 4. IRB materials * Application * Approval 5. Data