Chapter 7 Using Multivariate Statistics Multiple Regression Multiple

Chapter 7 Using Multivariate Statistics Multiple Regression Multiple Correlation What’s the difference between regression and correlation? Cutoff and Multiple Cutoff Models Validity Generalization & Replication Chap 7 Multivariate Statistics 1

Compensatory Prediction Models Table 7. 1 Composite Trait Scores for 3 Candidates, A, B, C Questions when summing scores (compensatory) are: 1. To weight or not to weight 2. Is a deficiency in one (or more) traits critical? Without weights (top) All composite scores are equal: Sum = 30 All 3 traits weighted equally (unit weighting) With weights (bottom) All composite scores are different: 60, 58, 45 If trait 2 is critical, is an additive model appropriate? Hint: look at candidate C Chap 7 Multivariate Statistics 2

COMPENSATORY PREDICTION MODELS Regression Equations Y = a + b 1 X 1 + b 2 X 2 what’s the difference between b and β weights? Why use one or the other? Multiple Correlation How are the correlations among the predictors related to the multiple R? Would you want high correlations among predictors? Suppressors and Moderator Variables Suppressor variables explained Suppressors How could reading ability act as a suppressor for security guard performance? Moderators How could social skills moderate the conscientiousness- performance relationship? Other Additive Composites Unit weighting is usually sufficient (Bobko, Roth, & Buster, 2007) Could you add veterans’ preference or religious preference? Chap 7 Multivariate Statistics 3

Non-compensatory models Based on Cutoffs Multiple Cutoff Models Two situations warrant it: 1. vital trait 2. if variance is too low (small) to yield sig r. What can happen if cutoffs are all: very all very high? Sequential Hurdles (multiple hurdles) (fig 7. 2) low? When could this be useful? Is it every advisable to select at random? Chap 7 Multivariate Statistics 4

Non-compensatory models Based on Cutoffs Norm-Referenced and Domain-Referenced Cutoffs Norm-referenced: Note Figure 7. 1 Score of 12 is considered Poor (Group A) %ile rank of 24. 6 Good (Group B) %ile rank of 54. 0 Excellent (Group C) %le rank of 99. 2 Domain-referenced (criterion referenced) The domain, not a point in the score distribution (e. g. 12) Is the criterion (bar to reach) For a Mechanic aptitude (norm-referenced) Certification Chap 7 Multivariate Statistics (domain referenced) 5

Non-compensatory models Based on Cutoffs Cutoff based on Local Information Instead of using national or published norms Cutoffs can be established by: Looking at test construction or validation process Contrasting Groups Predictive Yield Method Hire good applicants when available Need to know future need and probable qualifications Regression-Based Methods Identify high and low groups Solve regression equation for X desired Judgmental Methods Angoff (1971) Method –SMEs decide cutoff score Chap 7 Multivariate Statistics 6

Non-compensatory models Based on Cutoffs Multiple Cutoff Methods Non-compensatory – each must be vital to performance Only when predictors are perfectly reliable Partial Compensatory Cut Score Caveats Not for convenience -done too often Dichotomization –rarely justified Justified in some situations: 1. civil service; 2. License/ certification; 3. Cyclical hiring; 4. sequential (e. g minimal quals) Chap 7 Multivariate Statistics 7

REPLICATION AND CROSSVALIDATION Replication Repeating the original study Seldom Cross Validation (for MR) do validity coefficients replicate Do regression weights hold up in a different sample Why is cross validation necessary? Hint: Shrinkage Chap 7 Multivariate Statistics 8

Validity Generalization (we know about this!) Situational Specificity v. Validity Generalization Special form of meta-analysis All validity coefficients (across studies) Would be the same if not for Artifacts Hunter & Schmidt (‘ 90) Reject SS if: 75 % of the variance in coefficients is explained by known artifacts Chap 7 Multivariate Statistics Hunter & Schmidt (1990) 9