Training and Experience Dr Mike Aamodt Radford University

Training and Experience Dr. Mike Aamodt Radford University maamodt@radford. edu Updated 12/06/2004

Personnel Selection Methods • Skills • Training & Education – Work Samples • Experience – Assessment Centers – Applications/Resumes – References – Biodata • Personality & Character – Interviews – Personality Tests • Knowledge – Integrity Tests • Ability • Medical – Cognitive – Medical Exams – Physical – Psychological Exams – Perceptual – Drug Testing

Ratings of Training • Education • Work-Related Training • Military

Education as a Predictor of Job Performance • Education can be broken further into 4 categories: – – Degree Years of education Major GPA

Does Education Predict Performance?

Summary of Meta-Analyses Meta-analysis Aamodt (2004) Occupation K N ρ 9, 120 . 28 Police 54 Vineberg & Joyner (1982) Military 35 Hunter (1980) Hunter & Hunter (1984) Schmidt & Hunter (1998) Dunnette (1972) USES data base 425 32, 124. 10 Entry level petroleum 15 . 25 . 00

Validity of Education for Police Our Updated Meta-Analysis Academy Grades ____ Number of studies 32 Sample Size 6, 153 Uncorrected validity. 26 Corrected validity. 38 Sampling error % 100% Supervisor Ratings _____ 54 9, 120. 17. 28 80% Discipline Problems ____ 54 21, 416 -. 08 -. 12 73%

Our Meta-Analysis (Cont. ) Education level _______ GED/HS Diploma 12 -64 college hours AAS 64 college hours Bachelor’s Academy Grades _______ N Score _____ 376 -. 23 c 100 -. 05 ac 98. 16 ab 62. 01 ac 255. 32 b Patrol Performance ________ N Score _____ 915 -. 12 a 391. 02 b 139. 07 bc 45. 29 bc 441. 18 c

Interaction of Education and Experience Ratings of Patrol Performance Experience Educational Level High School Diploma < 2 years 2+ Years Total M -. 09 ab -. 16 a -. 13 a SD (. 92) (. 98) Some college or associate’s degree M -. 26 a . 07 b SD (1. 02) (. 94) B. A. or higher M . 04 ab . 34 c SD (1. 02) (1. 12) -. 09 a. 19 b

Education and Academy Grades Effect of Academy Length K N r ρ SE % 32 6, 153 . 26 . 38 100 12 -17 weeks 16 2, 477 . 31 . 46 100 > 20 weeks 15 3, 473 . 25 . 36 100 Overall Academy Length

Does Education Add Incremental Validity Over Cognitive Ability?

Education and Incremental Validity • Schmidt & Hunter (1998) say no – Cognitive ability (r =. 51) – Cognitive ability and education (r =. 52)

Education and Incremental Validity • Our meta-analysis for police says yes – Academy Grades • Cognitive ability (r =. 33) • Cognitive ability and education (r =. 34) – Patrol Performance (uncorrected) • Cognitive ability (r =. 16) • Cognitive ability and education (r =. 29)

Can You Substitute Education for a Cognitive Ability Test? • Education has less adverse impact • But – Cognitive ability and education are only slightly correlated • . 29 in our police data • . 00 assumption by Schmidt & Hunter (1998) of applicants applying for a particular job • . 55 estimate by Schmidt & Hunter (1998) for the general population – Education adds incremental validity for law enforcement – Our dataset contains plenty of college graduates with low cognitive ability

Does Grade Point Average Predict Performance?

Validity of GPA • GPA is a valid predictor of performance on the job, training performance, starting salary, promotions, and grad school performance • GPA is most predictive in the first few years after graduation (Roth et al. , 1996) • GPA will result in high levels (d=. 78) of adverse impact (Roth & Bobko, 2000) • People with high GPAs – Are intelligent (r=. 50; Jensen, 1980) – Are conscientious (r=. 34; Bevier et al. , 1998)

Validity of GPA Meta-Analysis Results r ρ Job performance (Roth et al. , 1996) . 16 . 36 Training performance (Dye & Reck, 1989) . 29 Promotions (Cohen, 1984) . 16 Work-Related Criteria Salary (Roth & Clarke, 1996) Starting salary . 13 . 20 Current salary . 18 . 28 Grades . 28 . 30 Faculty ratings . 25 . 35 Graduate School Performance (Kuncel et al. , 2001)

Predicting Graduate School Performance Graduate GPA Teaching Ratings GRE Score GPA – Overall GPA – Psych . 33* . 00 . 31* . 02 . 32* . 03 GPA – Junior/Senior Faculty References Mental agility Vigor Extroversion Agreeableness Conscientiousness . 40* . 09 . 25* -. 08 -. 26* -. 07 . 03 . 24* -. 11 . 06 . 16 -. 04

Does College Major Predict Performance?

College Major as a Predictor of Job Performance • Aamodt (2004) meta-analysis • Validity of having a criminal justice major __________________ r k n _____ Patrol performance -. 02 9 1, 706 Academy grades. 01 6 976 Discipline problems. 00 10 2, 621 __________________

Will Education Requirements Result in Adverse Impact?

Adverse Impact Technique White. Black White. Hispanic Meta-analysis Cognitive ability 1. 10 . 72 GPA . 78 Roth & Bobko (2000) Biodata . 33 Bobko et al. (1999) Reference Ratings . 25 Aamodt (2002) Structured interview . 23 Huffcutt & Roth (1998) Recommendations . 22 Aamodt (2002) HS diploma . 20 . 52 Aamodt (2002) Bachelor’s degree . 18 . 26 Aamodt (2002) Personality . 09 Integrity tests . 07 Roth et al. (2001) Schmitt et al. (1996) -. 05 Ones & Viswesvaran (1998)

Lingering Questions • Is the validity of education job specific? • What is the actual incremental validity of education over cognitive ability? • Why would education predict performance? – – Knowledge Liberal arts skills Mental ability Motivation

Work-Related Training as a Predictor of Job Performance Aamodt (2000) Meta-Analysis • Performance in the academy predicts probationary performance (ρ=. 44) • Performance in the academy predicts patrol performance (ρ =. 38) • Performance in the probationary period predicts patrol performance (ρ =. 44)

Military Training as a Predictor of Job Performance • It is common for Police departments to give preferential treatment to candidates with Military experience for two reasons: – Reward veterans for serving their country – They believe veterans will be better cops because the training that they’ve gone through and that of police are similar

Validity of Military Experience and Police Performance Academy Grades Patrol Performance Commendations K 9 16 8 N 2, 061 4, 090 1, 337 r . 02 -. 03 -. 07 ρ . 04 -. 05 -. 10 SE % 90% 23% 81%

Military Experience in Candidate Selection • In selecting applicants, the employer must be careful because: – Minorities tend to have a higher rate of undesirable discharges; therefore any questions about military service dates, type of discharge, and discipline received while in the service can result in adverse impact. - Gatewood & Feild (2001)

Experience

Experience • Past behavior predicts future behavior – Experience is a valid predictor of future performance (r =. 27; Quinones et al. , 1995) • Types of Experience – Work – Life

Experience • Evaluated through: – – – Application blanks Resumes Interviews Reference checks Biodata instruments

Experience • Considerations – How much experience? – How well did the person perform? – How related is it to the current job?

Experience Predicts Best… • Credit prior work experience only: – In the same occupational area as that in which performance is to be predicted – In the performance of tasks or functions that have direct application on the job • Recency of experience should be used as a decision rule for awarding credit only when justified on a case-by-case basis • Credit for duration of work experience should be limited to a few years. • High prediction up to about 3 years of experience, declining to low prediction for more than 12 years of experience.

Experience for Selection : Some Concerns • Sullivan (2000) claims that “experience in solving ‘past problems’ is rapidly losing its applicability to current and future problems. ” • Organizations will increase their applicant pool if they delete the “ancient history” requirements (i. e. “Ten years experience required”).

Sullivan (2000) 1) Reduce or eliminate the number of years required in your ads and replace them with “the demonstrated ability to solve problems with our required level of difficulty. 2) Use simulations and actual problems to assess applicants. 3) Develop “future-oriented” questions for applicants. 4) Train evaluators and compensation professionals to put less weight on experience of candidates. 5) Revise job descriptions to include level of difficulty. 6) Identify the amount and type of experience and competencies that would predict job performance. 7) Check to see if there is a correlation between the number of years of experience an employee has and their success in your firm.

Experience: Some More Concerns • • Performance matters “Haven’t done” doesn’t mean “can’t do” Experience has a shelf life Listing something on a resume is not experience Where you get your experience matters Experience does not guarantee success Experience is expensive More experience might be bad (old ways and ideas)