BBA 229 Training and Development Lecture 5 Training
BBA 229 Training and Development Lecture 5 Training Evaluation Dr. S. Chan Department of Business Administration charmaine@chuhai. edu. hk http: //home. chuhai. hk/~charmaine/ 1
Training Evaluation (1 of 2) Training effectiveness – the benefits that the company and the trainees receive from training Training outcomes or criteria – that the trainer and the company use to evaluate training programs 2
Training Evaluation (2 of 2) Training evaluation – the process of collecting the outcomes needed to determine if training is effective Evaluation design – from whom, what, when, and how information needed for determining the effectiveness of the training program will be collected 3
Reasons for Evaluating Training (1 of 2) Companies are investing millions of dollars in training programs to help gain a competitive advantage Training investment is increasing because learning creates knowledge – This differentiates between those companies and employees who are successful and those who are not 4
Reasons for Evaluating Training (2 of 2) Because companies have made large dollar investments in training and education and view training as a strategy to be successful, they expect the outcomes or benefits related to training to be measurable 5
Training evaluation provides the data needed to demonstrate that training does provide benefits to the company. 6
Formative Evaluation The evaluation of training that takes place during program design and development Helps to ensure that: – the training program is well organized and runs smoothly – trainees learn and are satisfied with the program Provides information about how to make the program better 7
Pilot Testing The process of previewing the training program with potential trainees and managers or with other customers It can be used: – as a “dress rehearsal” to show the program to managers, trainees, and customers – formative evaluation 8
Summative Evaluation conducted to determine the extent to which trainees have changed as a result of participating in the training program May also measure the return on investment (ROI) that the company receives from the training program 9
Why Should A Training Program Be Evaluated? (1 of 2) To identify the program’s strengths and weaknesses To assess whether content, organization, and administration of the program contribute to learning and the use of training content on the job To identify which trainees benefited most or least from the program 10
Why Should A Training Program Be Evaluated? (2 of 2) To gather data to assist in marketing training programs To determine the financial benefits and costs of the programs To compare the costs and benefits of training versus non-training investments To compare the costs and benefits of different training programs to choose the best program 11
The Evaluation Process Conduct a Needs Analysis Develop Measurable Learning Outcomes and Analyze Transfer of Training Develop Outcome Measures Choose an Evaluation Strategy Plan and Execute the Evaluation 12
Outcomes Used in Evaluating Training Programs: (1 of 4) Cognitive Outcomes Affective Outcomes Skill-Based Outcomes Results Return on Investment 13
Outcomes Used in Evaluating Training Programs: (2 of 4) Cognitive outcomes – determine the degree to which trainees are familiar with the principles, facts, techniques, procedures, or processes emphasized in the training program – measure what knowledge trainees learned in the program Skill-based outcomes – assess the level of technical or motor skills – include acquisition or learning of skills and use of skills on the job 14
Outcomes Used in Evaluating Training Programs: (3 of 4) Affective outcomes – include attitudes and motivation – reaction outcomes – trainees’ perceptions of the program including the facilities, trainers, and content Results – determine the training program’s payoff for the company 15
Outcomes Used in Evaluating Training Programs: (4 of 4) Return on Investment (ROI) – comparing the training’s monetary benefits with the cost of the training direct costs indirect costs benefits 16
Determining Whether Outcomes Are Good training outcomes need to be: Relevant Reliable Discriminative Practical 17
Good Outcomes: Relevance Criterion deficiency – failure to measure training outcomes that were emphasized in the training objectives 19
Good Outcomes (continued) Reliability – degree to which outcomes can be measured consistently over time Discrimination – degree to which trainee’s performances on the outcome actually reflect true differences in performance Practicality – the ease with which the outcomes measures can be collected 20
Training Program Objectives and Their Implications for Evaluation: Objective Learning Transfer Outcomes Reactions: Did trainees like the program? Did the environment help learning? Was material meaningful? Skill. Based: Ratings by peers or managers based on observation of behavior Cognitive: Pencil-and-paper tests Affective: Trainees’ motivation or job attitudes Skill. Based: Performance on a work sample Results: Did company benefit through sales, quality, productivity, reduced accidents, and complaints? 21 Performance on work
Factors That Influence the Type of Evaluation Design Factor How Factor Influences Type of Evaluation Design Change potential Can program be modified? Importance Does ineffective training affect customer service, product development, or relationships between employees? Scale How many trainees are involved? Purpose of training Is training conducted for learning, results, or both? Organization culture Is demonstrating results part of company norms and expectations? Expertise Can a complex study be analyzed? Cost Is evaluation too expensive? Time frame When do we need the information? 22
Conditions for choosing a rigorous evaluation design: (1 of 2) 1. The evaluation results can be used to change the program 2. The training program is ongoing and has the potential to affect many employees (and customers) 3. The training program involves multiple classes and a large number of trainees 4. Cost justification for training is based on numerical indicators 23
Conditions for choosing a rigorous evaluation design: (2 of 2) 5. You or others have the expertise to design and evaluate the data collected from the evaluation study 6. The cost of training creates a need to show that it works 7. There is sufficient time for conducting an evaluation 8. There is interest in measuring change from pre-training levels or in comparing two or more different programs 24
Importance of Training Cost Information To understand total expenditures for training, including direct and indirect costs To compare costs of alternative training programs To evaluate the proportion of money spent on training development, administration, and evaluation as well as to compare monies spent on training for different groups of employees To control costs 25
- Slides: 25