ChapterNine Evaluation of Training Evaluation Phase Input Evaluation

Chapter-Nine Evaluation of Training Evaluation Phase Input Evaluation Objectives Organizational Constraints Design Issues Process Evaluation Strategy and Design Output Process Measures Outcome Measures • Reaction • Learning • Behavior • Results

Evaluation of Training Potential Questions To Be Addressed tin a process Analysis (Before Training) Were needs diagnosed correctly? • What data sources were used? • Was a knowledge /skill deficiency identified? • Were trainees assessed to determine their prerequisites KSAs? Where needs correctly translated into training objectives? • Were all objectives identified? • Were the objectives written in clear, appropriate manner? Was in evolution system designed to measure accomplishment of objectives? Was the training program designed to meet all the training objectives? • Was previous learning that might either support or inhibit learning in training identified? • Were individual differences assessed an taken into consideration in training design? • Was trainee motivation to learn assessed? • What steps were taken to address trainee motivation to learn? • Were process built into the training o facilitate recall and transfer? • What steps are included in the training to call attention to key learning events? • What steps are included in the training to aid trainees in symbolic coding and cognitive organization? • What opportunities are included in the tainting to provided symbolic and behavioral practice? • What actions are included in the training to ensure transfer of learning to the job? Are the training techniques to be used appropriate for each of the learning objectives of the training?

Evaluation of Training Potential Question to Be Addressed in a Process Analysis (During Training) Were the trainer, training techniques, and learning objectives well matched? Were lecture portions of the training effective? • Was involvement encouraged or solicited? • Were questions used effectively? Did the trainer conduct the various training methodologies (case, role-play, etc. ) appropriately? • Were they explained well? • Did the trainer use allotted time for activities? • Was enough time allotted? • Did trainees follow instructions? • Was here effective debriefing following the exercises? Did the trainer follow the training design and lesson plans? • Was enough time design and lesson plans? • Was time allowed for questions?

OUTCOME DATE Evaluation of Training To determine how well the training met or is meeting its goals, it is necessary to examine various outcomes measures. The four outcomes measures that are probably the best known are reaction, learning, behavior, and organza tonal results. These outcome are ordered as follows: Ø Reaction outcomes come first and will influence how much can be learned. Ø Learning outcomes influence how much behavior can change back on the job. Ø Behavior outcomes are the changes of behavior on the job that will influence organizational results. Ø Organizational results are the changes in the bottom line related to the reason for training in the first place, such as high grievance rate, low productivity, and so forth. Reaction outcomes are measures of the trainee’s perceptions, emotions, and subjective evaluations of the training experiences. They represent the first level of evaluation and are important because favorable reaction create motivation to learn. Learning may also occur even if the training is boring or alternatively, it may not occur even if it is interesting. Learning outcomes are measured by how well the learning objectives and the overall training objective were achieved. The learning objectives for the training that were developed in the design phase specify the types of outcomes that will signify that training has been successful. Note the critical relationship between the needs analysis and evaluation.

OUTCOME DATE/Cont. . . Evaluation of Training Job behavior outcome are measures of the degree to which the learned behavior has transferred to the job. During the TNA, performance gaps were indentified and traced to areas in which employees were behaving the in a manner that was creating the gap. The methods used for measuring job behavior in the TNA should be sued in measuring job after the completion of training. Once again, the link between needs analysis and evaluation is evident. The degree to which job behavior improves places a cop on how much training can improve organizational results. Organizational results occupy the highest level in the hierarchy. They reflect the gap in performance identified in the TNA. The organizational result is often what triggers archive (as opposed to proactive) training. Here are some examples: ØHigh levels of scrap are being produced ØEmployees are quitting in record number. ØSales figures dropped over the last two quarters. ØGrievances are on the increases. ØThe number of rejects from quality control is rising.

Evaluation of Training Reaction Questionnaire for the Trainer Please circle the number that reflects the degree to which you agree or disagree with the following statements. 1 = Strongly disagree, 2 = Disagree, 3 = Neither agree nor disagree, 4 = Agree, 5 = Strongly agree, 1. The trainer did a good job of stating the objectives at the beginning of training. 1 2 3 4 5 2. The trainer made good use of visual aids (easel, ) white-board) when making the presentations. 1 2 3 4 5 3. The trainer was good at keeping everyone interested in the topics. 1 2 3 4 5 4. The trainer encouraged questions and participation from trainees. 1 2 3 4 5 5. The trainer made sure that everyone understood the concepts before moving on to the next topic. 1 2 3 4 5 6. The trainer summarized important concepts before moving to the next module. 1 2 3 4 5 7. Overall, how would you rate this trainer? (Check one) –– 1. Poor–I would not recommend this trainer to others. –– 2. Adequate–I would recommend this trainer only if no others were available. –– 3. Average –– 4 Good –I would recommend this trainer above most others. –– 5. Excellent–This trainer is among the best I’ve ever worked with.

Evaluation of Training 1. Determine what needs to be found out. 2. Develop a written set of questions to obtain the information. Steps to consider In Developing a Reaction Questionnaire 3. Develop a scale to quantify respondent’s data. 4. Make forms anonymous so that participants feel free to respond honestly. 5. Ask for information that might be useful in determining differences in reactions by subgroups taking the training (e. g. , your vs. old; minority vs. nonminority). This could be valuable in determines effectiveness of training by different cultures, for example, which might be lost inane overall assessment. Note: Care must be taken when asking for this information. If you ask too many questions about race, gender, age, tenure, and so on, participants will begin to feel that they can be identified without their name on the questionnaire. 6. Allow space for additional comments to allow participants the opportunity to mention things you did not consider. 7. Decide the best time to give the questionnaire to get the information you want. a. If right after training, ask someone others than the instructor to administer and pick up the information b. If some time later, develop a mechanism for obtaining a high response rate (e. g. ;

Job Behavior Data Evaluation of Training Once it is determined that learning took palace, the next step is to determine whether the training transferred to the job. Assessment at step is certainly more complex and is often ignored because to the difficulties of measurement. Example of an Attitudinal Measure Attitudes Toward Empowerment Please circle the number that reflects the degree to which you agree or disagree with the following statements. 1 = Strongly disagree, 2 = Disagree, 3 = Neither agree nor disagree, 4 = Agree, 5 = Strongly agree, 1. Empowering employees is just another way to get more Work done with fewer people [reverse scored] 1 2 3 4 5 2. Empowering employees allows everyone to contribute their ideas for the betterment of the company. 1 2 3 4 5 3. The empowerment program improved my relationship with my supervisor. 1 2 3 4 5 4. Empowerment brought more meaning to my life at this company 1 2 3 4 5 5. Empowerment brought more meaning to my life in other plants in this company. 1 2 3 4 5 6. The empowerments process provided a positive influence in labor- 1 management relations. 2 3 4 5

Evaluation of Training Types of Outcomes and examples of Factors affecting Those outcomes Organizational Results External environment of the organization: Economy, regulations, suppliers, etc. Internal environment of the organization: Policies, procedures, systems Employee performance, KSAs, and needs Job Behavior Transfer of training Motivational forces in to job setting Opportunity to apply training of the job KSAs Learning Trainee readiness for the course Trainee motivation to learn Design, materials, and content Trainer(s) behaviors Reactions Perceived match between trainees expectations and what training provided

Evaluation of Training Cost/Benefit and Cost-Effectiveness Evaluations Was the training cost worth the results? This question can be answered in either of the following two ways: Cost/Benefit Evolution: A cost/benefit evolution of training compares the monetary cost training with the non-monetary benefits. It is difficult of place a value on these benefits, which include attitudes and working relationships. The labor peace brought about by the reduction in grievances is difficult to assess. But it rates high in value compared with the cost of tainting. The conflict resolution skills learned by supervisors provide the non-monetary benefit of better relationships between supervisors and union officials, and this in important. However, it is also possible to assess the reduction in grievances (for example) in a way that directly answers the cost-effectiveness question. Cost-Effectiveness Evolution: A cost-effectiveness evaluation compares the monetary costs of training with the financial benefits accrued from training. There are two approaches for assessing cost-effectiveness: 1. Cost saving, a calculation of the actual cost savings, based on the change in “results” 2. Utility analysis, an examination of value of overall improvement in the performance of the trained employees.