Return on Investment ROI in Training Programme developed

Return on Investment (ROI) in Training Programme developed and delivered by Dr Belinda Ketel Faranani Facilitation Services 1

2

Activity: “Hello” to group Your link to ROI / training? 3

Activity: Expectations for this workshop? 4

Programme 08 h 00 – 08 h 30 Registration 08 h 30 – 09 h 00 Fasset Welcome, Briefing and 09 h 00 – 10 h 30 Understanding ROI 10 h 30 – 10 h 50 Tea Break 10 h 50 – 12 h 45 Working with ROI 12 h 45 – 13 h 30 Lunch break 13 h 30 – 15 h 00 Application of ROI in practice 15 h 00 – 15 h 20 Comfort break 15 h 20 – 16 h 00 Q&A on ROI 16 h 00 – 16 h 15 Fasset closure Introductio

Lets map this out?

About this seminar The seminar aims to challenge Skills Development Facilitators (SDFs), Human Resource Professionals and Managers with understanding and applying ROI to their training interventions in order to make business sense

"GENIUS IS ONE PERCENT INSPIRATION AND NINETYNINE PERCENT PERSPIRATION“ THOMAS EDISON

Understanding ROI

Trends in training evaluation Results Based Activity Based • • no effort to prepare the work environment to support transfer no efforts to build partnerships with key managers no measurement of results or cost benefit analysis planning and reporting on training is input focused • • environment prepared to support transfer partnerships established with key managers and clients measurement of results and cost benefit analysis planning and reporting on training is output focused

Trends in training evaluation Activity Based Results Based • no business need for the program • program linked to specific business needs • no assessment of performance issues • assessment of performance effectiveness • no specific measurable objectives • specific objectives for behavior and business impact • no effort to prepare program participants to achieve results • results expectations communicated to participants

The “catch” in the Measurement Cycle: What counts gets measured What gets rewarded counts What gets measured gets done What gets done gets rewarded

So WHERE is ROI? How do you see it?

A Rational Approach to ROI • • Keep the process simple Use sampling for ROI calculations Always account for the influence of other factors Involve management in the process Educate the management team Communicate results carefully Give credit to participants and managers Plan for ROI calculations

The Journey to Increased Accountability Profit Center Level 5 (ROI) Level 4 (Business Impact) HRD / Performance Improvement Accountability Level 3 (Application) Level 2 (Learning) Level 1 (Reaction) Time

Establishing ROI • Showing added value of ROI must be a top priority • Information management should align its goals with organizational strategy • Communication of ROI is key • ROI is a short list of added values expressed in: Money Time Quality

ROI of what?

Calculating ROI: What are the costs?

ROI explained from an HR perspective Canadian Society for T&D ROI of Training - keeping it simple

THE ROI PROCESS Calculating the Return on Investment of Performance Solutions Training Solutions Ø HR Solutions Ø Technology Solutions Ø Organization Development Solutions Ø Change Initiatives Ø EVALUATION PLANNING DATA COLLLECTION 1. SATISFACTION/ REACTION 2. LEARNING ROI Calculation - Example Net Program Benefits ROI DATA ANALYSIS COLLECT DATA DURING SOLUTION IMPLEMENTATION DEVELOP EVALUATION PLANS AND BASELINE DATA DEVELOP OBJECTIVES OF SOLUTION = Program Costs ROI = R 230, 000 - R 88, 000 = 1. 61 x 100 = 161% ISOLATE EFFECTS OF SOLUTION COLLECT DATA AFTER SOLUTION IMPLEMENTATION CONVERT DATA TO MONETARY VALUE 3. APPLICATION/ IMPLEMENTATION 4. BUSINESS IMPACT REPORTING CALCULATE THE RETURN ON INVESTMENT 5. ROI EVALUATION FRAMEWORK Level Measurement Focus 1. Reaction, Satisfaction & Planned Action Measures participant satisfaction with the program and captures planned actions. 2. Learning Measures changes in knowledge, skills, and attitudes. 3. Application Measures changes in on-the-job behavior. 4. Business Impact Measures changes in business impact variables. 5. Return on Investment Compares program benefits to the costs. Costs per program (38 participants) R 88, 000 Benefits per program (1 st year) R 230, 000 from 38 participants CAPTURE COSTS OF SOLUTION IDENTIFY INTANGIBLE MEASURES INTANGIBLE BENEFITS GENERATE IMPACT STUDY

Evaluating Training Programs: 4 Levels 4: Final Results 3: Change in B. V. Behavior 2: KSA Learning 1 : Exp Reaction

Relationship Between Levels Level 4 - Results Was it worth it? Level 3 - Behavior KSA being used on the job? Level 2 - Knowledge Did they learn anything Level 1 - Reaction Was the environment suitable for learning? • Each subsequent level is predicated upon doing evaluation at lower level • A Level 3 will be of marginal use, if a Level 2 evaluation is not conducted

Types of Assessments Used at Each Level 4 - Results Type Form Summative Correlation of business results with other assessment results Was it worth it? Level 3 - Behavior KSA being used on the job? Level 2 - Knowledge Did they learn anything Level 1 - Reaction Was the environment suitable for learning? Summative Observation of Performance 360° Survey Diagnostic Summative Self-assessment Test Reaction Formative Survey Real-time Polling Quizzing

Levels of Measurement Key Business Results Competencies Learning and performance solutions Level 4: Business Impact Did the applied knowledge and skills (solution) positively influence the organization’s business measures? Level 3: Application in Work Setting Level 1: Reaction/ Planned Actions Were the participants satisfied with the program and what do they plan to do differently? Level 5: Return on Investment Did the benefits exceed the financial investment? Were the participants able to apply the knowledge and skills (solution) on the job? Level 2: Learning What knowledge, skills or attitudes improved as a result of the program?

Activity: Identify training eg’s that your organisation has invested in Were these evaluated?

Applying ROI Methodology

The ROI Process EVALUATION PLANNING Develop Objectives of Solution Develop Evaluation Plans and Baseline Data DATA COLLECTION Collect Data During Solution Implementation Collect Data After Solution Implementation 1. Satisfaction Reaction 3. Application / Implementation 2. Learning 4. Business Impact

The ROI Process Provides 6 types of data Tabulate Costs REPORTING DATA ANALYSIS Isolate the Effects of the Solution Convert Data to Monetary Values Calculate the Return on Investment 5. ROI Identify Intangible Measures Intangible Benefits Generate Impact Study

Results Chain • Link between “problem” impact and training decision • What should happen after the training? • And then what…. • Etc…

Results Chain What would you expect / like to see happening because of this training? Your training example And then what? And then what? And then what?

Example: Attend Fasset ROI Seminar Increase in understanding & knowledge on ROI Application at work = skills Sharing of new info & skills with colleagues Business more aware of potential impact of training decisions Added value to business bottom line increases Business makes more strategic performance related training decisions Performance and productivity improves Training interventions better planned, resourced and evaluated Business more successful

Activity: Develop a Results Chain for a training example from your organisation

Indicators for measurement • • Baseline Input Process Output Outcome Impact Composite

Example: Attend Fasset ROI Seminar Increase in understanding & knowledge on ROI Business more aware of potential impact of training decisions Added value to business bottom line increases OUTCOME Business makes more strategic performance related training decisions Performance and productivity improves OUTPUT Application at work = skills Sharing of new info & skills with colleagues OUTCOME Training interventions better planned, resourced and evaluated OUTCOME Business more successful IMPACT

Activity: Identify the levels of RESULTS for your training example

Indicators for ROI Rules for designing indicators: 1. Indicators are always written as a number 2. Indicators are neutral (no change in direction) 3. Indicators must be very specific and unambiguous 4. Indicator can only measure one result at a time

Example: Attend Fasset ROI Seminar Increase in understanding & knowledge on ROI Application at work = skills Sharing of new info & skills with colleagues Business more aware of potential impact of training decisions Added value to business bottom line increases Business makes more strategic performance related training decisions Performance and productivity improves Training interventions better planned, resourced and evaluated Business more successful

Activity: Design indicators for the ROI training example

Example indicators: Increase in understanding & knowledge on ROI • Number of staff attending ROI session • Number of staff passing knowledge assessment • % increase in self-assessment (before & after training)

Example indicators: Application at work = skills • Number of ROI tools used • Number of staff using ROI tools

Example indicators: Sharing of new info & skills with colleagues • Number of staff sharing knowledge • Number of formal info sharing sessions held • Number of staff able to work on ROI that did not attend training session

Example indicators: Training interventions better planned, resourced and evaluated • R-value of training planned for period • R-value of training implemented for period • R-value of training programmes evaluated

Example indicators: Business makes more strategic performance related training decisions • Number of strategic training related decisions made by business • R-value of rewards provided to staff for performance improvement • R-value of training linked directly to business objectives

Example indicators: Business more aware of potential impact of training decisions • Number of strategic business decisions impacted by ROI results • R-value of training decisions made at strategic level • R-value of resources allocated to ROI processes

Example indicators: Added value to business bottom line increases • R-value of ROI budget • No of strategic meetings where ROI results discussed • No of strategic decisions influences by ROI results • No of staff having ROI as part of their responsibility

Example indicators: Performance and productivity improves • No of staff scoring above average on performance ratings in period • No of staff receiving performance rewards • R-value of performance rewards • No of units produced / completed • R-value of units produced / invoiced • Turn around time on client requests • R-value of time spent on client requests • No of complaints received • No of errors picked up • R-value of errors corrected

Example indicators: • • Business more successful % increase market share (R-value? ) % increase in client base (R-value? ) R-value of new accounts generated R-value of turnover for period R-value of new referrals (quotes) No of current clients maintained (R-value) No current client portfolios extended (R-value)

Data Collection and Analysis for ROI

Types of methods Quantitative methods…. . . Answer the question, “How many? ”. . . Are used when a number, rate, ratio or proportion is required … Directly measure the status or change of a specific variable, for example changes in crop yield, kilometres of road built over a period of time, or number of hours per week that women spend fetching water

Types of methods Qualitative methods…. . . Answer questions such as, “Why? ” and “How? ” … Are used when the attitudes, beliefs, perceptions and experiences of the target population are required to understand its reactions and responses to interventions … Gather data by asking people to explain what they have observed, do, believe or feel

Types of data Primary Data… • Collection is necessary when a researcher cannot find the data needed in secondary sources • Primary data can be qualitative or quantitative depending on what methodology was chosen Common Primary Data Collection Methods for Monitoring • Observations • Interviews • Groups- large and small • Surveys

Types of data Secondary Data… • Refers to data that already exists and is available from national and local agencies, academic institutions or published in papers and books (e. g. , reports, maps, photographs, diagrams) Possible limitations of using secondary data • Inadequacy • Potential for poor quality • Variation in concepts • Data may be outdated • Inaccessibility

The Four Major Categories of Hard Data Ou Inc tp rea ut se s e m Ti ings v a S ty i l a e u Q vem ro p Im Primary Measurements of Improvement “Hard Data” nt Co Sa st vin gs

Examples of Hard Data OUTPUT • Units Produced • Tons Manufactured • Items Assembled • Money Collected • Items Sold • Forms Processed • Loans Approved • Inventory Turnover • Patients Visited • Applications Processed • Students Graduated • Tasks Completed • Output Period • Productivity • Work Backlog • Incentive Bonus • Shipments • New Accounts Generated TIME • Equipment Downtime • Overtime • On Time Shipments • Time to Project Completion • Processing Time • Supervisory Time • Break in Time for New Employees • Training Time • Meeting Schedules • Repair Time • Efficiency • Work Stoppages • Order Response • Late Reporting • Lost Time Days COSTS • Budget Variances • Unit Costs • Cost By Account • Variable Costs • Fixed Costs • Overhead Cost • • • QUALITY • Scrap • Waste • Rejects • Error Rates • Rework • Shortages • Product Defects • Deviation From Standard • Product Failures • Inventory Adjustments • Time Card Corrections • Percent of Tasks Completed Properly • Number of Accidents Operating Costs Number of Cost Reductions Project Cost Savings Accident Costs Program Costs Sales Expense

Major Categories of Soft Data Ini tia tiv e Work Habits Typical Measures of Improvement “Soft Data” t/ n e t m n p e o l ve cem e D van Ad New Skills k or te W a im l C Fe At elin tit gs ud / es

Examples of Soft Data WORK HABITS • Absenteeism • Tardiness • Visits to the Dispensary • First Aid Treatments • Violations of Safety Rules • Number of Communication Break-downs • Excessive Breaks • Follow-up WORK CLIMATE • Number of Grievances • Number of Discrimination Charges • Employee Complaints • Job Satisfaction • Employee Turnover • Litigation FEELINGS/ATTITUDES • Favorable Reactions • Attitude Changes • Perceptions of Job Responsibilities • Perceived Changes in Performance • Employee Loyalty • Increased Confidence NEW SKILLS • Decisions Made • Problems Solved • Conflicts Avoided • Grievances Resolved • Counseling Problems Solved • Listening Skills • Interviewing Skills • Reading Speed • Discrimination Charges Resolved • Intention to Use New Skills • Frequency of Use of New Skills DEVELOPMENT/ADVANCEMENT • Number of Promotions • Number of Pay Increases • Number of Training Programs Attended • Requests for Transfer • Performance Appraisal Ratings • Increases in Job Effectiveness INITIATIVE • Implementation of New Ideas • Successful Completion of Projects • Number of Suggestions Submitted • Number of Suggestions Implemented • Work Accomplishment • Setting Goals and Objectives

Activity: Identify the data sources for your own training example

Example: Attend Fasset ROI Seminar Increase in understanding & knowledge on ROI Application at work = skills Sharing of new info & skills with colleagues Business more aware of potential impact of training decisions Added value to business bottom line increases Business makes more strategic performance related training decisions Performance and productivity improves Training interventions better planned, resourced and evaluated Business more successful

ROI Calculations

ROI Calculation Steps: 1. 2. 3. 4. 5. Create an ROI measurement plan Collect data Isolate the effects of training Covert data to monetary value Calculate ROI

ROI Calculation: Example Net Program Benefits ROI = Program Costs Benefits per program (1 st year) R 230, 000 from 25 participants Costs per program (25 participants) R 88, 000 ROI = - = x 100 = %

R-values to work with: Benefits • R-value of training planned for period (R 200 000) • R-value of training implemented for period (R 120 000) • R-value of training programmes evaluated (R 80 000) • R-value of rewards provided to staff for performance (R 350 000) • R-value of training linked directly to business objectives (R 80 000) • R-value of training decisions made at strategic level (R 80 000) • R-value of resources allocated to ROI processes (R 25 000) • R-value of ROI budget (R 25 000) • R-value of units invoiced (R 2 000) • R-value of time spent on client requests (R 800 000) • R-value of errors corrected (R 550 000) • R-value of new accounts generated (R 800 000)

R-values to work with: Costs • R-value of payments to training providers (R 120 000) • R-value of accom/travel costs for training (R 120 000) • R-value of training venue/catering (R 60 000) • R-value of staff man hours while on training (R 200 000) • R-value of hours could be billed to client (R 600 000) • R-value of errors occurring while staff on training (R 60 000)

ROI Calculation: Formula Net Program Benefits ROI = Program Costs ROI = - = x 100 = %

Activity: Calculate the ROI of the ROI training example

Activity: What does your ROI result tell you? (What does the business need to know & do)

ROI Challenges

ROI Dilemmas • If the CEO/top executive requires ROI, it may be too late for the process • The time to pursue ROI is when you do not have to pursue it • The ROI process is not a quick fix • If training staff do not see the need for ROI, it will usually fail • Without cooperation of participants, the ROI process will usually fail • Without the support of management, the ROI process will usually fail • The ROI process requires much education with a variety of groups

Implementation Concerns • Resources (staffing / budget) • Leadership (individual, group, cross functional team) • Timing (urgency, activities) • Communication (various audiences) • Commitment (staff, managers, top executives)

ROI Lessons from practice

Tips for HR Managers • Understand that business needs will change over the duration of the project. The longer it takes to do a project, the more likely that change will occur. Be prepared to switch gears or re-tool. Stay alert for indications of learning gaps or obsolescence of course materials created by changes in technology, economic factors, or employee demographics • Anticipate and plan for glitches. They will happen. Do not get discouraged. Deal realistically with unexpected events and surprises. Some of them will be problems that may reduce expected outcomes. Others may stimulate creativity and produce very positive results

Tips for HR Managers • Keep an eye on costs and monitor any deviations from estimated or budgeted expenditures. Quick action or adjustments may enable the HR manager to curtail or reduce costs and keep the project within or near budget • Balance short term and long-term goals. The pressure to deliver consistent short-term financial performance may limit the resources available for investment in growth opportunities, such as the "development" of employees for future roles • Use the experience of senior managers and employees in estimating the results of training. Estimating requires a little math and lots of experience

Tips for HR Managers • Secure managers' and supervisors' support for training. Effective training depends on three persons: the trainer, the trainee and the supervisor or manager. All three must agree on expected outcomes and how and when they will be measured • Align training initiatives with strategic business plans. Training should improve your organization's competitive edge, which may be efficiency, first class service, or creativity and innovation • Track training costs by employee for career pathing and succession planning

Final Advice Using ROI as the “bottom line” may not be the best criteria for determining the value of training. It can be a good indicator, but it may not be the indicator

What Next? Action Items To Implement A Comprehensive ROI Process • Assess your progress and readiness. • Set targets for evaluation levels. • Develop a policy and philosophy statement on results-based approach. • Develop guidelines • Clarify roles of: • Staff • Build staff skills • Establish a management support system • Employees • Enhance management commitment and support. • Supervision and management • Communicate results to selective audiences • Establish accountability • Develop a transition plan • Establish a quality review process.

Useful website: www. roiinstitute. net

ROI: Is it complex or is it simple?

Thank you …. Thank You Fasset Call Centre 086 101 0001 www. fasset. org. za

Thank you …. Head Office (Cape Town): 206 Rosmead Ave, Wynberg, 7800, PO Box. 457, Plumstead 7801 Gauteng: Postnet Suite 469, Private Bag X 51, Bryanston. Ph: (021) 762 - 5742 Fax: (021) 762 – 9745 Email: marketing@farananiservices. co. za http: //www. farananiservices. co. za (PTY) LTD 2007/019026/07 Accreditation No: Services Seta: 0866 Directors: Benedict Pillay & Deborah Williams