10 Steps for Developing Safety Performance Measures Indiana

10 Steps for Developing Safety Performance Measures Indiana Safety & Heath Conference February 28, 2017 Barry S. Spurlock, Esq, CSP Earl H. Blair, Ed. D, CSP Blair & Spurlock, LLC

Step 1. Evaluate, Inventory and Prioritize • Trends • Critical Safety Initiatives • High Impact Drivers 2

Tools for Identifying and Prioritizing Potential Measures Perception Surveys Risk Assessment Injury/Loss Data Analysis Trending/Benchmarking Pareto Principle 3

Pareto – The 80/20 Rule #/Rate of Incidents 5 4 3. 5 3 2. 5 2 1. 5 1 0. 5 0 #/Rate of Incidents Failure to Lack of PM Walking on Failure to Wear PPE Use Lift on Lift Pallets Secure Assist Product

Pareto – The 80/20 Rule % of Injuries/Rate By Cause 100 90 80 70 60 50 40 30 20 10 0 33. 3 25. 9 20. 7 11. 1 Walking on Pallets 9 Lack of PM on Failure to Use Failure to Wear Failure to Assists Lift Assist PPE Secure Product

Step 2: Verify Efficacy of Hazard Controls ID Obstacles • Ensure Efficacy of Existing Controls • Implement Needed Controls • Identify System and Cultural Blocks 64

“Before becoming a KPI, a performance measure needs to be tested to ensure that it creates the desired behavioral outcome…” [emphasis added] Parmenter, David. Key Performance Indicators; Developing , Implementing, and Using Winning KPIs, 3 rd Ed. , 2015, Wiley, New York, 2015.

Key Questions • Does the intervention/hazard control really work? • Does measuring the control drive superior performance? • Is the measure a focal point?

Most Common Drivers of Safety Performance - 2015 ASSE Seminar. Fest Management Led Stretching Prgm/ Stretching; 2 Employee Accountability; 2 Safety Communications; 2 PPE; 2 JHA/JSA/THA/Pre Task Planning; 9 SOPs/Manuals/Policies; 2 Preventive Maintenance Activities; 3 Inspections; 9 Safety Meetings; 3 Audits; 4 Training; 8 Management Support/Engagement; 4 Near Miss/Near Hit; 4

Tools • • • Pareto Analysis Perception/Climate Surveys Performance Ratios SPC Correlation

Training Evaluation Instruments Level One – Reaction Level Two - Learning Results Level Three – Application Level Four – Business Impact & Return on Investment adapted from Phillips & Stone

Value Training Evaluation Level of Training Evaluation Sophistication 66

Measuring Knowledge Change KC = Ka Kb KC = Knowledge Change Ka = Knowledge after training Kb = Knowledge before training 67

Evaluating Training: Knowledge Transfer Ratio = Post Training Score – Pre Training Test Score Possible Score – Pre Test Score

Measuring Skill Change SC = Sa Sb SC = Observable change in skill as a result of training Sa = Skill demonstration after training; output / critical items Sb = Skill before training based on same criteria as Sa Fitz-enz 68

Measuring Attitude Change AC = Aa Ab AC = Attitude Change Aa = Attitude after training Ab = Attitude before training Fitz-enz 69

Measuring Performance Change PC = Pa Pb PC = Performance Change Pa = Performance after training Pb = Performance before training Fitz-enz 70

SPC UCL mean LCL

Correlations

Correlations – The Numbers Strong Negative -1 No Correlation 0 Strong Positive +1

Correlations 16 14 12 10 Safe Behavior 8 Incidents 6 4 2 0 Qtr 1 Qtr 2 Qtr 3 Qtr 4 Qtr 5 Qtr 6

Correlations 12 10 8 Safe Behavior 6 Incidents 4 2 0 Qtr 1 Qtr 2 Qtr 3 Qtr 4 Qtr 5 Qtr 6 Qtr 7 Qtr 8

Step 3: Consider Metric Owners w/in Organization • Top Management • Middle Management • Front Line Supervisor • Employee 59

Organizational Level MACRO Middle management MACRO / micro Front Line Supervisors macro / MICRO Employees More Activity Oriented More Results Oriented Top Management MICRO Petersen 60

Organizational Level Need for Engagement MACRO Middle management MACRO / micro Front Line Supervisors macro / MICRO Employees Need for Data Collection Integrity Top Management MICRO Petersen 60

Top Management MACRO Middle management MACRO / micro Front Line Supervisors macro / MICRO Employees More Timely Feedback Ability to Achieve Statistical Reliability Organizational Level MICRO Petersen 60

Level of Org. Chart Micro or Macro Focus Activity or Outcome Ability for Statistical Accuracy Need for Data Integrity Need for Employee Timely Engagement Feedback Top Mgmt. Macro Outcome High Lower Middle Mgmt. Macro w/ some micro Mix Moderate Frontline Sup. Micro w/ some macro Mix Moderate Prod. / Rank Micro Activity Lower High

4 Determine Simple List of Measures Prioritize Your Most Important Safety Metrics 71

Simple Measures Case Study You Can’t Measure Everything at Once! Jack Toellner 72

Select and Customize a Short List of Measures Identify Three to Five Key Measures 30

Gulf Platform Case Study: Hoover-Diana Project Specific measures: 1. 2. 3. 4. 5. Safety meetings Housekeeping Barricade performance JSA Safety walks Jack Toellner 31

Results (Outcome Measures) Hoover-Diana Platform Project: • 2 Mil. Hrs of Work • 1 Recordable Injury 32

5. Identify Means for Employee Engagement 33

Large Brewery: SMP Measures Employee Engagement 34

Important Distinctions about SMP • Scorecards: Employees are “score-carded” for safety • Quotas: Employees are given a quota for safety activities, however… • Choices: Employees have a choice on which safety activities they will participate in (Coors considers this a key factor in their success) Anne Bevington 35

36

Individual Participation 60 pts • • • • • Observation Cards Job Safety Analyses JSA Training JSA Auditing Safety Meetings Safety Audits Maintenance Walkthroughs Pre-Shift Stretching IH Sampling Requests IH Sampling Results Ergonomic Assessment Requests Ergonomic Assessment Actions Project Walkthroughs Safety Work Orders Incident Reviews Safety Visual Aids Hazard Alerts Individual Safety Initiative Housekeeping Audit 37

SMP Results (Trailing Indicators) • Average SMP Score for plant 83 • 1 Million Hours w/o Lost Time July 2004 and again in March 2005 • LWCIR 2005 < 1 • Total Case 2005 - 2. 4 Anne Bevington 38

6. Develop Methods and Tools • • Checklists Audits Observations Surveys Scorecards Dashboards Audit Reports Inspections 81

Example Safety Scorecard Safety Work Orders # or % Completion # or % Completed on Time Safety Coaching # of Observations % Safety Committee Activity # of Meetings # of Audits # Completed Projects Supervisor Activity # or % Training Completed # or % JSA Completed/ Updated Housekeeping Cleanliness Score Maintenance Score 82

7. Develop Delivery Strategies Audience Consideration Impact Frequency 83

8. Set Performance Goals • • Specific Measurable Actionable Realistic Timely Aim High Employee Engagement 84

Alcoa Case Study Demonstrates Meeting Compliance Goals Measures Critical Processes 100% Compliance Expected Monitored Regularly with Feedback 85

Compliance Case Study: Requires 100% Compliance Measures Critical Four Safety Processes LOTO CSE Fall Prot. Mobile Eqpt. Must have 100% Compliance to be Acceptable Monitors Regularly and Provides Timely Feedback Paul Woerz 44

Compliance Example: Leading Indicators 45

9. Monitor Safety Progress • Look for Correlations with Injury Trends • Engagement of Employees • Improved Systems 87

10. Adjust and Modify for Continuous Improvement 88

Concluding Activity Directions: We would like to conclude with an opportunity for you to draft a blueprint for your organization. You can ask questions from the Instructors or other Participants. 1. Do you have a safety measure you use at your organization but you are uncertain if it is effective? Would you like some feedback? 1. Would you like to share a success story regarding a safety measure that drives success that other participants might find helpful? 48

Questions • earl@blairspurlock. com • barry@blairspurlock. com – (502) 595 -7227

Appendix A REFERENCES AND READING

References & Recommended Reading ANSI Z 10 – 2012. American National Standard for Occupational Health and Safety Management Systems, American National Standards Institute and American Industrial Hygiene Association, Alexandria, VA: AIHA, 2012. (Note: ASSE is now the Secretariat of ANSI Z 10 and holds the copyright) Bevington, Anne M. , 2005. “Safety Management Process – Proactive Safety Metrics that Drive Performance in Manufacturing Facilities. ” Proceedings of the American Society of Safety Engineers 2005 Professional Development Conference, New Orleans Blair, Earl & Geller, E. Scott Becoming World Class in HSE Management, Occupational Health & Safety, September 2000. Blair, Earl & O’Toole Michael F. , Leading Measures for Safety Performance: One Way to Enhance Your Organization’s Culture, Professional Safety, August, 2010. Blair, Earl & Seo, D. C. Safety Training: Making the Connection to High Performance , Professional Safety, October, 2007. Blair, E. H. & Spurlock, Barry S. , “Leading Measures for Safety Performance, ” Proceedings of the American Society of Safety Engineers 2008 Professional Development Conference, Las Vegas: ASSE, 2008. CDC & Prevention, 2001. Guide to Evaluating the Effectiveness of Strategies for Preventing Work Injuries: How to show whether a safety intervention really works, NIOSH & Institute for Work & Health. Authors: Linda M. Goldenhar, Andrew R. Hale, Lynda S. Robson, Harry S. Shannon Cooper, Dominic, 2009. Behavioral Safety: A Framework for Success, BSMS. Daniels, Aubrey C. & Daniels, James E. Performance Management: Changing Behavior That Drives Organizational Effectiveness, Performance Management Publications, 2006. 51

References & Recommended Reading Daniels, Aubrey C. , Bringing Out the Best in People: How to Apply the Astonishing Power of Positive Reinforcement, Mc. Graw-Hill: 1994 Geller, E. Scott, The Psychology of Safety: How to Improve Behaviors and Attitudes on the Job, Chilton: 1996 Franklin. Covey: The 4 Disciplines of Execution, 2006. Janicak, Christopher A. Safety Metrics: Tools and Techniques for Measuring Safety Performance, Second Edition, Government Institutes, 2010. Krause, Thomas R. , Safety Incentives from a Behavioral Perspective, Ch. 21 in Current Issues in Behavior-Based Safety, BST, 1999 Manuele, Fred A. , 2008. Advanced Safety Management: Focusing on Z 10 and Serious Injury Prevention. John Wiley & Sons. Mathis, Terry L. “ 5 New Metrics to Transform Safety” Workplace HR & Safety, September 22, 2009. Mc. Sween, Terry E. , The Values-Based Safety Process: Improving Your Safety Culture with Behavior-Based Safety (2 nd Ed. ) John Wiley & Sons: 2003 Niven, Paul R. , Balanced Scorecard Step-By-Step: Maximizing Performance and Maintaining Results, John Wiley & Sons, 2002. Parmenter, David, Key Performance Indicators: Developing, Implementing, and Using Winning KPIs, Third Edition, Wiley & Sons, 2015. Petersen, Dan, 2005. The Measurement of Safety Performance, American Society of Safety Engineers. Pfeffer, Jeffrey & Robert Sutton, The Knowing-Doing Gap: How Smart Companies Turn Knowledge Into Action, Harvard Business School Press, 2000. 52

References & Recommended Reading Phillips, J. J. & Stone, R. D. , How to Measure Training Results: A Practical Guide to Tracking the Six Key Indicators, Mc. Graw-Hill, 2004. Seo, D. C. , Torabi, M. R. , Blair, E. H. & Ellis, N. T. , “A cross-validation of safety climate scale using confirmatory factor analytic approach, ” Journal of Safety Research, Vol. 35, 427 – 445, National Safety Council & Elsevier, 2004. Spitzer, Dean R. , (2007), Transforming Performance Measurement: Rethinking the Way We Measure and Drive Organizational Success, New York, NY: American Management Association. Spurlock, Barry S. & Wertz, Keith R. , Chomp Comp: Take a Bite Out of Your Workers’ Comp, Lighted Path Publications, 2009. Stewart, James M. , 2002, Managing for World Class Safety, New York: John Wiley & Sons Swartz, George, 2001. Job Hazard Analysis: A Guide to Identifying Risks in the Workplace. Government Institutes Toellner, Jack, “Improving Safety & Health Performance: Identifying and Measuring Leading Indicators, ” Professional Safety, September 2001. Woerz, Paul , Personal Correspondence, 2006. 53

Appendix B CALCULATIONS

UCL / LCL Calculations for #s of Events / Samples • 95% Statistical Significance = 2 std. deviations from mean = 1. 96 = normal distribution • UCL = x + normal dist. (std. deviation of population) • LCL = x – normal dist. (std. deviation of population) • X = mean • Z = normal distribution • S= Std. Deviation of Population 55

UCL / LCL Calculations for Proportions / % • 95% Statistical Significance = 2 std. deviations from mean = 1. 96 = normal distribution • UCL = p + 1. 96[p(1 -p)/n]ˆ0. 5 • LCL = p - 1. 96[p(1 -p)/n]ˆ0. 5 • p= mean proportion 56

Standard Deviation * If only a sample of population. Standard Deviation = √ Σ (v 1 – mean)² + (v 2 -mean)²…. (n -1) 57

Standard Deviation * If entire population sampled. Standard Deviation = √ Σ (v 1 – mean)² + (v 2 -mean)²…. n 58

Pearson Correlation Coefficient Tip: Use a Calculator with stats functions.