How to Measure Impact of Health Leadership Development

How to Measure Impact of Health Leadership Development Speakers: Dr. Graham Dickson, CHLNet Suzanne Schell, ROI Institute Canada Moderator: Kelly Grimes October 26, 2017 CHLNet/LEADS Collaborative Webinar Series

2 Dr. Graham Dickson CHLNet Senior Policy Advisor

3 CHLNet ROI Study Partners Ø 15 Partners: ROI Institute Canada, Manitoba Centre for Healthcare Innovation, Canadian College of Health Leaders, Health. Care. CAN, Canadian Institute for Health Information, Canadian Medical Protective Association, Canadian Nurses Association, Canadian Society of Physician Leaders, Joule/CMA, Accreditation Canada, University of Ottawa, Ontario Hospital Association, Alberta Health Services, Alberta Innovates – Health Solutions, and Canadian Foundation for Healthcare Improvement. Ø Looking for More Partners!

4 ROI Steering Group Ø To provide stewardship of the entire Return on Investment (ROI) project over its duration to ensure it achieves its goals and is conducted in an efficient manner (i. e. within budget). Ø Core Group: CHLNet Senior Policy Advisor, CHLNet Executive Director, Network Partner ROI Funder Reps, ROI Experts

5 LDI Assessment Toolkit Principles Ø Evidence-based Ø Simple to use Ø Customizable Ø Economical

6 LDI Study Phases Pre-Phase Funding (May to Dec. 2015) Phase I Scoping Review (Jan. to May 2016) Phase II Design Instrument and Pilot Test (September 2017 to May 2018) Phase III Finalize Instrument and Implement (June 2018)

7 ROI Scoping Review Three objectives: 1. ROI determinants associated with healthcare leadership quality/style. 2. ROI determinants associated with leadership development programs/tactics. 3. ROI determinants associated with existing ROI tools. Evidence Brief and Journal Publication www. chlnet. ca/tools-resources/research

8 Rethink of Approach Ø Staged approach Ø LD defined: Ø “Deliberate, structured learning interventions aimed a growing or enhancing individual and organizational leadership capability. ” Ø LD Investment goals: Ø Improve individual leadership capability Ø Collective leadership capacity

9 Three LDI User Options Ø Automated Ø Partnership with ROI Institute Canada Ø Companion Document on Designing an Impactful LDI

10 Suzanne Schell ROI Institute Canada Proving value, impact and ROI

Chain of Impact 11 Reaction & Planned Action Learning Application & Implementation Isolate the Effects of the Program Impact Intangible Benefits ROI Institute Canada

1. Reaction To and Satisfaction with the Program ROI Institute Canada 12

2. New Skills, Knowledge and Competencies Acquired - “Capability” ROI Institute Canada 13

14 3. Application and Implementation Barriers Enablers ROI Institute Canada

15 4. Impact – Tangible and Intangible ROI Institute Canada

16 ROI – The Calculation The finance and accounting calculation ROI (%) = Net Program Benefits Program Costs X 100 Example Net Program Benefits = $38, 776 Program Costs = $32, 984 ROI =117% ROI Institute Canada

17 6. Intangible Measures, Not Converted to Money ROI Institute Canada

18 Chain of Impact Reaction & Planned Action ROI Institute Canada Learning Application & Implementation Isolate the Effects of the Program Impact Intangible Benefits

Measuring Impact What level will you evaluate your leadership program? Level 1 – Short programs, briefings, policy introductions Level 2 – When participants need to acquire specific knowledge or skills – for example a CPR course Level 3 – Essential to ensure the behaviour change in the workplace is occurring to create the desired impact Level 4 – Proves the value the program is adding to the organization ROI Institute Canada 19

20 Criteria for Level 4 or ROI Ø Expected life cycle Ø Linkage to strategic objectives Ø Cost Ø Time commitment Ø Visibility Ø Management/executive interest Ø Client requirements ROI Institute Canada

21 LDI Tools to Evaluate Tool kit gives you access to: Ø Online tools Ø Downloadable tools Ø How to use sections ROI Institute Canada

22 Developing Objectives ROI Institute Canada

23 Evaluation Planning ROI Institute Canada

24 Data Collection ROI Institute Canada

Isolating the Effects of the Program, Project, Initiative • Case Application ROI Institute Canada 25

26 Convert Data to Monetary Values ROI Institute Canada

27 Measuring ROI Institute Canada

28 Pilot Test: A Fantastic Opportunity!! Ø Diversity of program, i. e. geography, size of organization, type of organization, interprofessional vs. profession specific, and executive vs. front line leaders Ø Timing of program, i. e. fall 2017 Ø User option desired, i. e. LD outcomes, LD impact, ROI Ø Overall LD investment in last three years, i. e. robust vs. new endeavour Ø In-kind contribution, i. e. willingness to devote staff time and resources to pilot test including write-up as a case study

29 Your Feedback Required Ø Does the new phased approach make sense and would it be useful for your organization to generate evidence? Ø Any issues or red flags we should consider as you think about it for your organization? Ø Any advice for us in pilot testing? And would you want to be part of this fantastic opportunity?

30

31 Next Steps and Thank You! For more information: www. chlnet. ca

Please take 2 minutes to complete a survey at www. questionpro. com/t/AEeu. VZayv 5 Next Webinar: February 2018 on Cultural Transformation chlnet. ca leadscanada. net