Introduction to Executive Reporting for CLOs Tamar Elkeles

Introduction to Executive Reporting for CLOs Tamar Elkeles Vice President of QUALCOMM’s Corporate Learning Center Version: 26 -Feb-11

Learning measurement: Key performance indicators Does your organization have a standard set of key performance indicators that you measure and monitor regularly? Yes 34% No 66% Source: Knowledge. Advisors Learning Analytics Best Practices Research Study, 2004. © 2011 Knowledge. Advisors 2

The Measurement Gap for CEOs We current measure this We should measure this 100% 80% 60% 40% 20% 0% Activity Cost Reaction Learning Application Impact ROI Awards 2009 ROI Institute Study based on responses from 96 CEOs © 2011 Knowledge. Advisors 3

The Idea is Born § Conversation between Kent and Tamar in Spring 2010 § Executive Council created to provide guidance § Dave Vance asked to provide leadership § Industry thought leaders have provided advice, insight, and further guidance © 2011 Knowledge. Advisors 4

Executive Council Members • • • Tamar Elkeles, Qualcomm, Chair Terry Bickham, Deloitte Cedric Coco, Lowe’s Marilyn Figlar, Lockheed Martin Karen Kochler, CIGNA Sundar Nagarathnam, Net App Sandy Shaw, Sodexo Don Shoultz, BP Tom Simon, CNA Insurance Carrie Beckstrom, ADP © 2011 Knowledge. Advisors 5

Industry Thought Leaders Providing Advice and Counsel to the Project § § § Laurie Bassi Josh Bersin Rob Brinkerhoff Jack Phillips Frank Anderson Jac Fitz-enz © 2011 Knowledge. Advisors 6

The Reporting Effort Goals § Practical, relevant guidance for reporting in the talent development field § Standard measures and definitions § Standard reports § Adopted by many organizations and becomes the standard for the industry § Greater clarity and efficiency for each organization § Ability to compare across the industry © 2011 Knowledge. Advisors 7

The Three Phase Plan Phase I Basic Reporting for L&D Phase II Expanded Reporting for L&D Phase III Reporting for Talent Development Focus of today’s presentation: progress on Phase I © 2011 Knowledge. Advisors Page: 8

Executive Reporting for CLO’s: What They Want and How to Get It Dave Vance President, Manage Learning LLC

The Need § Clear guidance for L&D § Reporting § Data measures and definitions § Similar to what GAAP provides for accounting § Generally Accepted Accounting Principles § Reports: Income Statement, Balance Sheet and Cash Flow Statement § Measures and definitions: Income, expenses § How to calculate and report § Examples: Depreciation, value of an asset © 2011 Knowledge. Advisors 10

Benefits Talent Development Reporting Principles (TDRP) § Clear, practical reporting guidance for CLOs to use within their own organization § Manage the L&D function to produce desired outcomes effectively and efficiently § Provide appropriate, meaningful reports to the CEO § Standard measures and definitions to benchmark and learn from other organizations © 2011 Knowledge. Advisors 11

Phase 1 Deliverable Reflects Efforts of Many § § Kent’s vision and commitment Executive Council input and support Thought leader’s guidance and counsel Key leaders at Knowledge Advisors § Peggy Parskey § Jeffrey Berk § Kendall Kerekes © 2011 Knowledge. Advisors Page: 12

Overview § Two primary audiences for reporting § CLO, including managers and staff § CEO, including governing boards and senior leaders § Four recommendations § Adopt a set of guiding principles § Adopt standard measures and definitions § Adopt three standard statements § Adopt three standard reports © 2011 Knowledge. Advisors 13

Focus of the Executive Reporting Process Summary Conclusions, Actionable Recommendations, Issues for Further Analysis Scope of White Paper Reports Statements Program/ Initiative Analytics Specific methodology (e. g. Phillips, Brinkerhoff) Standard Measures Data Sources and Systems System Wide Analytics Systems and processes for organizing data, calculating measures

Data Aggregation and Reporting High Scope of White Paper Summary Reports Summary Statements Aggregation of Data Standard Measures Detailed Reports Detailed Statements Low Level of Customization © 2011 Knowledge. Advisors High Page: 15

The Assumptions § L&D broadly defined: All programs and initiatives designed to increase human capital to enable greater organizational and personal capacity, effectiveness, and efficiency § Primary purpose of L&D: Enable organization to achieve its goals § Learning is strategically aligned to goals § Recommended reports used appropriately © 2011 Knowledge. Advisors Page: 16

The Eight Principles 1. Employ concise and balanced measures reported in a consistent and clearly defined format 2. Produce and communicate Executive Reporting with a frequency and thoroughness to enable appropriate management of the function 3. Include actionable recommendations in all Executive Reporting 4. Maintain data integrity and completeness © 2011 Knowledge. Advisors Page: 17

The Eight Principles 5. Employ appropriate analytical methods 6. Provide the impact and value or benefit of programs and initiatives whenever appropriate 7. Capture and report the full costs of L&D 8. Support continuous improvement of executive reporting and the underlying data bases © 2011 Knowledge. Advisors Page: 18

Executive Reporting Process Executive Reports Business Executive Summary Report Learning Executive Reports L&D Program Report L&D Operational Report Statements Business Outcomes Statement Learning Effectiveness Statement Learning Efficiency Statement Data Sets Business Outcomes Effectiveness Efficiency Guiding Principles L&D Summary Report Extract, convert and calculate Standard Measures Data Sources Financial Data Learning Management System © 2011 Knowledge. Advisors Evaluation System Other Sources (e. g HRIS, ERP, CRM) Page: 19

Standard Measures § Business Outcomes § Organizational goals § L&D’s impact on those goals § Effectiveness § Levels 1 -5, timeliness § Efficiency § § § Costs Volume (participants, programs, classes, hours) Ratios Utilization rates Program and vendor management © 2011 Knowledge. Advisors 20

The Statements § Three standard statements § Business Outcome § Effectiveness § Efficiency § Each statement shows § Last year’s actual § Plan (or goal) for this year § Year-to-date results § Standard measures are used but choice of measures depends on organization © 2011 Knowledge. Advisors 21

Business Outcome Statement © 2011 Knowledge. Advisors 22

Effectiveness Statement May 25, 2021 © 2011 Knowledge. Advisors 23

Efficiency Statement May 25, 2021 © 2011 Knowledge. Advisors 24

The Reports § Three levels of reports § L&D Summary Report § L&D Program Report § L&D Operations Report § Highly customized, pulling measures from the statements § Each report shows § § Last year’s actual Plan (or goal) for this year Year-to-date results Forecast for this year © 2011 Knowledge. Advisors 25

Data Aggregation and Reporting High Scope of White Paper Summary Reports Summary Statements Aggregation of Data Standard Measures Detailed Reports Detailed Statements Low Level of Customization © 2011 Knowledge. Advisors High Page: 26

L&D Summary Report May 25, 2021 © 2011 Knowledge. Advisors 27

L&D Program Report May 25, 2021 © 2011 Knowledge. Advisors 28

L&D Operations Report May 25, 2021 © 2011 Knowledge. Advisors 29

Summary: Talent Development Reporting Principles § TDRP provides § § Guiding principles for reporting Standard measures and definitions Standard statements Executive reports § Benefits § Provide industry guidance on reporting § Support better management of the function § Support benchmarking and cross company comparisons © 2011 Knowledge. Advisors 30

Next Steps § Continue to share and socialize TDRP § Webinars § Workshops § Industry begins to implement § Refine and expand the effort for L&D (Phase II) § Extend the effort to the rest of talent development (Phase III) § This is an open, collaborative effort; no copyrights © 2011 Knowledge. Advisors 31

Conclusion § Please share your thoughts with us § § Other suggested measures Difficulty in applying standard measures Need for better definitions Write to Dave Vance: dave@poudrerivergroup. com § At www. Knowledge. Advisors. com § This Power. Point deck § White Paper (70 -page) § Overview Paper (25 -page) © 2011 Knowledge. Advisors 32