ROI in Agile Transformation Using Design Thinking to

ROI in Agile Transformation: Using Design Thinking to Deliver and Measure Results JACK J. PHILLIPS, PH. D. CHAIRMAN, ROI INSTITUTE

Objectives After completing this session, participants should be able to: • Identify the types of data and measures for Agile software development. • Follow the steps to calculate the ROI on Agile software development. • Use the results from agile programs to influence Agile transformation and increase budget. 2

WHICH OF THESE ARE TRUE? • Most clients see Agile learning as a cost and not an investment. • The Agile learning outcomes desired by clients are rarely measured. • Most Agile coaches and trainers do not have data showing that they make a difference to clients. 3

“The disconnect between the delivery organization and the business is prevalent within the software industry. Somewhere along the line, the real vision behind our projects gets lost. We all know it. ” — Don Mc. Greal VP, Training Solutions Improving Enterprises 4

CEO: What is the ROI of executive coaching? 422 executives being coached by 85 coaches 5

Client: What is the ROI of SAP education? Education Revenue 6

What Executives Want * Currently Measure Should Measure Importance Inputs and Indicators 94% 86% 6 Efficiency 78% 82% 7 Reaction 53% 22% 8 Learning 32% 28% 5 Application 11% 61% 4 Impact 8% 96% 1 ROI 4% 74% 2 Awards 40% 44% 3 Measures *Source: Ninety-six Fortune 500 CEOs. 7

Changing Paradigms for Agile Training Value Traditional Emerging Input Focus Outcome Focus Cost Investment Necessity Value Driver Science of Learning Analytics Needs Assessment Performance Consulting Reaction and Learning Impact and ROI 8

WHAT IS VALUE? 9

*Can predict The Agile Value Chain LEVEL ISSUE This is easy Always measured 0 Inputs This is easy Almost always measured 1 Not difficult Usually measured Possible Often measured MEASURES TARGETS Volume, Hours, Convenience, Cost 100% Reaction *Relevance, Engaging, *Important, Useful, *New Content, *Intent to Use, *Recommend to Others 100% 2 Learning Concepts, Trends, Facts, Contacts, Skills, Competencies 90% 3 Application Use of content, Frequency of Use, Success with Use, Barriers, Enablers 30% Executives prefer Must take a step to isolate the effects Not so difficult to connect Sometimes measured 4 Impact Possible for many programs Rarely measured 5 ROI Productivity, Time, Quality, Costs, Image, Reputation, Engagement, Compliance 10% Benefit Cost Ratio or Return on Investment, Expressed as a Percent 5% *Best Practice: Percent of Programs Evaluated at this level each year. Copyright © 2018 ROI Institute, Inc. No part of this may be reproduced, stored in a retrieval system, or transmitted in any form or by a means without written permission. 10

What is Design Thinking? Basic Principles* 1. A problem-solving approach to handle problems on a systems level 2. A mind-set for curiosity and inquiry 3. A framework to balance needs and feasibility 4. A way to take on design challenges by applying empathy 5. A culture that fosters exploration and experimentation *Mootee, Idris. (2013). Design Thinking for Strategic Innovation. Hoboken, NJ: Wiley. 11

What is Design Thinking? Basic Principles* 6. A fixed process and a tool kit 7. A storytelling process to inspire senior executives 8. A new competitive logic of business strategy 9. A means to solve complex or wicked problems 10. A means to reduce risks *Mootee, Idris. (2013). Design Thinking for Strategic Innovation. Hoboken, NJ: Wiley. 12

Design for Business Results: Use Design Thinking to Deliver Business Value 1. Start with Why: Aligning Programs with the Business 2. Make it Feasible: Selecting the Right Solution 3. Expect Success: Designing for Results 4. Make it Matter: Designing for Input, Reaction, and Learning 13

Design for Business Results: Use Design Thinking to Deliver Business Value 6. Make it Credible: Measuring Results and Calculating ROI 7. Tell the Story: Communicating Results to Key Stakeholders 8. Optimize Results: Using Black Box Thinking to Increase Funding 14

Start with Why Aligning Programs with the Business Design Thinking Principle 1 A problem-solving approach to handle problems on a systems levels • Alignment is the key • Is it a problem or opportunity? • Need specific business measure(s) 15

Alignment Model Start Here Payoff Needs 5 ROI Objectives A C 4 Performance Needs 5 ROI B Impact Objectives Business Needs Initial Analysis End Here 3 Learning Needs 4 Application Objectives 2 Preference Needs 3 Learning Objectives 1 Business Alignment and Forecasting Reaction Objectives Project Impact Application 2 Learning 1 Measurement and Evaluation Reaction The ROI Process Model 16

Make it Feasible Selecting the Right Solution Design Thinking Principle 2 A mind-set for curiosity and inquiry Ask: • What are we doing (or not doing) that’s influencing the business measure? • How can we achieve this performance? 17

Expect Success Designing for Results Design Thinking Principle 3 A Framework to balance needs and feasibility • Set objectives at multiple levels • Redefine success of Agile training and coaching • Expand responsibilities 18

OBJECTIVES 19

Matching Evaluation Levels with Objectives After completing this Agile development program, participants should: 1. Decrease citizen complaints by 20% in one year. 2. Use Agile development skills to react to changes quickly. 3. Be able to demonstrate how to conduct a process review meeting. 4. Decrease the amount of time required for a customer transaction. 5. Achieve a 20% ROI one year after implementation of Agile development. 20

Matching Evaluation Levels with Objectives After completing this Agile development program, participants should: 6. Perceive Agile development to be relevant to their work (4. 5 out of 5). 7. Decrease security breaches by 25% in six months. 8. Reduce the number of build failures by 10%. 9. Score an average of 75 or better on Agile principles quiz. 10. Recommend the program to other developers. 21

Define Success Write a one sentence description of the success of your Agile training and coaching programs. 22

The Possible Measures What’s Your Business? 0 Serve the largest number of people with the least amount of disruption and cost. 1 Participants are engaged and see their experience as valuable. 2 Participants are learning the latest information and skills to make great software. 3 Participants leave the program, take action, use the content, and make great software. 4 Participants are driving important business measures and having an impact in their work and the organization. 5 Participants and the organization have a positive return on the investment of their time and the resources of the organization. 23

Make it Matter Designing for Input, Reaction, and Learning Design Thinking Principle 4 A way to take on design challenges by applying empathy • Focus on the objectives • Think about the definition of success • Make it relevant • Make it important • Make it action-oriented 24

Make it Stick Designing for Application and Impact Design Thinking Principle 5 A culture that fosters exploration and experimentation • Collect data • Focus on objectives • Ensure transfer to the workplace • Design for application 25

Make it Stick Application — Level 3 • • • Number of tests Tests passed Code coverage Build failures Process adherence • • Cohesion Velocity Process defects Use Agile principles 26

Make it Stick Impacts — Level 4 • Revenue • Customer Satisfaction • Time to market • • Defect (quality) Efficiency Innovation rate Customer Usage 27

Data Collection During and After Program Method Type of Data Level 1 Level 2 Level 3 Surveys ü ü ü Questionnaires ü ü ü Observation ü ü ü Interviews ü ü ü Focus Groups ü ü ü Tests/Quizzes ü Demonstrations ü Simulations ü Level 4 ü Action Planning/ Improvement Plans ü ü Performance Contracting ü ü Performance Monitoring ü 28

Make it Credible Measuring Results and Calculating ROI Design Thinking Principle 6 A fixed process and a tool kit • Isolating the effects of programs • Converting data to money • Tabulating Costs • Calculating ROI 29

• • • Control groups Trend line analysis Mathematical modeling Participant’s estimate of impact Supervisor’s estimate of impact Management’s estimate of impact Use of experts/previous studies Calculate the impact of other factors Customer input 30

• • • Profit/savings from output Cost of quality Employee time as compensation Historical costs/savings Expert input External studies Linking with other measures Participant estimation Supervisor/manager estimation Staff estimation 31

• • Increased happiness Improved engagement Reduced complaints Reduced conflicts Reduced stress Increased teamwork Increased brand awareness Improved image 32

• Research, design, and development (prorated) costs • Materials costs • Facilitator and/or coordinator costs • Facilities costs • Travel/lodging/meals • Participant salaries and benefits • Administrative/overhead costs • Evaluation costs 33

34

Measurement Targets Level % of Programs 0 Input 100% 1 Reaction 100% 2 Learning 80 -90% 3 Application 30% 4 Impact 10% 5 ROI 5% 35

What percent of your Agile projects are evaluated at Level 4 per year? 36

Bean Counters Need Beans to Count 37

Tell the Story Communicating Results to Key Stakeholders Design Thinking Principle 7 A storytelling process to inspire senior executives • Define audience • Identify why they need it • Select method • Move quickly • Consider one-page summary 38

Optimize Results Using black box thinking to increase funding Design Thinking Principle 8 A new competitive logic of business strategy EVALUATION OPTIMIZATION ALLOCATION Measure Improve Fund 39

… y s a E t a h T s ’ It 40

Remember, when it comes to delivering results from organization development: • Hope is not a strategy. • Luck is not a factor. • Doing nothing is not an option. Change is inevitable. Progress is optional. 41

The Payoff • Align projects to business needs • Show contributions of selected projects • Earn respect of senior management/administrators • Build staff morale • Justify/defend budgets • Improve support for projects 42

• Enhance design and implementation processes • Identify inefficient projects that need to be redesigned or eliminated • Identify successful projects that can be implemented in other areas • Earn a “seat at the table” COA CH 43

And The Winner Is… WE CAN SEND YOU A FREE COPY OF THE E-BOOKS MEASURING THE SUCCESS OF COACHING OR REAL WORLD TRAINING EVALUATION Email Kylie@roiinstitute. net to claim your book! 44

Thank you! Email me: jack@roiinstitute. net Visit us online: roiinstitute. net Connect with us on: 45

46