DIY Metrics Copyright Martin Klubeck Michael Langthorne Don

DIY Metrics Copyright Martin Klubeck, Michael Langthorne, Don Padgett 2006. This work is the intellectual property of the authors. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the authors. To disseminate otherwise or to republish requires written permission from the authors © 2005 Klubeck, Langthorne, & Padgett

D. I. Y. Metrics ü Your Presenters üMartin Klubeck üDon Padgett üMichael Langthorne ©

Metrics What are metrics? üWhat is data? üWhat is a measure? üWhat is information?

Help Desk Example Data Number of trouble calls Number of opened cases Number of closed cases Number of employees Number of survey responses Measures Number of calls per hour Number or cases closed by worker Information Number of calls for each hour compared to number of workers on shift. Average length of time to close a case, grouped by type Average customer satisfaction rating © 2005 Klubeck, Langthorne, & Padgett

Help Desk Example Average number of open cases from 1999 -2002 225 (cont) manning Max manning 200 11 175 10 150 9 125 8 100 Min manning 7 50 6 25 METRIC 12 5 l Ju n 0 Ju ay M r Ap ar M b Fe n Ja c De v No t Oc p Se Au g 0 Explanation: The manning over the academic year was not in line with the number of trouble calls received – based on data collected over the last three years. Result: We’ve re-aligned our manning for the coming year to match the level of need each month. © 2005 Klubeck, Langthorne, & Padgett open cases manning

Metrics ü ü ü ü Metric Name Purpose (a go/no go proposition) Metric Area/Category Customer Graphical Representation Explanation Metrics Analysis Measures used to develop metric Collection Schema Schedule Assumptions and Constraints Related Metrics and Data Dependencies Lessons Learned © 2005 Klubeck, Langthorne, & Padgett

Implementation Guide © 2005 Klubeck, Langthorne, & Padgett

1. Metric Name: Quantitative or Qualitative Definition: Summary: History: 2. Purpose: The information need that drove the development of the metric. What was the root question? Why do we want to tell the story? What do we hope to achieve? © 2005 Klubeck, Langthorne, & Padgett

3. Metric Area/Category: This is the Metric Area/Category any way you want to group the metric (Dept/Office, function, customer, etc. ) One method is to list the information of the Quality Box 4. Customer: Who will you present the metric to? The customer could be you © 2005 Klubeck, Langthorne, & Padgett

5. Graphical Representation: What Charts/Graphs will be used to tell story? – trends, Pareto, benchmarking, bar, line, etc. ): Can be hand drawn in the early stages of development. Insert actual graphical representations when reach that phase of the implementation Average number of open cases from 19992002 Max manning 225 manning 12 200 11 175 10 150 9 125 8 100 Min manning 50 25 manning g Se p Oc t No v De c Ja n Fe b M ar Ap r M ay Ju n Ju l 6 5 0 Au 0 7 open cases 6. Explanation: A prose version of the story the metric tells. Using the analysis above, explain how the metric should be interpreted. How to read the picture? Include how the metric will be used - and how it will NOT be used. © 2005 Klubeck, Langthorne, & Padgett

7. Metrics Analysis: This is where we log any formulas or mathematical equations used. Goals and thresholds also qualify. Format: Threshold: Target: Derived Information: 8. Measures used to develop the metric: The specific data points to be collected and used to develop the metric. The lowest-level view of the data. Data & Measures: © 2005 Klubeck, Langthorne, & Padgett

9. Collection Schema: Be as detailed as possible in this section - it should be a guide for the collector to follow Collector: Who will have the joy of actually collecting the data? The less human interaction, the better Source: Where is the information (data points or measures) available from? Frequency: How often should the data be collected/reported? When? Methodology: How does it get collected? Is it automated? Where is it stored? 10. Schedule: Milestones in the collection, reporting, and use of the metric should be tracked here Estimated completion dates. This helps to show the history of the metric. © 2005 Klubeck, Langthorne, & Padgett

11. Assumptions, constraints, and any known flaws: Log any assumptions made, like data will be available from the source when needed. Also track any constraints you expect to have to deal with such as lack of resources. 12. Related Metrics and Data Dependencies: Any related metrics that should be looked at with this metric to tell a bigger story. Metrics rarely should be looked at in isolation © 2005 Klubeck, Langthorne, & Padgett

13. Lessons Learned: You will eventually log lessons learned from the activity of collecting, reporting, and using the metric here. You can also log lessons learned from the metric itself. It would be useful for you to also log any successes or accomplishments in the improvement arena as a result of the metric. Did the metric meet its purpose? © 2005 Klubeck, Langthorne, & Padgett