What Gets Measured Gets Undone Dr Jim Mirabella

What Gets Measured Gets Undone Dr. Jim Mirabella Director of Institutional Research Professor of Statistics Florida Community College - Jacksonville (FCCJ) J. Michael Adams Corporate Manager, Quality Services Florida Power & Light/ FPL Group, Inc. Florida Sterling Conference Wednesday, May 31, 2000 Disney World, Orlando, Florida 1

What’s getting “undone” ? • The desired outcome and supporting processes that the measure(s) intended to describe • Why? ? – Poor measures drive poor performance! 2

Objectives of the workshop • Know good measurements from poor measurements • Distinguish the implications of good and poor measures on performance • Balance an array of measures that favorably tracks a process’s outcome for planning, assessing performance, and analysis • Assess attendees’ measures • Any other expectation? 3

“Why” Measure at All? • Measurement is the language of progress • • and comparison Provides a sense of where we are AND where we are going (planning) Can guide a steady advancement toward established goals (tracking) Can identify goal shortfalls, or overachievement (analysis) Communicates to the work force what is important to the organization (behavioral) 4

Good measures are born from SMART goals • Specific • Measurable • Agreed upon • Realistic • Time-Bound 5

The Challenge of Measuring • Workers might perceive it as a threat • Workers might disregard organizational goals, customers, products and services • Workers might focus on obtaining favorable measurements • Measuring items with no influence on organizational success = waste of time – result is a bean-counting approach that focuses on irrelevant details • Expected to do something! 6

The Challenges- examples • “Apples and oranges” • Impact on comparisons, benchmarking and performance • turnover • consistency • defects vs. defectives • commutes (measured in time or miles) • school achievement 7

“Where” are these measures • • Organizational Process Department/work unit Individual – Exercise 8

Your measures Jot down your various performance measures for you, your department, unit and organization. Indicate the purpose of the measure ( 1= planning, 2= tracking and analysis; 3: behavioral changes) • Organization • Department/ Unit • Process • Individual 9

“What” Do We Measure? • Quality • Defects • Delivery • Satisfaction • Cycle Time • Complaints • Waste • Financials • Cost • Price 10

Measurement Examples 4 Operations-related measures - reliability - timeliness of delivery - order processing time - errors / defects - product lead time - inventory turnover - cost of quality - employee 11

Measurement Examples 4 Customer-related measures - customer satisfaction - customer complaints - customer retention 4 Financial measures - market share - sales per employee - return on assets - return on sales 12

Baldrige/ Sterling: Results Category Private, Education, Health Care 7. 1: 7. 2: 7. 3: 7. 4: 7. 5: Customer Focus Results; Student Performance, Patient and other Customer Focus Financial and Market; Student and Stakeholder Focused Results, Human Resource Result; Budgetary and Financial Results, Staff and Work System Supplier and Partner Results; Faculty and Staff Results Organizational Effectiveness 13

Determining and Reviewing Measurements for Balance Baldrige Results 7. 0 • Customer • Financial • Human Resources process dept • Supplier/ Partner • Organizational supplier 14

Desired Outcomes- Airlines overall organization balance scorecard • “Assumed Quality”- arrive safely • Loyal Customers – Market differentiation, key value attributes: • on-time arrival • baggage handling • customer complaints • Profitability and Market share – Satisfied stakeholders (shareholders, partners) • Employee growth and retention 15

Airline Measurement System • Individual measures for each flight process • Group measures for overall airline • If problems occur with a flight, who do you blame -- the flight crew or the airline? • What measures do you affiliate with the flight crew? • What measures do you affiliate with the airline? • What is important to you for a satisfactory flying experience? 16

Review your measures • Are the organization measures reflective of all Baldrige results items? • Do the measures reward favorable behaviors? • Is there alignment with the contributing departments and suppliers? • Are the overall results managed as an outcome of a process? 17

What’s the opportunity? • Many organizational measurement systems are too short, too rigid, or used like a strict teacher’s ruler. . . to whack rather than to motivate • Need to replace these outdated measurement systems with more dynamic measurement system that motivates continuous improvement in customer satisfaction, flexibility, and productivity … SIMULTANEOUSLY 18

Mis-measurement Systems • Unless specifically tuned to flight plan, measurement systems may: – yield irrelevant / misleading information – provoke behavior not conducive to strategy • Traditional measures ignore requirements & perspectives of customers (internal/external) • Bottom-line measures (profitability) too late for mid-course correction / remedial action 19

Mis-measurement Systems • Many measurement systems overlook key non- financial performance indicators • Measures often used for punishment rather than to promote learning • Many measurement systems are inflexible and limited in what they can do 20

What Should a Measurement System Do? • Measures must link operations to strategic goals – departments should know how they contribute separately and together toward strategic mission • System has to integrate financial / non-financial info in a way usable by managers – managers need right info at right time • Measurement system’s real value lies in its ability to focus all business activities on customer requirements 21

What Should a Measurement System Do? • Measure what is important to the customers • Motivate operations to continually improve against customer expectations • Identify and eliminate waste -- of both time and resources • Help to accelerate organizational learning and build a consensus for change when customer expectations shift 22

Measurement Mismanagement In an effort to increase market share, a computer service bureau strategizes to improve the timeliness of its voicemail service. The goal is to provide new customer with service w/in 24 hours. To help speed the process, a program is developed to accept verbal phone orders. Most new customers were online w/in a day as promised, causing the company to celebrate their success. A later audit revealed 70% error rate in order entry, with 30% of customers disputing their bill and eventually canceling service. . 23

Measurement Mismanagement A Hi-Tech company sets objective to become highly profitable by being a product leader. To measure the performance of its marketing and R&D functions, the number of new products developed is the most watched barometer. An internal review reveals that in a sample of 20 new product introductions, 80% were delivered over a month late, and significant waste piles up in production. Management cannot understand why the accountant’s ink is red – after all, their yardstick tells them they are developing new products at a record rate. 24

Beware of ……. . Seemingly Simple Measures • How satisfied are your customers? • What is your employee turnover? • What is your client retention rate? • Is the value of our service worth the price? 25

Beware of ……. . Treating customer perceptions as objective measures • Customer satisfaction product quality • Customer satisfaction is a complex phenomenon • A well-handled complaint results in higher customer satisfaction than does no complaint • More reasons for complaint more dissatisfaction 26

Beware of ……. . Non-specific measurements • Results are actionable • Hard to improve what you cannot assess Failing to measure adequately • Think BALANCED SCORECARD • Don’t give employees an outlet for gaming success • Identify all areas important to customers 27

Beware of ……. . Using results incorrectly • Don’t tie results to employee pay unless employees can directly influence results • Don’t base employee pay on results that cannot be measured 28

Recap of Objectives and Expectations • Know good measurements from poor measurements • Distinguish the implications of good and poor measures on performance • Balance an array of measures that favorably tracks a process’s outcome for planning, assessing performance, and analysis • Assess attendees measures 29

Thank you!!!!!! Jim Mirabella Mike Adams 30