Jim Stuart Kevin White Manager Applied Statistics Eastman

Jim Stuart Kevin White Manager, Applied Statistics Eastman Chemical Company jestuart@eastman. com Senior Statistician Voridian, a Division of Eastman Chemical Company kwhite@eastman. com 1

Statistical Thinking. . . A habitual way of looking at work that: · recognizes all activities as PROCESSES, · recognizes that all processes have VARIABILITY, · uses DATA to understand variation, and to drive effective DECISION MAKING. 2

Outline · Process Thinking Principles · 8 Lessons for Visualizing Variability · Databased Decision Making – Control Charts – Special Cause Rules – Change Point Analysis 3

Process Thinking Managerial Data Should: • Summarize performance on what is key to business success. • Provide history of how the business has performed. • Help predict the future. • Provide the foundation for improvement. (Gap Identification) • Provide a signal for reinforcement of accomplishments. • Serve as a means for holding the gains. 4

Principles for Selecting Measures • Sufficient number to adequately cover all the important facets of the business. If it isn’t important to the business, don’t track it. • Each measure should impact at least one stakeholder including suppliers, publics, investors, customers, or employees (SPICE). • Needs to be an appropriate mix of leading and lagging measures. • Lend themselves to charts that are easy to read and interpret. • Measures should be analyzed for appropriateness if the situation or strategy changes. Well-charted measures can provide a mechanism for concise communication with stakeholders. 5

Indicators of Performance Outputs Inputs Process Customers Products or Services 3 2 1 Gauge Quality of Inputs Process Quality/ Reliability Supplier Quality (Leading Indicators of Product and Service Quality) 4 5 Product and Service Quality Customer Satisfaction Customer Dissatisfaction 6 Gauge Financial/Cost People 7 Gauge Health, Safety & Environmental 6

His project is 10% over budget… Good News? Bad News? No Earthly Idea? 7

Statistical Thinking Lesson #1: “It Depends” • Was the budget set at the best current estimate or was it a “guaranteed not to exceed number”? • What are the implications of financial planning if everyone uses guaranteed not to exceed numbers? • What would you suspect if a particular project manager finished every project exactly on schedule? 8

Statistical Thinking Lesson #2: “Variation Happens…At Least It Should” 120 Distribution of Project Cost Variances ESTIMATED COST NUMBER OF PROJECTS 100 80 60 40 overrun 20 0 -20 -10 0 10 MANAGER A 20 -30 overrun -20 -10 0 10 MANAGER B 20 -10 0 10 20 MANAGER C 9

Statistical Thinking Lesson #3: “Show Data in Time Order” 10

Statistical Thinking Lesson #4: “Beware Your Axes” The selection of the scale of your vertical axis can have a profound effect on the interpretation by the audience…particularly if it is not their data. Daily Sales in Thousands 11

Statistical Thinking Lesson #5: “Don’t Over-Summarize” Collect and display data at sufficient frequency to understand the variation, and beware the trappings of bar-charts! Opportunity Seeking Improvement Motivating (Let’s fix the dips!) Management Review And Presentation (I’m OK, you’re OK) 12

I’m OK, You’re OK Slide 250. 0 Summary presentations utilizing averages, ranges or histograms should not mislead the user into taking action that would not have been taken if presented as a time series. 200. 0 150. 0 100. 0 50. 0 2001 13

Statistical Thinking Lesson #6: “Display History to Provide Context” Key Result Area: Stakeholder: Chart Title Brief Description (Measure & Scope) 500 Measure (Clear Description & Units) Date Prepared GOOD 450 400 350 300 250 200 150 100 50 0 J M 1998 Comparison 1 M J S N Comparison 2 J M 1999 M J Source of Data, How Measure is Calculated Population (i. e. , Entire Company, All U. S. , etc. ) S N Plot sufficient history to visualize trends relative to the variation Goal J M 2000 Month/Year M J S N J M 2001 M J S N List of Supporting Information (Tables, Bar Charts, Pie Charts, etc. ) 14

Statistical Thinking Lesson #7: “Provide Comparisons to Enable Gap Analysis” Key Result Area: Stakeholder: Chart Title Brief Description (Measure & Scope) 500 Measure (Clear Description & Units) Date Prepared GOOD 450 400 350 300 250 200 150 Relevant comparisons should be placed in the appropriate locations on the graph 100 50 0 J M 1998 Comparison 1 M J S N Comparison 2 J M 1999 M J Source of Data, How Measure is Calculated Population (i. e. , Entire Company, All U. S. , etc. ) S N Goal J M 2000 Month/Year M J S N J M 2001 M J S N List of Supporting Information (Tables, Bar Charts, Pie Charts, etc. ) 15

Statistical Thinking Lesson #8: “Use Moving Averages with Caution” • Helps visualize trends through the noise. • Length should cover expected cycles. Annual is most common. • Tend to be sluggish. • Can generate the appearance of cycles or shifts which are not truly present. – Cannot use run rules to signal special causes • Control limits for moving averages can be calculated, but prefer not to place them on the graph itself. 16

The Headlines Scream - Great News! 17

What We All Imagine… ‘cause this is what newspaper graphs look like 1995 1996 1997 1998 18

Reality Projected 1998 19

Statistical Thinking. . . A habitual way of looking at work that: · recognizes all activities as PROCESSES, P · recognizes that all processes have VARIABILITY, P · uses DATA to understand variation, and to drive effective DECISION MAKING. 20

Databased Decision Making Managers are routinely faced with interpreting their metrics and making a real-time decision as to whether the latest data point tells them to do something. · Good graphical depiction goes a long way · Seasoned managers can see signals through the noise · Statistics can take the subjectivity out of such decisions · One size does not fit all 21

Control Charts - The Two Mistakes · The False Alarm - Interpreting noise as a signal · The Missed Alarm - Failure to detect a signal 22

Control Charts in Data Rich Environments · Control limits set at 3 standard errors · Approximate 0. 3% risk of a false alarm · The risk of the missed alarm is often overlooked · In parts manufacturing, greater sensitivity can be obtained by giving consideration to the selection of the rational subgroup 23

The “Run of 8” Rule · Sometimes 7 and sometimes 9 · Also provides low risk of false alarms · Used with 3 standard error limits, sensitivity is improved · Takes 8 points to initiate signal Many other rules are also often used in the data rich environment for greater sensitivity but the tradeoff is a higher false alarm rate. 24

Average Run Lengths for Typical Data Rich Environments 25

Average Run Lengths for Typical Data Rich Environments (Reduced Scale) 26

Why These Rules Work for Data Rich Environments · High false alarm rates would lead to wasted time doing investigation and possibly excessive process adjustments. · Poor sensitivity is often an acceptable trade-off because for a lower false alarm rate · And the next point is never far behind 27

Why Managerial Data Is Different · The Obvious - less frequent data · Detection of large process shifts is not as important · Actions taken are different · Improve mindset, not maintain 28

Traditional Rules Applied to Low Frequency Managerial Data · A shift of 1. 5 standard errors takes eight points on average to detect · This is little comfort if dealing with monthly managerial data 29

The Individuals Chart · An excellent all-purpose tool · Very robust - low false alarms for virtually any data distribution (typically < 1%) · A single option for managers will get more use · But, don’t forget the poor sensitivity 30

Sensitivity for Managerial Data · Data is usually individual observations (cannot subgroup) · With traditional special cause rules, there is no control over risk of the missed alarm · User can control the width of the control limits · User can employ some modified run rules These modifications do come with a higher false alarm rate! 31

Alternative Special Cause Rule Sets A - Control limits set at 2. 5 std. errors from the centerline. B - Control limits set at 2. 5 std. errors from the centerline plus two points past 1. 5 standard errors. C - Control limits set at 2. 0 std. errors from the centerline. D - Control limits set at 2. 0 std. errors from the centerline plus a run of 6. E - Control limits set at 2. 0 std. errors from the centerline plus three points past 1. 0 standard errors. F - Runs of 6 consecutive points on one side of the centerline. 32

Why Alternative Rules? · Greater sensitivity is desired with an acceptable number of false alarms · What’s acceptable? It depends (See Lesson #1)! – The data frequency – Time to do investigation – Importance of detecting quickly – Magnitude of change deemed important 33

Average Run Lengths for Alternative Rules - Chart #1 34

Average Run Lengths for Alternative Rules - Chart #2 35

Average False Alarms Per Year 36

Situational Recommendations Situation Recommendation Goal is to Maintain Use 3 Std Error Limits and consider Run of 6 Goal is to Improve Alternative Rules Strong Slope in the Metric Predictable Slope - Place limits around sloped center line. Highly Unstable Process Where’s the Average? Control Charts will not apply. 37

Change Point Analysis · The general principle is Monte Carlo simulation · Advantages include: – Very easy to use – Detects mean and variation changes – Excellent graphics 38

Change Point Analysis · Confidence Levels for the probability a change is real · Confidence Levels for when the change occurred. · Handles any type of data · More sensitive than control charts · Not confused by outliers 39

Change Point Analysis Example Graph 40

Change Point Analysis Example Table 41

Change Point Analysis Serial Dependency Change Point Analysis of $ Accounts Receivable 440000 330000 220000 Jan-1990 Dec-1990 Nov-1991 Oct-1992 Sep-1993 Aug-1994 Jul-1995 Jun-1996 Month/Yr 42

Conclusions · Process thinking and careful selection of measures can help keep managers focused · Appropriate plotting is 90% of the battle · Traditional control charts may not be optimal · Alternative special cause rule sets should be considered · Change point analysis may be the closest thing to an all-purpose tool for managers. 43

REFERENCES 1) Stuart J, White K, Methods for Handling Low Frequency Managerial Data, 2002 ASQ Annual Quality Congress Proceedings 2) Balestracci D, Data “Sanity”: Statistical Thinking Applied to Everyday Data, ASQ Statistics Division Special Publication, Summer, 1998 (Available through ASQ, www. asqstatdiv. org) 3) Britz G, Emerling D, Hare L, Hoerl R, Shade J, Statistical Thinking, ASQ Statistics Division Special Publication, Summer, 1996 (Available through ASQ, www. asqstatdiv. org) 4) Leitnaker, MG, Using the Power of Statistical Thinking, ASQ Statistics Division Special Publication, Summer 2000 (Available through ASQ, www. asqstatdiv. org) 5) Wheeler DJ, Understanding Variation: The Key to Managing Chaos. Knoxville, TN: SPC Press, Inc. 1986. (www. spcpress. com) 6) Taylor WA, "Change-Point Analysis: A Powerful New Tool For Detecting Changes, " WEB: www. variation. com/cpa/tech/changepoint. html, 2000 7) Wheeler DJ, Building Continual Improvement, Knoxville, TN: SPC Press, Inc. , 1998. (www. spcpress. com) 44