MER 301 Engineering Reliability LECTURE 16 Measurement System

Measurement as a Process We must submit the output from our design process to a second (measurement) process Parts (Example) Inputs Process Outputs Inputs Measurement Process L Berkley Davis Copyright 2009 Outputs MER 301: Engineering Reliability • Measurements 2

Measurement System Concerns. . l l l How big is the measurement error? What are the sources of measurement error? Are the measurements being made with units which are small enough to properly reflect the variation present? Is the measurement system stable over time? How much uncertainty should be attached to a measurement system when interpreting data from it? How do we improve the measurement system? L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 3

Measurement System Analysis o o Total Error in a measurement is defined as the difference between the Actual Value and Observed Value of Y Two general categories of error – Accuracy or Bias and Precision Accuracy or Bias of Measurement System is defined as the difference between a Standard Reference and the Average Observed Measurement Precision of a Measurement System is defined as the standard deviation of Observed Measurements of a Standard Reference Total Error = Bias Error + Precision Error for independent random variables n n n o Measurement System Error is described by Average Bias Error (Mean Shift)and a statistical estimate of the Precision Error (Variance) Measurement System Analysis is a Fundamental Part of Every Experiment L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 4

Precision and Accuracy Not Accurate, Not Precise Not Accurate, Precise L Berkley Davis Copyright

Measurement System Analysis o o Bias or Accuracy error is a constant value and is dealt with by calibrating the measurement system Variation or Precision error is a random variable which depends on the measurement equipment(the instruments used) and on the measurement system repeatability and reproducibility. Instrument Capability Analysis, Test/retest (repeatability)and Gage R&R studies are used to quantify the size of these errors. L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 6

Impact of Measurement System Variation on Variation in Experimental Data = Product Std. Dev. = Product Mean Actual Defects LSL USL Product variance Measurement system variance Observed Defects LSL L Berkley Davis Copyright 2009 USL MER 301: Engineering Reliability 7

Example 16. 1 -Effect of Measurement System Variation o Calculate the effect of measurement system variation on the acceptance rate for a part with USL and LSL at Z= +/-1. 96 respectively. If then what is the percentage of acceptable parts that will be rejected? If on the other hand what is the percentage of acceptable parts that will be rejected? L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 8

Impact of Measurement System Variation on Variation in Experimental Data… L Berkley Davis Copyright

Example 16. 1(con’t: ) o For the product, the spec limits of +/-1. 96 mean that the 2. 5% of parts in each tail are out of spec. Thus o For the observed standard deviation is and Then the acceptable parts now rejected are L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 10

Example 16. 1(con’t: ) o For the product, the spec limits of +/-1. 96 mean that the 2. 5% of parts in each tail are out of spec. Thus o For the observed standard deviation is and Then the acceptable parts now rejected are L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 11

Example 16. 1(con’t) Unacceptable Acceptable Set Measurement System Requirements Based on the Process Variation

Gage Repeatability & Reproducibility ---GRR or GR&R--- o Gage Repeatability & Reproducibility compares measurement system variation and product variation o The term is the size of an interval containing 99% of the measured values made on a specific item o The Tolerance- often equal to - is the size of the interval where a product has acceptable dimensions, performance, or other characteristics L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 13

Gage Performance relative to required Tolerance Band o R&R less than 10% - Measurement system is acceptable. o R&R 10% to 30% - Maybe acceptable - make decision based on classification of characteristic, hardware application, customer input, etc. o R&R over 30% - Not typically acceptable. Find the problem using root cause analysis(fishbone), remove root causes GRR is a measure of “noise” in the data L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 14

Effect of Gage R&R on Variation o GRR <10% means < 0. 7% of the variation in the experimental data is from the measurement system o GRR> 30% means that > 5. 9% of the variation in the experimental data is from the measurement system L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 15

GRR Example 16. 2 o The GRR values for the previous Example 16. 1 are o The capabilities of two (or more) measurement systems can be compared by comparing the GRR’s for each. Since GRR 2<GRR 1 , the second measurement system is more capable than the first. The observed standard deviations quantify how much better…. L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 16

GRR Example 16. 2 o The GRR values for the previous Example 16. 1 are at best marginally acceptable(GRR 2 ) or not acceptable(GRR 1 ) o For a GRR value equal to 10% (0. 10) there results L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 17

Example 16. 2 ( so ) o For the product, the spec limits of +/-1. 96 mean that the 2. 5% of parts in each tail are out of spec. Thus o For the observed standard deviation is and Then the acceptable parts now rejected are L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 18

Summarizing how it all fits together…. . o When a set of measurements are made, the results are always observed values, o If the actual mean and standard deviation are known the measurement system bias and variance can be calculated n o If the item being measured is a standard reference If the measurement system bias and variance are known the actual mean and actual variance can be calculated L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 19

Sources of Measurement System Error Inputs Process Inputs Outputs Measurement Process Outputs • Observations • Measurements Observed Process Variation Measurement Variation Actual Process Variation Long-term Process Variation Short-term Process Variation within sample variation Variation due to operator Variation due to gauge Measurement System Repeatability Reproducibility Resolution Accuracy (Bias) Precision (Pure Error) Stability (time dependent) Linearity (value dependent) Repeatability Total Variation made up of Actual Process Variation and Measurement L Berkley Davis Copyright 2009 System. Engineering Variation MER 301: Reliability 2

Measurement System Errors True Accuracy (Bias) Time 2 Time 1 Repeatability (precision) Observed Average True Average Operator A Observed Average (Low End) Accuracy (Low End) Stability True Average Observed Average (High End) Accuracy (High End) Operator B Linearity Reproducibility L Berkley Davis Copyright 2009 Engineering Reliability Lecture 16 21

Elements that contribute to Accuracy and Precision Errors o Instrument Capability n n n Resolution Gage Repeatability Linearity o Measurement System - Short Term (ST) n Instrument Capability n Equipment Calibration(Bias) n Test/Re-Test Study(Repeatability) o Measurement System - Long Term (LT) Use n Measurement System - Short Term Use n Reproducibility n Stability L Berkley Davis Copyright 2009 First Two are Entitlement…. Third MER 301: Engineering is Reality Reliability 2 2

Elements that contribute to Precision or Variation Errors o Instrument Capability n n n Resolution Gage Repeatability Linearity o Measurement System- Short Term (ST) Use n Instrument Capability n Equipment Calibration(Bias) n Test/Re-Test Study(Repeatability) o Measurement System - Long Term (LT) Use n Measurement System - Short Term Use (ST) n Reproducibility(Gage R&R) n Stability(Gage R&R) L Berkley Davis Copyright 2009 First Two are Entitlement…. Third is Reality MER 301: Engineering Reliability 23

Measurement System Analysis From pages 119 -120… L Berkley Davis Copyright 2009 MER 301:

Updating how variances all fit together o When a set of measurements are made, the results are always observed values, o If the actual mean and standard deviation are known the measurement system bias and variance can be calculated n o If the item being measured is a standard reference If the measurement system bias and variance are known the actual mean and actual variance can be calculated L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 25

Emissions Sampling Heated Sampling Line Cal/Zero Gases Calibration Gas NOx Instrument Sample Conditioning Yactual- NOx from Gas turbine Yobs- NOx Reading L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 26

Elements that contribute to Accuracy and Precision Errors o Instrument Capability Resolution Gage Repeatability Linearity Measurement System. Short Term(ST) Use Instrument Capability Equipment Calibration Test/Re-Test Study Measurement System. Long term (LT) Use Measurement System -Short Term(ST) Use Reproducibility Stability L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 27

How Can we Address Accuracy and Precision Errors? o Establish magnitude and sources of measurement system error due to bias and precision errors o Tools Instrument Capability Analysis Test/Re-test – system precision/repeatability Calibration - bias “Continuous Variable” Gage R&R (Gage Reproducibility and Repeatability) n Attribute Variable Gage R&R n Destructive Gage R&R n n L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 28

Measurement System Analysis o Instrument Capability Analysis…. . o o Resolution-smallest increment that the gage can resolve in the measurement process. Gage should be able to resolve tolerance band into ten or more parts. Resolution Uncertainty = Instrument Accuracy- measure of instrument repeatability or instrument “noise”. . Found by repeated measurements of the same test item. Uncertainty = Linearity- consistency of the measurement system across the entire range of the measurement system. Linearity Uncertainty = The variations are combined as follows L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 29

Instrument Capability Analysis…. . o Variation for any one instrument equals the sum of the resolution, repeatability and linearity terms o The Variation for “n” instruments equals the sum of the variations for each individual instrument n L Berkley Davis Copyright 2009 Each of the “n” instruments has resolution, repeatability, and linearity terms that must be taken into account MER 301: Engineering Reliability 30

Instrument Capability Analysis - Resolution of Instruments/Sensors o The measurement uncertainty due to resolution is generally taken as a specified fraction of the smallest increment an instrument can resolve, ie as a fraction of the smallest scale division o General Rule: assign a numerical value for the mean value of equal to one half of the instrument resolution. This means that half of the smallest scale division is assumed to equal a 95% Confidence Interval ( a wide band) for variation due to resolution L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 31

Instrument Capability Analysis Repeatability and Linearity o The manufacturer of an instrument will provide information on the capability of the instrument in the specification sheets provided with the instrument o The numerical values given for Instrument or Sensor Accuracy and Linearity are almost always uncertainties n Let = uncertainty due to the equipment accuracy/repeatability error where n Let = uncertainty due to linearity error where o The inherent capability/uncertainty of the instrument/sensor is then estimated as: L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 32

Example 16. 3 -Instrument Capability o The Capability of a force measuring instrument is described by catalogue data. Calculate an estimate of the variation attributable to this instrument. Express the result both in dimensional terms (N) and in dimensionless terms for a reading R=50 N Resolution Range Linearity Repeatability L Berkley Davis Copyright 2009 0. 25 N 0 to 100 N within 0. 20 N over range within 0. 30 N over range MER 301: Engineering Reliability 33

Example 16. 3(con’t) o An estimate of the instrument uncertainty depends on the combined uncertainties due to resolution, repeatability and linearity The instrument uncertainty is then L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 34

Measurement System Analysis o Instrument Capability Analysis Summary…. . o o Resolution-smallest increment that the gage can resolve in the measurement process. Gage should be able to resolve tolerance band into ten or more parts. Resolution Uncertainty = Instrument Accuracy- measure of gage repeatability or gage “noise”. . Found by repeated measurements of the same test item. Uncertainty = Linearity- consistency of the measurement system across the entire range of the measurement system. Linearity Uncertainty = The variations are combined as follows L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 35

Measurement System Analysis o Measurement System Short Term Use n n n Includes Instrument Capability Repeatability - variation when one operator repeatedly makes the same measurement with the same measuring equipment Test/Re-test Study Calibration/Bias o Measurement System-Long Term Use n n n L Berkley Davis Copyright 2009 Includes Measurement System –Short Term Use Reproducibility- variation when two or more operators make same measurement with the same measuring equipment Stability-variation when the same operator makes the same measurement with the same equipment over an extended period of time MER 301: Engineering Reliability 36

Test/Retest Example 16. 4 o Test/Retest (Repeatability) Study on a Measurement System. Thirty repeat measurements were taken on a Standard Reference Item with a thickness of 50 mils The tolerance band for the application is 20 mils(+/10). o Data, in mils n 53, 45, 52, 47, 54, 52, 55, 52, 48, 53, 55, 51, 47, 52, 47, 35, 45, 54, 48, 51, 53, 44, 52, 55, 59, 53 L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 37

Example 16. 4(con’t) o Objective is to establish the precision and accuracy of the measurement system o Precision-Repeatability In a good Measurement System, 99% of the measurements of a given item should fall within a band less than 1/10 of tolerance band o Accuracy/Bias = sample mean- true value L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 38

Example 16. 4 Run Chart and Histogram These results look bad to the eye…

Test/Retest Study Example 16. 4 Summary o Descriptive Statistics Variable N C 1 30 Variable Minimum C 1 35. 000 Mean 50. 567 Maximum 59. 000 Median 52. 000 Q 1 47. 750 St. Dev 4. 561 Q 3 53. 000 SE Mean 0. 833 o Conclusions n Given the tolerance band of 20 mils, there is an unacceptable level of device precision n Given the Reference Test item had a known thickness of 50 mils, the bias(inaccuracy) is: bias = 50. 57 – 50. 0 = 0. 57 mils L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 40

Measurement System Analysis o Measurement System-Short Term Use o Repeatability-variation when one operator repeatedly makes the same measurement with the same measuring equipment Test/Re-test Study o Measurement System - Long Term Use o Reproducibility- variation when two or more operators make same measurement with the same measuring equipment o Stability-variation when the same operator makes the same measurement with the same equipment over an extended period of time L Berkley Davis Copyright 2009 MER 301: Engineering Reliability 41

Elements that contribute to Accuracy and Precision Errors o Instrument Capability n n n Resolution Gage Repeatability Linearity o Measurement System - Short Term (ST) n Instrument Capability n Equipment Calibration(Bias) n Test/Re-Test Study(Repeatability) o Measurement System - Long Term (LT) Use n Measurement System - Short Term Use n Reproducibility n Stability L Berkley Davis Copyright 2009 Engineering Reliability First Two are MER 301: Entitlement…. Third is Reality Lecture 16 4 2