EUROMET Project 732 Towards new Fixed Points Workshop

EUROMET Project 732 « Towards new Fixed Points » Workshop COMPONENTS OF VARIANCE MODEL FOR THE ANALYSIS OF REPEATED MEASUREMENTS Eduarda Filipe St Denis, 23 rd November 2006

Summary • • 2 Introduction General principles and concepts The Nested or Hierarchical Design Application example of a three-stage nested experiment. Eduarda Filipe Workshop Pr. 732

Introduction • The realization of the International Temperature Scale of 1990 - ITS 90 requires that the Laboratories usually have more than one cell for each fixed point. • The Laboratory may consider one of the cells as a reference cell and its reference value, performing the other(s) the role of working standard(s), or the Laboratory considers its own reference value as the average of the cells. 3 Eduarda Filipe Workshop Pr. 732

Introduction v In both cases, the cells must be regularly compared and the calculation of the uncertainty of these comparisons performed. v A similar situation exists when the laboratory compares its own reference value with the value of a travelling standard during an interlaboratory comparison. 4 Eduarda Filipe Workshop Pr. 732

Introduction v These comparisons experiments are usually performed with two or three thermometers to obtain the differences between the cells. v The repeatability measurements are performed each day at the equilibrium plateau and the experiment is repeated in subsequent days. This complete procedure may also be repeated some time after. 5 Eduarda Filipe Workshop Pr. 732

Introduction v The uncertainty calculation should take into account these time-dependent sources of variability, arising from short-term repeatability, the day-to-day or ”medium term” reproducibility and the long-term random variations in the results; 6 Eduarda Filipe Workshop Pr. 732

Introduction v The Type A method of evaluation by the statistical analysis of the data obtained from the experiment is performed using the Analysis of Variance (ANOVA) for designs, consisting of nested or hierarchical sequences of measurements. 7 Eduarda Filipe Workshop Pr. 732

ANOVA definition ISO 3534 -3, Statistics – Vocabulary and Symbols “Technique, which subdivides the total variation of a response variable into meaningful components, associated with specific sources of variation”. 8 Eduarda Filipe Workshop Pr. 732

General principles and concepts v Experimental design is a statistical tool concerned with planning the experiments to obtain the maximum amount of information from the available resources v This tool is used generally for the improvement and optimisation of processes 9 Eduarda Filipe Workshop Pr. 732

General principles and concepts • it can be used to test the homogeneity of a sample(s) • to identify the results that can be considered as “outliers” • to evaluate the components of variance between the “controllable” factors 10 Eduarda Filipe Workshop Pr. 732

General principles and concepts • tool applied to Metrology for the analysis of large amount of repeated measurements • Measurements in: – short-term repeatability – day-to-day reproducibility – long-term reproducibility • permitting “mining” the results and to include this “time-dependent sources of variability” information at the uncertainty calculation. 11 Eduarda Filipe Workshop Pr. 732

General principles and concepts v In the comparison experiment to be described, the factors are the standard thermometers, the subsequent days measurements and the run measurements. v These factors are considered as random samples of the population from which we are interested to draw conclusions. 12 Eduarda Filipe Workshop Pr. 732

The nested or hierarchical design. Definition ISO 3534 -3, Statistics – Vocabulary and Symbols “The experimental design in which each level of a given factor appears in only a single level of any other factor”. Purpose of this model To deduce the values of component variances that cannot be measured directly. 13 Eduarda Filipe Workshop Pr. 732

The nested or hierarchical design. General model • The factors are hierarchized like a “tree” and any path from the “trunk” to the “extreme branches” will find the same number of nodes. The analysis of each factor is done with: “Random Effects One-way ANOVA” or components-of-variance model, nested in the subsequent factor. 14 Eduarda Filipe Workshop Pr. 732

Nested design 1 1 A 1 1 1 A 2 1 1 B 1 1 1 B 2 M = 2 Measurements T = 2 Factor T D = 10 Factor D P = 2 1 2 Factor P Observer 15 Eduarda Filipe Workshop Pr. 732

The nested or hierarchical design. General model For one factor with a different levels taken randomly from a large population, where Mi is the expected (random) value of the group of observations i, m the overall mean, i the i th group random effect and eij the random error component. 16 Eduarda Filipe Workshop Pr. 732

The nested or hierarchical design. General model • For the hypothesis testing, the errors and the factor-levels effects are assumed to be normally and independently distributed, respectively eij ~ N (0, s 2) and i ~ N (0, s 2). • The variance of any observation y is composed by the sum of the variance components 17 Eduarda Filipe Workshop Pr. 732

The nested or hierarchical design. General model • The test is unilateral and the hypotheses are: • That is, if the null hypothesis is true, all factor-effects are “equal” and each observation is made up of the overall mean plus the random error eij ~ N (0, s 2). 18 Eduarda Filipe Workshop Pr. 732

The nested or hierarchical design. General model The total sum of squares (SST), a measure of total variability in the data, may be expressed by: SSFactor SSE = 0 SSFactor sum of squares of differences between factor-level averages and the grand average or a measure of the differences between factor-level SSE a sum of squares of the differences of observations within a factorlevel from the factor-level average, due to the random error 19 Eduarda Filipe Workshop Pr. 732

The nested or hierarchical design. General model Dividing each sum of squares by the respectively degrees of freedom, we obtain the corresponding mean squares (MS): MSFactor - mean square between factorlevel is: • an unbiased estimate of the variance s 2, if H 0 is true • or a surestimate of s 2, if H 0 is false. MSError - mean square within factor (error) is always 20 • the unbiased estimate of the variance s 2. Eduarda Filipe Workshop Pr. 732

The nested or hierarchical design. General model To test the hypotheses, we use the statistic: F is the Fisher sampling distribution with a and a x (n -1) degrees of freedom If F 0 > Fa, a-1, a(n-1) , we reject the null hypothesis. and conclude that the variance s 2 is significantly different of zero. 21 Eduarda Filipe Workshop Pr. 732

The nested or hierarchical design. General model The expected value of the MSFactor is: and the variance component of the factor is obtained by: 22 Eduarda Filipe Workshop Pr. 732

The nested or hierarchical design. General model Considering now the three level nested design of the figure the mathematical model is: yrdtm (rdtm) th observation m overall mean Pr P th random level effect Dd D th random level effect Tt T th random level effect ertdm random error component. 23 Eduarda Filipe Workshop Pr. 732

The nested or hierarchical design. General model The errors and the level effects are assumed to be normally and independently distributed, respectively with mean zero and variance s 2 and with mean zero and variances sr 2, sd 2 and s 2. The variance of any observation is composed by the sum of the variance components and the total number of measurements N is obtained by the product of the dimensions N = P × D × T × M. 24 Eduarda Filipe Workshop Pr. 732

The nested or hierarchical design. General model The total variability of the data can be expressed by 25 Eduarda Filipe Workshop Pr. 732

The nested or hierarchical design. General model • Dividing by the respective degrees of freedom P – 1 P (D -1) P D (T -1) P D T (M -1) • Equating the mean squares to their expected values • Solving the resulting equations • We obtain the estimates of the components of the variance. 26 Eduarda Filipe Workshop Pr. 732

The nested or hierarchical design. General model 27 Eduarda Filipe Workshop Pr. 732

Example of the comparison of two thermometric Water triple point cells in a three-stage nested experiment Two water cells, JA and HS Two standard platinum resistance thermometers (SPRTs) A and B. A – water vapour B – water in the liquid phase C – Ice mantle D – thermometer (SPRT) well t = 0, 01 ºC 28 Eduarda Filipe Workshop Pr. 732

v. Four measurement differences obtained daily with the two SPRTs v. This set of measurements repeated during ten consecutive days. v. And two weeks after a complete run was repeated Run 2/Plateau 2. 29 Eduarda Filipe Workshop Pr. 732

Example of the comparison of two thermometric Water triple point cells in a three-stage nested experiment § In this nested experiment, are considered the effects of: § Factor‑P from the Plateaus (P = 2) § Factor-D from the Days (D = 10) for the same Plateau § Factor-T from the Thermometers (T = 2) for the same Day and for the same Run § the variation between Measurements (M = 2) for the same Thermometer, the same Day and for the same Plateau or the residual variation. 30 Eduarda Filipe Workshop Pr. 732

Example of the comparison of two thermometric Water triple point cells in a three-stage nested experiment 31 Eduarda Filipe Workshop Pr. 732

Example of the comparison of two thermometric Water triple point cells in a three-stage nested experiment Schematic representation of the observed temperatures differences 32 Eduarda Filipe Workshop Pr. 732

Example of the comparison of two thermometric Water triple point cells in a three-stage nested experiment Analysis of variance table F 0 values compared with the critical values Fn 1, n 2 (a = 5%) F 0, 05, 1, 18 = 4, 4139 for the Plateau/Run effect F 0, 05, 18, 20 = 2, 1515 for the Days effect and 33 F 0, 05, 20, 40 =1, 8389 for the Thermometers effect. Eduarda Filipe Workshop Pr. 732

Example of the comparison of two thermometric Water triple point cells in a three-stage nested experiment v F 0 values are inferior to the F distribution for the factor-days and factor-thermometers so the null hypotheses are not rejected. v At the Plateau factor, the null hypothesis is rejected for a = 5% so a significant difference exists between the two Plateaus 34 Eduarda Filipe Workshop Pr. 732

Example of the comparison of two thermometric Water triple point cells in a three-stage nested experiment • equating the mean squares to their expected values • we can calculate the variance components and • include them in the budget of the components of uncertainty 35 Eduarda Filipe Workshop Pr. 732

Example of the comparison of two thermometric Water triple point cells in a three-level nested experiment Uncertainty budget for components of uncertainty evaluated by a Type A method 36 Eduarda Filipe Workshop Pr. 732

Example of the comparison of two thermometric Water triple point cells in a three-level nested experiment • These components of uncertainty, evaluated by Type A method, reflect the random components of variance due to the factors effects • This model for uncertainty evaluation that takes into account the time-dependent sources of variability it is foreseen by the GUM 37 Eduarda Filipe Workshop Pr. 732

Example of the comparison of two thermometric Water triple point cells in a three-level nested experiment Comparison of different approaches • Components of uncertainty evaluated by a Type A method – Nested structure: u. A= 20, 5 m. K – Standard deviation of the mean of 80 measurements u. A= 2, 2 m. K – Standard deviation of the mean 1 st Plateau (40 measurements) u. A= 2, 8 m. K (the 2 nd plateau considered as reproducibility and evaluated by a type B method) 38 Eduarda Filipe Workshop Pr. 732

Concluding Remarks v The plateaus values continuously taken where previously analyzed in terms of its normality. v The nested-hierarchical design was described as a tool to identify and evaluate components of uncertainty arising from random effects. v Applied to measurement, it is suitable to calculate the components evaluated by a Type A method standard uncertainty in time-dependent situations. 39 Eduarda Filipe Workshop Pr. 732

Concluding Remarks v An application of the design has been used to illustrate the variance components analysis in a three-factor nested model of a short, medium and long-term comparison of two thermometric fixed points. v The same model can be applied to other staged designs, easily treated in an Excel spreadsheet. 40 Eduarda Filipe Workshop Pr. 732

REFERENCES 41 1. BIPM et al, Guide to the Expression of Uncertainty in Measurement (GUM), 2 nd ed. , International Organization for Standardization, Genève, 1995, pp 11, 83 -87. 2. ISO 3534 -3, Statistics – Vocabulary and Symbols – Part 3: Design of Experiments, 2 nd ed. , Genève, International Organization for Standardization, 1999, pp. 31 (2. 6) and 40 -42 (3. 4) 3. Milliken, G. A. , Johnson D. E. , Analysis of Messy Data. Vol. I: Designed Experiments. 1 st ed. , London, Chapmann & Hall, 1997. 4. Montgomery, D. , Introduction to Statistical Quality Control, 3 rd ed. , New York, John Wiley & Sons, 1996, pp. 496 -499. 5. ISO TS 21749 Measurement uncertainty for metrological applications — Simple replication and nested experiments Genève, International Organization for Standardization, 2005. 6. Guimarães R. C. , Cabral J. S. , Estatística, 1 st ed. , Amadora: Mc-Graw Hill de Portugal, 1999, pp. 444 -480. 7. Murteira, B. , Probabilidades e Estatística. Vol. II, 2 nd ed. , Amadora, Mc-Graw Hill de Portugal, 1990, pp. 361 -364. 8. Box, G. E. P. , Hunter, W. G. , Hunter J. S. , Statistics for Experimenters. An Introduction to Design, Data Analysis and Model Building, 1 st ed. , New York, John Wiley & Sons, 1978, pp. 571 -582. 9. Poirier J. , “Analyse de la Variance et de la Régression. Plans d’Experience”, Techniques de l’Ingenieur, R 1, 1993, pp. R 260 -1 to R 260 -23. Eduarda Filipe Workshop Pr. 732

Thank you! 42 Eduarda Filipe Workshop Pr. 732