Curve Fitting Goodness of Fit Curve Fitting What

Curve Fitting & Goodness of Fit

Curve Fitting What is ‘Curve Fitting’? Is the process of constructing a curve, or mathematical function, that has the best fit to a series of data points. Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Curve Fitting Why should I use curve fitting? Extract general trends from data, Model variability from a data set, Create predictive analytics with the data … Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Curve Fitting Interpolation vs Curve Fitting Interpolation : Given data for discrete values, fit a curve or a series of curves that pass directly through each of the points. — When data are very precise Curve Fitting (Least Squares regression): Given data for discrete values, derive a single curve that represents the general trend of the data. — When the given data exhibit a significant degree of error or noise. Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Curve Fitting 1. Linear curve fitting (linear regression) Given the general form of a straight line f(x)=ax+b How can we pick the coefficients that best fits the line to the data? First question: What makes a particular straight line a ‘good’ fit? Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Curve Fitting 1. Linear curve fitting (linear regression) Which line appears to fit the trend better? Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Curve Fitting 1. Linear curve fitting (linear regression) The red or the blue? Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Curve Fitting 1. Linear curve fitting (linear regression) Which line appears to fit the trend better? • Consider the distance between the data and points on the line • Add up the length of all the red and blue vertical lines • This is an expression of the ‘error’ between data and fitted line • The one line that provides a minimum error is then the ‘best’ straight line Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Curve Fitting 1. Linear curve fitting (linear regression) Quantifying error in a curve fit assumptions: 1) positive or negative error have the same value (data point is above or below the line) 2) Weight greater errors more heavily we can do both of these things by squaring the distance Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Curve Fitting 1. Linear curve fitting (linear regression) Quantifying the error in a curve fit assumptions: denote data values as (x, y) =======>> denote points on the fitted line as (x, f(x)) sum the error at the four data points Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Curve Fitting 1. Linear curve fitting (linear regression) Quantifying error in a curve fit assumptions: Our fit is a straight line, so now substitute f(x) = ax+b The ‘best’ line has minimum error between line and data points This is called the least squares approach, since we minimize the square of the error. Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Curve Fitting 1. Linear curve fitting (linear regression) Quantifying error in a curve fit assumptions: time to pull out the calculus. . . finding the minimum of a function 1) derivative describes the slope 2) slope = zero is a minimum ==> take the derivative of the error with respect to a and b, set each to zero Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Curve Fitting 1. Linear curve fitting (linear regression) Quantifying error in a curve fit assumptions: Solve for the a and b so the previous two equations both = 0 re-write these two equations Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Curve Fitting 1. Linear curve fitting (linear regression) Quantifying error in a curve fit assumptions: put these into matrix form what’s unknown? we have the data points (xi, yi) for i=1, …, n so we have all the summation terms in the matrix Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Curve Fitting 1. Linear curve fitting (linear regression) Quantifying error in a curve fit assumptions: so unknowns are a and b the coefficients a and b are solved >> X = A-1*B Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Curve Fitting What is happening with these data? Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Curve Fitting 2. Polynomial Curve Fitting Given the general form of a polynomial Just as was the case for linear regression, we ask: How can we pick the coefficients that best fits the curve to the data? We can use the same idea: The curve that gives minimum error between data and the fit is ‘best’ f(x) is ‘best’ Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Curve Fitting 2. Polynomial Curve Fitting Quantify the error for these two second order curves. . . • Add up the length of all the red and blue vertical lines • pick curve with minimum total error f(x)=α‧x 2+β‧x+γ Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Curve Fitting Degrees of Polynomial Curve Fitting Red line is a first degree polynomial, green line is second degree, orange line is third degree and blue is fourth degree Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Curve Fitting 3. Exponential Curve Fitting Q: Will a polynomial of any order necessarily fit any set of data? A: Nope, lots of phenomena don’t follow a polynomial form. They may be, for example, exponential. Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Curve Fitting Matlab – example … clear all; close all; clc plot(x, y’ko’) coeff=fminsearch(‘gauss_fit’, [1 1], [], x, y) function E=gauss_fit(x 0, x, y) E=sum( ( x 0(1)*exp(-x 0(2)*x. ^2)-y). ^2; A=coeff(1); B=coeff(2) x 2=-3: 0. 1: 3; y 2=A*exp(-B*x 2. ^2); hold on plot (x 2, y 2, ’m’, ’Linewidth’, [2]) Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Curve Fitting Matlab – more useful functions … poly / roots / polyval / conv / deconv / residue / polyfit / [p, fitted] = curvefit(x, y, n) / ypred=polyval(p, values) / figure, scatter(x, y. Neutral. Bg) / scatter(x, y, Dark. Bg) / Targets = […] / weights = ones(1, length(x)) Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Curve Fitting R – some useful functions … fit / lm(y~x) / predict / summary (fit) / lm(y~poly(x, 2, raw=TRUE)) / predict / lm(formula = y ~ x) / anova(fit, fit 2) / coef(fit) / yy <- third_order(xx, fit 3) / plot(xx, yy, type="l") / lines(x, y, col="red") Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Goodness of Fit Definition The goodness of fit test is used to test if sample data fits a distribution from a certain population. It tells you if your sample data represents the data you would expect to find in the actual population. Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Goodness of Fit Goodness of fit tests ü ü ü Giareni Eleftheria Kolmogorov–Smirnov test Cramér–von Mises criterion Anderson–Darling test Shapiro–Wilk test Chi-squared test Akaike information criterion Hosmer–Lemeshow test Kuiper's test kernelized Stein discrepancy[1] Zhang's ZK, ZC and ZA tests[2] Moran test Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Goodness of Fit 1. The Chi Square Goodness of Fit Test It is a non-parametric test that is used to find out how the observed value of a given phenomena is significantly different from the expected value. The term goodness of fit is used to compare the observed sample distribution with the expected probability distribution. Chi-Square goodness of fit test determines how well theoretical distribution fits the empirical distribution. In Chi-Square goodness of fit test, sample data is divided into intervals. Then the numbers of points that fall into the interval are compared, with the expected numbers of points in each interval. Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Goodness of Fit 1. The Chi Square Goodness of Fit Test Procedure for Chi-Square Goodness of Fit Test Set up the hypothesis for Chi-Square goodness of fit test: A. Null hypothesis: assumes that there is no significant difference between the observed and the expected value. B. Alternative hypothesis: assumes that there is a significant difference between the observed and the expected value. Compute the value of Chi-Square goodness of fit test using the following formula: Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Goodness of Fit 1. The Chi Square Goodness of Fit Test Procedure for Chi-Square Goodness of Fit Test Degree of freedom: In Chi-Square goodness of fit test, the degree of freedom depends on the distribution of the sample. The following table shows the distribution and an associated degree of freedom: Type of distribution No of constraints Degree of freedom Binominal distribution 1 n-1 Poisson distribution 2 n-2 Normal distribution 3 n-3 Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Goodness of Fit 1. The Chi Square Goodness of Fit Test Procedure for Chi-Square Goodness of Fit Test Hypothesis testing: Hypothesis testing in Chi-Square goodness of fit test is the same as in other tests, like t-test, ANOVA, etc. The calculated value of Chi-Square goodness of fit test is compared with the table value. If the calculated value of Chi-Square goodness of fit test is greater than the table value, we will reject the null hypothesis and conclude that there is a significant difference between the observed and the expected frequency. If the calculated value of Chi-Square goodness of fit test is less than the table value, we will accept the null hypothesis and conclude that there is no significant difference between the observed and expected value. Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Curve Fitting Matlab – some useful functions … Shapiro – Wilk Test fx : = stats: : normal. Random(0, 1/2): x : = fx() $ k = 1. . 1000 fy : = stats: : poisson. Random(10): y : = fy() $ k = 1. . 1000: stats: : sw. GOFT(x); / stats: : sw. GOFT(y) Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology

Goodness of Fit R – some useful functions … Kolmogorov – Smirnov ecdf / pnorm / pbinom / Shapiro – Wilk shapiro. test(sample) Giareni Eleftheria Curve Fitting & Goodness Of Fit Visual Perception and Psychology