Standard Errors Beside reporting a value of a

Standard Errors • Beside reporting a value of a point estimate we should consider some indication of its precision. • For this we usually quote standard error. • Standard error of an estimator is the standard deviation of its sampling distribution. STA 248 week 4 1

Example • Suppose X 1, X 2, …, Xn are iid N(μ, σ2) and we are interested in estimating μ. • We have seen that both the method of moments estimator and the MLE for μ is • The standard error of is: • If σ is also an unknown parameter, we substitute an estimate for σ, say • The estimated standard error of is STA 248 week 4 2

Example • Bernoulli example. . . STA 248 week 4 3

Some Special Cases of Sampling Distributions • Suppose X 1, X 2, …, Xn are iid N(μ, σ2). The estimator of μ is have that . We • Suppose X 1, X 2, …, Xn are iid N(μ, σ2). The estimator of σ2 is s 2 - the sample standard deviation. The sampling distribution of s 2 is given by This is the Chi-squared distribution with parameter n 1. The parameter is called the “degrees of freedom”. • Some properties of the Chi-squared distribution are… STA 248 week 4 4

Important Note • By the Central limit theorem, estimator that is the sum of iid random variables will have approximately a Normal distribution for large n. • Example… STA 248 week 4 5

Bootstrap Estimates of Standard Errors • Suppose we have an estimator of a parameter and we want to express its accuracy by its s. e. but its sampling distribution is too complicated to derive theoretically. • A possible solution for this problem is to use Bootstrap – substitute computation for theory. STA 248 week 4 6

Parametric Bootstrap • Suppose data are realization of a random variable with a probability distribution with density f (x | θ) with θ unknown. • We begin the bootstrap process by first estimating θ from the data to get • Next we simulate B “bootstrap samples” from the density f (x | θ) with θ being replaced by and for each bootstrap sample we calculate a “bootstrap estimate” of θ denoted by • Note that the bootstrap samples are always the same size as the original data set. • The bootstrap estimate of the s. e. of deviation of the bootstrap estimates STA 248 week 4 is the sample standard 7

Example • Consider a data set containing breakdown times of an insolative fluid between electrodes. • The theoretical model for this data assumes that this is an i. i. d sample from an exponential distribution… • The method of moment estimator of λ is…. • We want the s. e of this estimator and for this we use parametric bootstrap (see R). STA 248 week 4 8

Nonparametric Bootstrap • If we could take an infinite number of samples of size n from the probability distribution that generated the data and for each sample find , we would know the sampling distribution of. • In the nonparametric bootstrap procedure we get bootstrap samples of size n by re-sampling from the data. • Recall, the empirical distribution of the data puts probability mass 1/n at each data point and is used as the sampling distribution that generated the data. • Re-sampling is sampling with replacement from this empirical distribution. • See R for example… STA 248 week 4 9

Parametric Versus Nonparametric Bootstrap • In the parametric bootstrap we have to make an assumption about the form of the distribution that generated the data • Non-parametric – if n is small can behave oddly. STA 248 week 4 10