Statistics Sampling Distributions and Point Estimation of Parameters

Statistics Sampling Distributions and Point Estimation of Parameters Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5 th Edition, by Douglas C. Montgomery, John Wiley &

Point Estimation Statistic A function of observations, , , …, Also a random variable Sample mean Sample variance Its probability distribution is called a sampling distribution Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5 th Edition, by Douglas C. Montgomery, John Wiley &

Point estimator of ◦ A statistic Point estimate of ◦ A particular numerical value Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5 th Edition, by Douglas C. Montgomery, John Wiley &

Mean ◦ The estimate Variance ◦ The estimate Proportion ◦ The estimate ◦ is the number of items that belong to the class of interest Difference in means, ◦ The estimate Difference in two proportions ◦ The estimate

Sampling Distributions and the Central Limit Theorem Random sample ◦ The random variables , , …, are a random sample of size if (a) the ‘s are independent random variables, and (b) every has the same probability distribution

If , , …, are normally and independently distributed with mean and variance ◦ has a normal distribution ◦ with mean ◦ variance Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5 th Edition, by Douglas C. Montgomery, John Wiley &

Central Limit Theorem ◦ If , , …, is a random sample of size taken from a population (either finite or infinite) with mean and finite variance , and if is the sample mean, the limiting form of the distribution of ◦ as , is the standard normal distribution. Works ◦ when , regardless of the shape of the population ◦ , if not severely nonnormal

Two independent populations with means and , and variances and ◦ is approximately standard normal, or ◦ is exactly standard normal if the two populations are normal Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5 th Edition, by Douglas C. Montgomery, John Wiley &

Example 7 -1 Resistors ◦ An electronics company manufactures resistors that have a mean resistance of 100 ohms and a standard deviation of 10 ohms. The distribution of resistance is normal. Find the probability that a random sample of resistors will have an average resistance less than 95 ohms Example 7 -2 Central Limit Theorem ◦ Suppose that has a continuous uniform distribution ◦ Find the distribution of the sample mean of a random sample of size

Example 7 -3 Aircraft Engine Life ◦ The effective life of a component used in a jetturbine aircraft engine is a random variable with mean 5000 hours and standard deviation 40 hours. The distribution of effective life is fairly close to a normal distribution. The engine manufacturer introduces an improvement into the manufacturing process for this component that increases the mean life to 5050 hours and decreases the standard deviation to 30 hours. Suppose that a random sample of components is selected from the “old” process and a random sample of components is selected from the “improved” process. What is the probability that the difference in the two sample means is at least 25 hours? Assume that the old and improved processes can

Exercise 7 -10 ◦ Suppose that the random variable continuous uniform distribution has the ◦ Suppose that a random sample of observations is selected from this distribution. What is the approximate probability distribution of ? Find the mean and variance of this quantity. Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5 th Edition, by Douglas C. Montgomery, John Wiley &

General Concepts of Point Estimation Bias of the estimator is an unbiased estimator if Minimum variance unbiased estimator (MVUE) ◦ For all unbiased estimator of , the one with the smallest variance is the MVUE for ◦ If , , …, are from a normal distribution with mean and variance

Standard error of an estimator Estimated ◦ or standard error or If is normal with mean variance and Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5 th Edition, by Douglas C. Montgomery, John Wiley &

Mean squared error of an estimate Relative efficiency of to Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5 th Edition, by Douglas C. Montgomery, John Wiley &

Example 7 -4 Sample Mean and Variance Are Unbiased ◦ Suppose that is a random variable with mean and variance. Let , , …. , be a random sample of size from the population represented by. Show that the sample mean and sample variance are unbiased estimators of and , respectively. Example 7 -5 Thermal Conductivity ◦ Ten measurements of thermal conductivity were obtained: ◦ 41. 60, 41. 48, 42. 34, 41. 95, 41. 86 ◦ 42. 18, 41. 72, 42. 26, 41. 81, 42. 04 ◦ Show that and

Exercise ◦ 7 -31 and are the sample mean and sample variance from a population with mean and variance. Similarly, and are the sample mean and sample variance from a second independent population with mean and variance. The sample sizes are and , respectively. ◦ (a) Show that is an unbiased estimator of ◦ ◦ (b) Find the standard error of. How could you estimate the standard error? ◦ (c) Suppose that both populations have the same variance; that is, . Show that

Methods of Point Estimation Moments ◦ Let , , …, be a random sample from the probability distribution , where can be a discrete probability mass function or a continuous probability density function. The th population moment (or distribution moment) is , = 1, 2, …. The corresponding th sample moment is = 1, 2, …. Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5 th Edition, by Douglas C. Montgomery, John Wiley &

Moment estimators ◦ Let , , …, be a random sample from either a probability mass function or a probability density function with unknown parameters , , …, . The moment estimators , , …, are found by equating the first population moments to the first sample moments and solving the resulting equations for the unknown parameters. Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5 th Edition, by Douglas C. Montgomery, John Wiley &

Maximum likelihood estimator ◦ Suppose that is a random variable with probability distribution , where is a single unknown parameter. Let , , …, be the observed values in a random sample of size. Then the likelihood function of the sample is ◦ Note that the likelihood function is now a function of only the unknown parameter. The maximum likelihood estimator (MLE) of is the value of that maximizes the likelihood function.

Properties of a Maximum Likelihood Estimator ◦ Under very general and not restrictive conditions, when the sample size is large and if is the maximum likelihood estimator of the parameter , ◦ (1) is an approximately unbiased estimator for ◦ (2) the variance of is neatly as small as the variance that could be obtained with any other estimator, and ◦ (3) has an approximate normal distribution. Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5 th Edition, by Douglas C. Montgomery, John Wiley &

Invariance property ◦ Let , , …. , be the maximum likelihood estimators of the parameters , , …, . Then the maximum likelihood estimator of any function of these parameters is the same function ◦ of the estimators , , …, . Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5 th Edition, by Douglas C. Montgomery, John Wiley &

Bayesian estimation of parameters ◦ Sample , , …, ◦ Joint probability distribution ◦ Prior distribution for ◦ Posterior distribution for ◦ Marginal distribution

Example 7 -6 Exponential Distribution Moment Estimator ◦ Suppose that , , …, is a random sample from an exponential distribution with parameter. For the exponential, . ◦ Then results in. Example 7 -7 Normal Distribution Moment Estimators ◦ Suppose that , , …, is a random sample from a normal distribution with parameters and. For the normal distribution, and ◦. Equating to and to gives ◦ and

Example 7 -8 Gamma Distribution Moment Estimators ◦ Suppose that , , …, is a random sample from a gamma distribution with parameters and , For the gamma distribution, and ◦. Solve ◦ and Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5 th Edition, by Douglas C. Montgomery, John Wiley &

Example 7 -9 Bernoulli Distribution MLE ◦ Let be a Bernoulli random variable. The probability mass function is ◦ where is the parameter to be estimated. The likelihood function of a random sample of size is ◦ Find that maximizes . Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5 th Edition, by Douglas C. Montgomery, John Wiley &

Example 7 -10 Normal Distribution MLE ◦ Let be normally distributed with unknown and known variance. The likelihood function of a random sample of size , say , , …, , is ◦ Find . Example 7 -11 Exponential Distribution MLE ◦ Let be exponentially distributed with parameter. The likelihood function of a random sample of size , say , , …, is ,

Example 7 -12 Normal Distribution MLEs for and ◦ Let be normally distributed with mean and variance , where both and are unknown. The likelihood function of a random sample of size is ◦ Find and Example . 7 -13 ◦ From Example 7 -12, to obtain the maximum likelihood estimator of the function ◦ Substitute the estimators and into the function , which yields

Example 7 -14 Uniform Distribution MLE ◦ Let be uniformly distributed on the interval 0 to. Since the density function is for and zeros otherwise, the likelihood function of a random sample of size is ◦ for ◦ Find , , …, . Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5 th Edition, by Douglas C. Montgomery, John Wiley &

Example 7 -15 Gamma Distribution MLE ◦ Let , , …, be a random sample from the gamma distribution. The log of likelihood function is ◦ Find that Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5 th Edition, by Douglas C. Montgomery, John Wiley &

Example 7 -16 Bayes Estimator for the Mean of a Normal Distribution ◦ Let , , …, be a random sample from the normal distribution with mean and variance , where is unknown and is known. Assume that the prior distribution for is normal with mean and variance ; that is, ◦ The joint probability distribution of the sample is Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5 th Edition, by Douglas C. Montgomery, John Wiley &

◦ Show that ◦ Then the Bayes estimate of is Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5 th Edition, by Douglas C. Montgomery, John Wiley &

Exercise 7 -42 ◦ Let , , …, be uniformly distributed on the interval 0 to. Recall that the maximum likelihood estimator of is. ◦ (a) Argue intuitively why cannot be an unbiased estimator for. ◦ (b) Suppose that. Is it reasonable that consistently underestimates ? Show that the bias in the estimator approaches zero as gets large. ◦ (c) Propose an unbiased estimator for. ◦ (d) Let. Use the fact that if and only if each to derive the cumulative distribution function of. Then show that the probability density function of is

◦ Use this result to show that the maximum likelihood estimator for is biased. ◦ (e) We have two unbiased estimators for : the moment estimator and ◦ , where is the largest observation in a random sample of size. It can be shown that and that ◦. Show that if , is a better estimator than. In what sense is it a better estimator of ? Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5 th Edition, by Douglas C. Montgomery, John Wiley &

Exercise 7 -50 ◦ The time between failures of a machine has an exponential distribution with parameter. Suppose that the prior distribution for is exponential with mean 100 hours. Two machines are observed, and the average time between failures is hours. ◦ (a) Find the Bayes estimate for. ◦ (b) What proportion of the machine do you think will fail before 1000 hours? Contents, figures, and exercises come from the textbook: Applied Statistics and Probability for Engineers, 5 th Edition, by Douglas C. Montgomery, John Wiley &