Point Estimation Notes of STAT 6205 by Dr
Point Estimation Notes of STAT 6205 by Dr. Fan
Overview • • • Section 6. 1 Point estimation Maximum likelihood estimation Methods of moments Sufficient statistics o Definition o Exponential family o Mean square error (how to choose an estimator) 6205 -Ch 6 2
Big Picture • Goal: To study the unknown distribution of a population • Method: Get a representative/random sample and use the information obtained in the sample to make statistical inference on the unknown features of the distribution • Statistical Inference has two parts: o Estimation (of parameters) o Hypothesis testing • Estimation: o Point estimation: use a single value to estimate a parameter o Interval estimation: find an interval covering the unknown parameter 6205 -Ch 6 3
Point Estimator • Biased/unbiased: an estimator is called unbiased if its mean is equal to the parameter of estimate; otherwise, it is biased • Example: X_bar is unbiased for estimating mu 6205 -Ch 6 4
Maximum Likelihood Estimation • Given a random sample X 1, X 2, …, Xn from a distribution f(x; q) where q is a (unknown) value in the parameter space W. • Likelihood function vs. joint pdf • Maximum Likelihood Estimator (m. l. e. ) of q, denoted as is the value q which maximizes the likelihood function, given the sample X 1, X 2, …, Xn. 6205 -Ch 6 5
Examples/Exercises • Problem 1: To estimate p, the true probability of heads up for a given coin. • Problem 2: Let X 1, X 2, …, Xn be a random sample from a Exp(mu) distribution. Find the m. l. e. of mu. • Problem 3: Let X 1, X 2, …, Xn be a random sample from a Weibull(a=3, b) distribution. Find the m. l. e. of b. • Problem 4: Let X 1, X 2, …, Xn be a random sample from a N(m, s^2) distribution. Find the m. l. e. of m and s. • Problem 5: Let X 1, X 2, …, Xn be a random sample from a Weibull(a, b) distribution. Find the m. l. e. of a and b. 6205 -Ch 6 6
Method of Moments • Idea: Set population moments = sample moments and solve for parameters • Formula: When the parameter q is r-dimensional, solve the following equations for q: 6205 -Ch 6 7
Examples/Exercises Given a random sample from a population • Problem 1: Find the m. m. e. of m for a Exp(m) population. • Exercise 1: Find the m. m. e. of m and s for a N(m, s^2) population. 6205 -Ch 6 8
Sufficient Statistics • Idea: The “sufficient” statistic contains all information about the unknown parameter; no other statistic can provide additional information as to the unknown parameter. • If for any event A, P[A|Y=y] does not depend on the unknown parameter, then the statistic Y is called “sufficient” (for the unknown parameter). • Any one-to-one mapping of a sufficient statistic Y is also sufficient. • Sufficient statistics do not need to be estimators of the parameter. 6205 -Ch 6 9
Sufficient Statistics 6205 -Ch 6 10
Examples/Exercises Let X 1, X 2, …, Xn be a random sample from f(x) Problem: Let f be Poisson(a). Prove that 1. X-bar is sufficient for the parameter a 2. The m. l. e. of a is a function of the sufficient statistic Exercise: Let f be Bin(n, p). Prove that X-bar is sufficient for p (n is known). Hint: find a sufficient statistic Y for p and then show that X-bar is a function of Y 6205 -Ch 6 11
Exponential Family 6205 -Ch 6 12
Examples/Exercises Example 1: Find a sufficient statistic for p for Bin(n, p) Example 2: Find a sufficient statistic for a for Poisson(a) Exercise: Find a sufficient statistic for m for Exp(m) 6205 -Ch 6 13
Joint Sufficient Statistics Example: Prove that X-bar and S^2 are joint sufficient statistics for m and s of N(m, s^2) 6205 -Ch 6 14
Application of Sufficience 6205 -Ch 6 15
Example Consider a Weibull distribution with parameter(a=2, b) 1) Find a sufficient statistic for b 2) Find an unbiased estimator which is a function of the sufficient statistic found in 1) 6205 -Ch 6 16
Good Estimator? • Criterion: mean square error 6205 -Ch 6 17
Example • Which of the following two estimator of variance is better? 6205 -Ch 6 18
- Slides: 18