Order Statistics The order statistics of a set

Order Statistics • The order statistics of a set of random variables X 1, X 2, …, Xn are the same random variables arranged in increasing order. • Denote by X(1) = smallest of X 1, X 2, …, Xn X(2) = 2 nd smallest of X 1, X 2, …, Xn X(n) = largest of X 1, X 2, …, Xn • Note, even if Xi’s are independent, X(i)’s can not be independent since X(1) ≤ X(2) ≤ … ≤ X(n) • Distribution of Xi’s and X(i)’s are NOT the same. week 2 1

Distribution of the Largest order statistic X(n) • Suppose X 1, X 2, …, Xn are i. i. d random variables with common distribution function FX(x) and common density function f. X(x). • The CDF of the largest order statistic, X(n), is given by • The density function of X(n) is then week 2 2

Example • Suppose X 1, X 2, …, Xn are i. i. d Uniform(0, 1) random variables. Find the density function of X(n). week 2 3

Distribution of the Smallest order statistic X(1) • Suppose X 1, X 2, …, Xn are i. i. d random variables with common distribution function FX(x) and common density function f. X(x). • The CDF of the smallest order statistic X(1) is given by • The density function of X(1) is then week 2 4

Example • Suppose X 1, X 2, …, Xn are i. i. d Uniform(0, 1) random variables. Find the density function of X(1). week 2 5

Distribution of the kth order statistic X(k) • Suppose X 1, X 2, …, Xn are i. i. d random variables with common distribution function FX(x) and common density function f. X(x). • The density function of X(k) is week 2 6

Example • Suppose X 1, X 2, …, Xn are i. i. d Uniform(0, 1) random variables. Find the density function of X(k). week 2 7

Statistical Model • A statistical model for some data is a set of distributions, one of which corresponds to the true unknown distribution that produced the data. • The statistical model corresponds to the information a statistician brings to the application about what the true distribution is or at least what he or she is willing to assume about it. • The variable θ is called the parameter of the model, and the set Ω is called the parameter space. • From the definition of a statistical model, we see that there is a unique value , such that fθ is the true distribution that generated the data. We refer to this value as the true parameter value. week 2 8

Examples • Suppose there are two manufacturing plants for machines. It is known that the life lengths of machines built by the first plant have an Exponential(1) distribution, while machines manufactured by the second plant have life lengths distributed Exponential(1. 5). You have purchased five of these machines and you know that all five came from the same plant but do not know which plant. Further, you observe the life lengths of these machines, obtaining a sample (x 1, …, x 5) and want to make inference about the true distribution of the life lengths of these machines. • Suppose we have observations of heights in cm of individuals in a population and we feel that it is reasonable to assume that the distribution of height is the population is normal with some unknown mean and variance. The statistical model in this case is where Ω = R×R+, where R+ = (0, ∞). week 2 9

Point Estimate • Most statistical procedures involve estimation of the unknown value of the parameter of the statistical model. • A point estimate, , is an estimate of the parameter θ. It is a statistic based on the sample and therefore it is a random variable with a distribution function. • The standard deviation of the sampling distribution of an estimator is usually called the standard error of the estimator. • For a given statistical model with unknown parameter θ there could be more then one point estimate. • The parameter θ of a statistical model can have more then just one element. week 2 10

Properties of Point Estimators • Let be a point estimator for a parameter θ. Then estimator if is an unbiased • The bias of a point estimator is given by • The variance of a point estimator is Ideally we would like our estimator to have minimum variance. week 2 11

Mean Square Error of Point Estimators • The mean square error (MSE) of a point estimator is • Claim: week 2 12

Common Point Estimators • A natural estimate for the population mean μ is the sample mean (in any distribution). The sample mean is an unbiased estimator of the population mean. • A common estimator for the population variance is the sample variance s 2. week 2 13

Claim • Let X 1, X 2, …, Xn be random sample of size n from a normal population. The sample variance s 2 is an unbiased estimator of the population variance σ2. • Proof… week 2 14

Example • Suppose X 1, X 2, …, Xn is a random sample from U(0, θ) distribution. Let. Find the density of and its mean. Is unbiased? week 2 15

Asymptotically Unbiased Estimators • An estimator is asymptotically unbiased if • Example: week 2 16

Relative Efficiency • Given two estimators and of a parameter θ with variances respectively. The efficiency of relative to is the ratio • Interpretation… week 2 17

Example • In the uniform example above let and . Which point estimate is more efficient? week 2 18