Two Parameter normal We considered estimating the mean

  • Slides: 13
Download presentation
Two Parameter normal • We considered estimating the mean alone or the variance alone

Two Parameter normal • We considered estimating the mean alone or the variance alone • This lecture deals with estimating both together • We will consider various techniques of obtaining statistics from these distributions.

Posterior expression • objective – Estimate unknown two parameters q={m, s 2} of normal

Posterior expression • objective – Estimate unknown two parameters q={m, s 2} of normal distribution based on observations y = {y 1, y 2, …}. • Joint pdf of non-informative prior • Joint posterior distribution • With some algebra • Function of two variables - can be plotted as a surface or contour. No Matlab pdf.

Predictive distribution • Posterior Distribution • Predictive distribution of new y based on observed

Predictive distribution • Posterior Distribution • Predictive distribution of new y based on observed y. Likelihood of new y Posterior pdf of m & s 2 • Analytical solution can be found (and on Matlab)

Simulation of joint posterior distribution • Simulation by drawing random samples – We can

Simulation of joint posterior distribution • Simulation by drawing random samples – We can evaluate characteristics of joint posterior pdf using simulation techniques. • Why ? – Even when analytic solutions available, some are not easy to evaluate. – Once exercised, may find it more convenient and more general. – Practice simulation & validate with analytic solution. • Here we will do that using marginal distributions

Which statement is wrong 1. Marginal distributions depend only one random variable. 2. The

Which statement is wrong 1. Marginal distributions depend only one random variable. 2. The predictive distribution is wider than the posterior distribution. 3. In the case of no-prior, the posterior distribution is equal to the normalized likelihood.

Marginal distributions • Posterior – Marginal mean – Student t-distribution (Wikipedia) – Marginal variance

Marginal distributions • Posterior – Marginal mean – Student t-distribution (Wikipedia) – Marginal variance • Need to review how to calculate pdf and cdf of a transformed variable.

Simulation of joint posterior distribution • Approach 1: use marginal variance. Conditional pdf of

Simulation of joint posterior distribution • Approach 1: use marginal variance. Conditional pdf of m on s 2 Marginal pdf of s 2 – Conditional pdf of m on s 2 is already derived, which is the pdf of mean with known variance. – Marginal pdf of s 2 is given in slide 6. – In order to sample the posterior pdf of p(m, s 2|y) 1. Draw s 2 from the marginal pdf 2. Draw m from the conditional pdf

Practice • Approach 1: use marginal variance. – Once you have obtained samples of

Practice • Approach 1: use marginal variance. – Once you have obtained samples of joint pdf, compare (validate) results with the analytic solution. 1. Compare the samples of (M, S 2) with the analytic joint pdf. • In terms of scattered plot & contour. 2. Compare the samples of M with the marginal pdf, which is t distribution. 3. Compare the samples of S 2 with the marginal pdf which is invchi 2 distribution. 4. Extract features of the samples M and compare with analytic solution. 5. Extract features of the samples S 2 and compare with analytic solution.

Approach 2: use marginal mean Conditional pdf of s 2 on m Marginal pdf

Approach 2: use marginal mean Conditional pdf of s 2 on m Marginal pdf of m – Conditional pdf of s 2 on m is already derived, which is the pdf of variance with known mean. – Marginal pdf of m is given in slide 6 – In order to sample the posterior pdf of p(m, s 2|y) 1. Draw m from the marginal pdf 2. Draw s 2 from the conditional pdf – Compare (validate) samples with analytic solution.

Predictive distribution by simulation – Once we have posterior distribution for m & s

Predictive distribution by simulation – Once we have posterior distribution for m & s 2 in the form of samples, the predictive new y are easily obtained by drawing each one from conditional on each individual m & s 2. – Mean & conf. intervals of posterior and predictive can be obtained easily.

Homework

Homework

Homework -continued

Homework -continued

Homework – to do: Estimate the results that Newcomb probably drew from his measurements

Homework – to do: Estimate the results that Newcomb probably drew from his measurements by omitting the two outliers, which are at -5 and -45 on the figure. That is, estimate the posterior median of the mean and 95% confidence interval. Source: Smithsonian Institution Number: 2004 -57325