LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES In this slideshow we will explore the limiting distribution of the OLS estimator of the (slope) parameter of a univariate time series with one lag. 1

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES This time series is the simplest possible example of an ADL(1, 0) model. We will study it because the asymptotic properties of the OLS estimator of the slope coefficient are easy to establish. The issues that we will encounter arise in all models with lagged dependent variables. 2

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES We shall assume throughout that │b 2│<1, which means that the process is stationary. The meaning of stationarity will be explained in Chapter 13, together with the consequences of the violation of this condition. We shall in practice assume b 2 is positive and so 0 < b 2 < 1. 3

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES In the previous slideshow we explored the properties of the estimator graphically with a simulation. Now we shall be more analytical. 4

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES We shall assume throughout that ut is iid with zero mean and finite variance. us is a determinant of the explanatory variable Yt– 1 in observations s+1 to T and so the second part of Assumption C. 7 is violated. 5

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES We cannot obtain a closed-form expression for the expectation of b 2 and we have seen, through simulation, that in practice it will be biased for finite samples. 6

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES Despite the finite-sample bias, the simulation indicated that the estimator is consistent. We will now demonstrate this analytically. 7

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES In the third line, the plim of the error term is the plim of a ratio. The next step is to use the rule plim(A/B) = plim A / plim B, To use this rule, we must first show that both plim A and plim B exist, and that plim B is not zero. 8

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES We will begin by demonstrating that, apart from transitory initial effects, E(Yt) = 0 and hence plim Y = 0. Since the model is true for time t, it is also true for time t– 1. 9

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES Substitute for Yt– 1 in the first equation. 10

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES Lagging and substituting in this way t times, we express Yt in terms of the initial value Y 0 and the values of the disturbance term. 11

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES When we take expectations, the expectations of all of the values of the disturbance term are zero. The remaining term tends to zero in the limit because b 2 t tends to zero if │b 2│<1. 12

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES Now we can invoke the Weak Law of Large Numbers to say that the plim of the sample mean of Y is equal to the population mean, which is zero in the limit. 13

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES We can invoke the same law to state immediately that the plim of the sample mean of u is zero. 14

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES We can invoke the same law one more time to state that the plim of the sample mean square deviation of Yt– 1 is the variance of Y. 15

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES (Small completely unimportant technical point: Y here is understood to be the mean of Yt– 1. If Y 0 is known, the summation is as shown. If Y 0 is not known, the summation is from 2 to T. Since we are talking about the limit as T tends to infinity, this detail does not matter at all. ) 16

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES Finally, we can establish the last line by means of a continuous mapping theorem. Of course, for each of these results various conditions have to be satisfied. For the present model there is no problem, given that we are assuming that u is iid and │b 2│<1. 17

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES Now, making use of the identity shown and taking plims, we have one result that we need. 18

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES Similarly, making use of another identity and taking plims, we obtain a second result. 19

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES With this preparatory work in place, we are able to omplete the proof of consistency. We have shown that both the numerator and the denominator of the error term have limits. 20

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES The variance of Yt– 1 will be positive since the variance of u is positive. cov(Yt– 1, ut) = 0 because Yt– 1 is independent of ut. Hence b 2 is a consistent estimator of b 2. 21

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES Standard theory tells us that b 2 is asymptotically normally distributed. What does this mean? We have just shown that, as the sample size increases, the distribution degenerates to a spike at b 2, so how can we say that b 2 has an asymptotically normal distribution? 22

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES We encountered this problem when determining the asymptotic properties of IV estimators in Chapter 8. To deal with it, we again use the technique involving the use of a central limit theorem discussed in Section R. 15. 23

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES As a first step, we multiply the estimator by √T. This is sufficient to prevent the variance from tending to zero as T increases. However, (√T)b 2 does not have a limiting distribution, either, because b 2 tends to b 2 and (√T)b 2 increases without limit with T. 24

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES So, instead, we consider (√T)(b 2 – b 2). For the model under discussion, it can be shown that , provided that │b 2│< 1, the conditions for the application of a central limit theorem are satisfied and that the limiting distribution is normal with zero mean and variance (1 – b 22). 25

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES This asymptotic result is all that we have in analytical terms. We are not entitled to say anything analytically for finite samples. However, given the limiting distribution, we can start working back tentatively to finite samples and make some plausible assertions. 26

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES We can say, that for large T, the relationship may hold approximately. If this is the case, dividing the statistic by √T, we obtain the result shown, as an approximation, for sufficiently large samples. 27

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES Hence, adding b 2 to the statistic, we can say, that b 2 is distributed as shown, as an approximation, for sufficiently large samples. 28

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES Of course, there remains the question of what might be considered to be a ‘sufficiently large’ sample. To answer this question, we turn to simulation. 29

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES Simulation reveals that the answer depends on the value of b 2 itself. We will start by putting b 2 = 0. 6. The figure shows the distributions of √T(b 2 – 0. 6) for T = 25, 50, 100, and 200. 30

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES For the simulation, the disturbance term was drawn randomly from a normal distribution with zero mean and unit variance. 31

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES According to theory, the distribution of ought to converge to a normal distribution with mean zero and variance. This limiting normal distribution is shown as the red curve in the figure. 32

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES Although the overall shape of √T(b 2 – 0. 6) is not far from normal, even for T as small as 25, there are serious discrepancies in the tails, and it is the shape of the tails that matters for inference. 33

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES Even for T = 200, the left tail is far too fat and the right tail far too thin. This implies that we should not expect the N( b 2, (1 – b 22) / T ) to be an accurate guide to the actual distribution of b 2. 34

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES This is confirmed by the figure shown. It compares the actual distribution of b 2 for T = 100, obtained by simulation, with theoretical distribution (still with b 2 = 0. 6). 35

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES (Since is just a linearly scaled function of b 2, the relationship between the actual distribution of b 2 and its theoretical distribution is parallel to that between for T = 100 and its limiting normal distribution in the previous figure. ) 36

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES The finite-sample bias is the stronger, the closer that b 2 is to 1. The figure shows the distribution of √T(b 2 – b 2) when b 2 = 0. 9. In this case, it is clear that, even for T = 200, the distribution is far from normal. 37

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES The left tail contracts towards the limiting distribution as the sample size increases, as it did for b 2 = 0. 6, but more slowly. 38

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES The right tail actually shifts in the wrong direction as the sample size increases from T = 25 to T = 50. However, it then starts moving back in the direction of the limiting distribution, but there is still a large discrepancy even for T = 200. 39

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES We have seen that, for finite samples, the tails of the distributions of √T(b 2 – b 2) and b 2 differ markedly from their approximate theoretical distributions, even for T = 200, and this can be expected to cause problems for inference. This is indeed the case. 40

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES We know that inference is asymptotically valid in a model with a lagged dependent variable. However, as always, we have to ask how large the sample should be in practice. 41

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES We need to consider the effect on Type I error when the null hypothesis is true, and the effect on Type II error when the null hypothesis is false. 42

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES H 0: b 2 = 0. 9 is true We will start with the effect on Type I error and we again will illustrate the issue with the simple autoregressive model. The figure shows the distribution of the t statistic for H 0: b 2 = 0. 9 when the null hypothesis is true, and T = 100. 43

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES H 0: b 2 = 0. 9 is true The distribution is skewed, reflecting the fact that the distribution of b 2 is skewed. Further complexity is attributable to the fact that the standard error is also valid only asymptotically. 44

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES H 0: b 2 = 0. 9 is true Nominal critical value of t, T = 100, is 1. 98 at 5% level Actual 2. 5% tails start at – 2. 14 and 1. 72 According to the tables, for a 5 percent two-sided test, with T = 100, the critical values of t are 1. 98. However, in reality the lower 2. 5 percent tail of the distribution starts at – 2. 14 and the upper one at 1. 72. 45

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES H 0: b 2 = 0. 9 is true Nominal critical value of t, T = 100, is 1. 98 at 5% level p( t < – 1. 98 ) = 0. 036 p( t > 1. 98 ) = 0. 013 This means that, if one uses the critical values from the table, the risk of a Type I error when the null hypothesis is true is greater than 2. 5 percent when b 2 is negative and less than 2. 5 percent when it is positive. The figures are 3. 6 percent and 1. 3 percent. 46

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES H 0: b 2 = 0. 9 is true The potential effect on Type II error is often of greater practical importance, for typically our null hypothesis is that b 2 = 0 and if the process is truly autoregressive, the null hypothesis is false. 47

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES H 0: b 2 = 0. 9 is true Fortunately, for this null hypothesis, the t test is unlikely to mislead us seriously. If the true value of b 2 is low, the distorting effect of the failure of Assumption C. 7 part (2) can be expected to be minor and our conclusions valid, even for finite samples. 48

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES H 0: b 2 = 0. 9 is true If the true value of b 2 is large, H 0 is likely to be rejected anyway, even though the t statistic does not have its conventional distribution and the nominal critical values of t are incorrect. 49

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES H 0: b 2 = 0. 9 is true These remarks apply to the pure autoregressive model Yt = b 2 Yt– 1 + ut. In practice, the model will include other explanatory variables, with unpredictable consequences. 50

LIMITING DISTRIBUTION OF OLS ESTIMATOR FOR A UNIVARIATE TIME SERIES H 0: b 2 = 0. 9 is true The most that one can say is that, if there is a lagged dependent variable in the model, one should expect point estimates, standard errors, and t statistics to be subject to distortion and therefore one should treat them with caution, especially when the sample is small and when there is evidence that b 2 is large. 51