Forecasting the AR(1) Time Series Model

Intercept form of the model

The AR(1) model can be written in intercept form,

\[z_t = \beta_0 + \beta_1 z_{t-1} + a_t\],

where \(t=1,\ldots, T\), \(\beta_0\) is the intercept coefficient, \(\beta_1\) is the slope coefficient and \(a_t\) is assumed to be \({\rm NID}(0, \sigma_a^2)\). This model violates the standard regression assumptions that the input variables are not stochastic. Nevertheless standard regression least squares provides an efficient estimates \(\hat{\beta}_0\) and \(\hat{\beta}_1\) for the parameters. Recall the for bivariate data \((x_i, y_i), i=1,\ldots,n\) the least squares estimates are given by

\[\hat{\beta}_1 = \frac{\sum_{i=1} (x_{i}-\bar{x})(y_{i}-\bar{y})} {\sqrt{\sum_{i=1}^n (x_{i}-\bar{x}) \sum_{i=1}^n (y_{i}-\bar{y})}}.\]

In the AR(1) model we may set \(y_{t-1} = z_t, t=2, \ldots, T\), \(x_{t} = z_t, t=1, \ldots, T-1\) and \(n = T-1\) and plug-in the above formula to obtain an efficient estimate of \(\beta_1\).

Another frequently used efficient estimate is the sample lag-one autocorrelation coefficient,

\[r_1 = \frac{\sum_{t=2} (z_t-\bar{z})(z_{t-1} - \bar{z})} {\sum_{t=1} (z_t-\bar{z})^2},\]

where \(\bar{z}\) is the sample mean of \(z_1, \ldots, z_T\).

It may be shown that \(r_1\) is numerically approximately equal to the least squares estimate and that the difference is in most cases negligible provided that \(T\) is not too small. In general it is recommended that \(T\ge 50\) and preferably that \(T \ge 200\).

Theoretical variance of AR(1) process

From the model equation \(z_t = \beta_0 + \beta_1 z_{t-1} + a_t\),

\[{\rm Var}(z_t) = {\rm Var}(\beta_1^2 z_{t-1} + a_t) \\ =\beta_1^2 {\rm Var}(z_{t-1}) + \sigma_a^2.\]

Since \(z_{t-1}\) is by definition uncorrelated with \(a_t\), the innovation at time \(t\) and by stationarity \({\rm Var}(z_t) = {\rm Var}(z_{t-1}) = \sigma_z^2\). Hence,

\[\sigma_z^2 = \frac{\sigma_a^2}{1-\beta_1^2}\].

Since \(-1 < \beta_1 < 1\) in order for the AR(1) process to satisfy the stationarity requirement, we see that \(\sigma_z^2 \gt \sigma_a^2\) provided that \(\beta_1 \ne 0\).

Chain rule for forecasting

Consider the time series $z_t, t=1,2,$ that is assumed to be generated from an AR(1) process with parameters \(\beta_0\) and \(\beta_1\). Given data up to time \(T\), the optimal forecast at lead time \(\ell\) is the conditional expected value of \(z_{t+\ell}\) given \(z_T, z_{T-1}, \ldots\). Hence we may write,

\[z_T(\ell) = {\rm E}(z_{T+\ell} | z_T, z_{T-1}, \ldots) \\ ={\rm E}_T(z_{T+\ell}),\]

where \({\rm E}_T(\bullet)\) denotes conditional expected value at time \(t=T\).

From the model equation,

\[z_T(\ell) = {\rm E_T}(z_{T+1}) = \beta_0 + \beta_1 z_T.\]

More generally the chain rule for forecast computation may be written, for any \(\ell > 0\),

\[z_T(\ell) = \beta_0 + \beta_1 z_T(\ell-1),\]

where \(z_T(0) = z_T\).

Centered form of the AR(1) model

It is sometimes more convenient for mathematical derivations to work with the centered model,

\[z_t = \mu + \phi (z_{t-1} - \mu) + a_t,\]

where \(\mu\) is the mean of the time series and \(\phi\) is the autoregressive coefficient. Setting \(\beta_1 = \phi\) and \(\beta_0 = \mu (1-\phi)\) demonstrates the equivalence to the intercept form of this model.

Let \(w_t = z_t - \mu\). Then we may write,

\[w_{T+\ell} = a_{T+\ell} + \phi w_{T+\ell-1} \\ = a_{T+\ell} + \phi a_{T+\ell-1} + \phi^2 w_{T+\ell-2} \\ = a_{T+\ell} + \phi a_{T+\ell-1} + \phi^2 a_{T+\ell-2} + \phi^3 w_{T+\ell-3} \\ \dots =a_{T+\ell} + \phi a_{T+\ell-1} + \ldots \phi^{\ell-1} a_{T+1} + \phi^{\ell} w_{T}.\]

Taking conditional expectations, the optimal forecast may be written,

\[w_T(\ell) = \phi^{\ell} w_T\]

or equivalently,

\[z_T(\ell) = \mu + \phi^{\ell} (z_T-\mu).\]

The forecast error is given by,

\[a_T(\ell) = a_{T+\ell} + \phi a_{T+\ell-1} + \ldots \phi^{\ell-1} a_{T+1}.\]

Hence the variance of the \(\ell\)-step forecast is

\[V(\ell) = \sigma_a^2 \frac{1-\phi^{2 \ell}}{1-\phi^2}.\]

Note that as \(\ell \rightarrow \infty\), \(V(\ell) \rightarrow \sigma^2_z\) and \(z_T(\ell) \rightarrow \mu\).

Simulated AR(1), \(\phi=0.8\) with parameters assumed known

Simulated AR(1), \(\phi=0.8\) with parameters estimated

Forecast Comparison for Inflation Bond Yields

Diagnostic Checking Time Series Models

As in regression analysis it is important to check the validity of the model assumptions. In the case of the AR(1) model the most important assumption is that the residuals approximate white noise.

In R, the function arima() may be use to fit a wide range of time series models and then tsdiag() provides standard diagnostic checks.

library(tidyverse)
fileName <- 
 "http://fisher.stats.uwo.ca/faculty/aim/2018/3859A/data/InflationBond.csv"
tsdata <- read_csv(fileName)%>%
  mutate(date=lubridate::dmy(date))
#arima fitting
z <- tsdata$INFBOND
ans <- arima(z, order=c(1,0,0))
ans
## 
## Call:
## arima(x = z, order = c(1, 0, 0))
## 
## Coefficients:
##          ar1  intercept
##       0.8538     2.2896
## s.e.  0.0727     0.1210
## 
## sigma^2 estimated as 0.01902:  log likelihood = 28.02,  aic = -50.05
tsdiag(ans)

In the top chart the Standardized Residuals are defined as the residuals divided by their estimated standard deviation. Frees (textbook, p.) suggests plotting the residuals and their three-sigma limits, so in the above chart using the standardized residuals we see there are no residuals exceeding these limits.

Forecast comparison for Labor Force Participation Rate Series

The mean and standard deviation of the differenced time series was found to be 0.0120949 and 0.0100669. Hence the parameter estimates for the random walk model with deterministic drift are \(\hat{\delta} =\) 0.012 and \(\hat{\sigma}_a =\) 0.01. The last observed value of LFRP for 1998 was 0.6407296 so the forecast function may be written, \(z_{1998}(\ell) = 0.6407 + 0.121 \times \ell\) with variance function \(V_\ell = 0.01 \times \ell\).

Comparison LS and Random Walk

Comparison LS and Random Walk