REPARAMETERIZATION OF THE BOUNDARY OF THE ADMISSIBLE REGION FOR ARMA MODELS

A.I. McLeod and Y. Zhang The University of Western Ontario

October 23, 2003

Mailing Address for Corresponding Author:

Dr. A.I. McLeod Department of Statistical and Actuarial Sciences The University of Western Ontario London, Ontario N6A 5B7 Canada

A.I. McLeod Email & Phone: aimcleod@uwo.ca 519 661-3611 Y. Zhang Email & Phone: yzhangli@stats.uwo.ca 519 661-3821

Running Head: Reparameterization of the Boundary

The theorem of Barndorff-Neilsen and Schou (1973, Theorem 2) stating that the admissible region of autoregressive process may be obtained as the image of the admissible region defined by the partial autocorrelations under the Durbin-Levinson recursion is derived from first principles in the case of the boundary. Barndorff-Neilsen and Schou (1973, Theorem 2) is also extended to the case of the subset autoregressive model. The boundary transformation is useful for obtaining maximum likelihood estimates on the moving-average boundary as well as for constructing diagrams of the admissible region. The subset result suggests a new approach to subset autoregression which has been extensively developed in another article.

Keywords and phrases: Admissible region for the autoregressive and movingaverage time series; Maximum likelihood estimation on the moving-average boundary; Subset autoregression model.

1. INTRODUCTION

The AR (p) model with mean zero may be written, ϕ(B)z_{t} = a_{t}, where ϕ(B) = 1 − ϕ_{1}B − … ϕ_{p}B^{p} and a_{t} is Gaussian white noise with variance σ_{a}^{2}. The admissible region for stationary-causal processes is defined by the region in ℜ^{p} for which all roots of ϕ(B) = 0 lie outside the unit circle (Brockwell and Davis, 1991).

Let ϕ_{j,k} denote the coefficient of z_{t−j} in the minimum-mean-square-error predictor of z_{t} given z_{t−1}, …, z_{t−k}. Then the partial autocorrelations, denoted by ζ_{i} for i = 1, …, p are given by ζ_{i} = ϕ_{i,i}, where the ϕ_{i,i} are determined by the Durbin-Levinson recursion,

ϕ_{j,k+1} = ϕ_{j,k} − ϕ_{k+1,k+1}ϕ_{k+1−j,k}, j = 1, …, k  (1)

where k = 1, …, p. Barndorff-Neilsen and Schou (1973) showed that eq. (1) can be used to define a transformation, B : (ζ_{1}, …, ζ_{p}) ⇒ (ϕ_{1}, …, ϕ_{p}), and that this transformation is continuously differentiable and has a continuously differentiable inverse inside the admissible region. It will be convenient in §3, to use the notation ϕ_{i} = B_{i}(ζ) to refer to each parameter i = 1, …, p, where ζ = (ζ_{1}, …, ζ_{p}). It should be noted that since the determinant Jacobian of the transformation (Barndorff-Neilsen and Schou, 1973, p.414) is zero on the boundary, the transformation is not 1:1 there. For example, in the AR (2) case B(ζ_{1}, ζ_{2}) = (ζ_{1}(1 − ζ_{2}), ζ_{2}) so all the points on the boundary of the form (ζ_{1}, 1) are mapped into the point (0, 1).

2. Boundary of AR (p) admissible region

Let D_{ϕ} denote the admissible region for an AR (p) in the parameter space (ϕ_{1}, …, ϕ_{p}). The admissible region for the transformed parameters (ζ_{1}, …, ζ_{p}) is the interior of the unit cube,

D_{ζ} = {(ζ_{1}, …, ζ_{p}) ∈ ℜ^{p} : |ζ_{i}| < 1, i = 1, …, p}.  (2)

Denote the boundary sets of D_{ϕ} and D_{ζ} by ∂_{ϕ} and ∂_{ζ} respectively. Barndorff-Nielsen and Schou (1973, Theorem 2) showed that B maps D_{ζ} onto D_{ϕ} and that this mapping is one-to-one. But B is no longer one-to-one on the boundary ∂_{ζ}. It is non-trivial to show that B maps ∂_{ζ} onto ∂_{ϕ}. For completeness we have included an elementary proof using real analysis of this result.

Theorem 1. ∂_{ϕ} = B(∂_{ζ})

Proof. First we show that ∂_{ϕ} ⊂ B(∂_{ζ}). Let D̄_{ϕ} and D̄_{ζ} denote the closures of D_{ϕ} and D_{ζ} respectively. Since B is a polynomial and hence continuous on D̄_{ζ}, B(D̄_{ζ}) ⊂ \overline{B(D_{ζ})} = D̄_{ϕ}. Meanwhile, B(D̄_{ζ}) = D_{ϕ} ∪ B(∂_{ζ}) is closed since D̄_{ζ} is compact and therefore B(D̄_{ζ}) ⊃ D̄_{ϕ}. Hence B(D̄_{ζ}) = D̄_{ϕ}, i.e., D_{ϕ} ∪ B(∂_{ζ}) = D_{ϕ} ∪ ∂_{ϕ}. Since B^{−1} is continuous, it follows that D_{ϕ} is open since D_{ζ} is open. D_{ϕ} ∩ ∂_{ϕ} = ∅. Hence ∂_{ϕ} ⊂ B(∂_{ζ}).

Next, we show ∂_{ϕ} ⊃ B(∂_{ζ}). Let ϑ ∈ ∂_{ζ}. There exists a sequence {ϑ_{n}} ∈ D_{ζ} such that ϑ_{n} → ϑ. Hence B(ϑ_{n}) = φ_{n} ∈ D_{ϕ} → B(ϑ) = φ by the continuity of B on D̄_{ζ}. If φ ∈ D_{ϕ}, B^{−1}(φ_{n}) = ϑ_{n} → B^{−1}(φ) = ϑ ∈ D_{ζ} by the continuity of B^{−1} on D_{ϕ}, which causes a contradiction with ϑ ∈ ∂_{ζ}. Therefore, B(ϑ) ∉ D_{ϕ}. It follows that B(ϑ) ∈ ∂_{ϕ} since D_{ϕ} ∪ B(∂_{ζ}) = D_{ϕ} ∪ ∂_{ϕ}. Hence ∂_{ϕ} ⊃ B(∂_{ζ}). □

For the AR (4) model, z_{t} = ϕ_{1}z_{t−1} + ϕ_{2}z_{t−2} + ϕ_{3}z_{t−3} + ϕ_{4}z_{t−4}. In this case the admissible region is 4-dimensional and we have,

ϕ_{1} = ζ_{1} − ζ_{1} ζ_{2} − ζ_{2} ζ_{3} − ζ_{3} ζ_{4},
ϕ_{2} = ζ_{2} − ζ_{1} ζ_{3} + ζ_{1} ζ_{2} ζ_{3} − ζ_{2} ζ_{4} + ζ_{1} ζ_{3} ζ_{4} − ζ_{1} ζ_{2} ζ_{3} ζ_{4},
ϕ_{3} = ζ_{3} − ζ_{1} ζ_{4} + ζ_{1} ζ_{2} ζ_{4} + ζ_{2} ζ_{3} ζ_{4}
ϕ_{4} = ζ_{4}.  (3)

Three dimensional projections of this admissible region may be visualized using standard graphics software such as MatLab or Mathematica (Zhang, 2002).

Another application of Theorem 1 is to maximizing the likelihood function for moving-average models over the region formed by the admissible region and its boundary. As noted by Monahan (1984) the eq. (1) may be used to reparameterize ARMA models as well by applying the transformation separately to the autoregressive and moving-average components. Similarly Theorem 1 shows that we may re-parameterize for the parameters on movingaverage boundary of the invertible region. It is well-known that there is a positive probability that the moving-average parameter may lie on the noninvertible boundary (Cryer and Ledholter, 1981) in the case of the MA(1) model. Moreover, Davis and Dunsmuir (1996) showed that the asymptotic distribution for maximum likelihood estimate in the closed region which included the boundary differed from that of the estimate restricted to be inside the admissible region.

We conducted a simulation experiment to see how frequently unit roots occur in the MA(2) case with exact maximum likelihood estimator defined over the closed admissible region obtained by taking the closure of the invertible region. Reparameterizing the MA(2) model, z_{t} = a_{t} −θ_{1}a_{t−1} −θ_{2}a_{t−2} using (θ_{1}, θ_{2}) ⟷ (ζ_{1}, ζ_{2}) and with series of lengths n = 15, 30, 50, 100, 200 and with parameters (ζ_{1}, ζ_{2}) chosen at random and uniformly distributed inside the unit square, 10^{3} simulations of each series were generated and fit by an exact maximum likelihood algorithm modified for the closed admissible region, D̄_{ζ} = {|ζ_{1}| ≤ 1 and |ζ_{2}| ≤ 1 }.

For computational purposes the maximum in each of the nine subregions is determined: D_{ζ} = {|ζ_{1}| < 1 and |ζ_{2}| < 1 }, ∂_{ζ,1} = {ζ_{1} = 1 and |ζ_{2}| < 1 }, ∂_{ζ,2} = {ζ_{1} = −1 and |ζ_{2}| < 1 }, ∂_{ζ,3} = {|ζ_{1}| < 1 and ζ_{2} = 1 }, ∂_{ζ,4} = {|ζ_{1}| < 1 and ζ_{2} = −1 }, ∂_{ζ,5} = {ζ_{1} = 1 and ζ_{2} = 1 }, ∂_{ζ,6} = {ζ_{1} = 1 and ζ_{2} = −1 }, ∂_{ζ,7} = {ζ_{1} = −1 and ζ_{2} = 1 }, and ∂_{ζ,8} = {ζ_{1} = −1 and ζ_{2} = −1 }. Then the overall maximum is selected to obtain the global maximum likelihood estimate. As shown in Table 1, the probability of a unit root is quite high when n is small but decreases as n increases.

[ Table 1 here ]

2. EXTENSION TO SUBSET MODELS

Subset autoregressive provide a parsimonious alternative to the full AR (p) model which is especially useful in modelling seasonal or periodic time series. Consider the subset AR (4) model, z_{t} = ϕ_{1}z_{t−1} + ϕ_{4}z_{t−4}. Then the transformation,

B^{−1}(ϕ_{1}, 0, 0, ϕ_{4}) = (ζ_{1}, ζ_{2}, ζ_{3}, ζ_{4})  (4)

is one-to-one, continuous and differentiable in the admissible region. However the admissible region in the four-dimensional space D_{ζ} is no longer simply the interior unit 4D-cube. Instead it is a complicated two-dimensional subspace of the 4D-cube. Setting the ϕ_{2} = 0 and ϕ_{3} = 0 in the second and third equations in (3) these equations may be solved for ζ_{2} and ζ_{3} to determine B. Since ζ_{1} ≠ 0, ζ_{4} ≠ 0, |ζ_{1}| < 1 and ζ_{4} > −1,

ζ_{2} = \frac{1 + 2 ζ_{1}^{2} ζ_{4} − \sqrt{1 + 4 ζ_{1}^{2} ζ_{4} (1 + ζ_{4})}}{2 (−1 + ζ_{1}^{2}) ζ_{4}}
ζ_{3} = \frac{−1 + \sqrt{1 + 4 ζ_{1}^{2} ζ_{4} (1 + ζ_{4})}}{2 ζ_{1} (1 + ζ_{4})}.  (5)

Note that not all solutions of (5) are admissible. For example taking ζ_{1} = 0.9 and ζ_{4} = −0.9, (5) produces, ζ_{2} ≐ −0.88 and ζ_{3} ≐ −3.8. Since (4) is also a one-to-one, continuous and differentiable transformation, these equations establish that the transformation (ϕ_{1}, ϕ_{4}) → (ζ_{1}, ζ_{4}) is one-to-one, continuous and differentiable with the appropriate admissible regions. The admissible region for ζ_{1} and ζ_{4} is given by,

D_{ζ} = {(ζ_{1}, ζ_{4}) ∈ (−1, 1) × (−1, 1) : |ζ_{2}| < 1 ∧ |ζ_{3}| < 1}  (6)

The admissible region for ζ_{1} and ζ_{4} is shown in Figure 1 below. Applying the transformation B : (ζ_{1}, ζ_{2}, ζ_{3}, ζ_{4}) → (ϕ_{1}, 0, 0, ϕ_{4}), the admissible region, Figure 2, for ϕ_{1} and ϕ_{4} is obtained.

[ Figures 1 and 2 ]

In the general case of a subset AR (p) model, let

z_{t} = a_{t} + ϕ_{i_{1}}z_{t−i_{1}} + … + ϕ_{i_{m}}z_{t−i_{m}}  (7)

where i_{1} < … < i_{m}. Let Ḃ_{i_{1},...,i_{m}}^{−1} denote the transformation,

Ḃ_{i_{1},...,i_{m}}^{−1} : (ϕ_{i_{1}}, …, ϕ_{i_{m}}) → (ζ_{i_{1}}, …, ζ_{i_{m}})  (8)

obtained by selecting the i_{1}, …, i_{m} elements from,

B^{−1} : (ϕ_{1}, …, ϕ_{p}) → (ζ_{1}, …, ζ_{p}).  (9)

Theorem 2. The transformation and its inverse defined in eqn. (8) is one-to-one, continuous and onto in the admissible region.

Proof. The transformation given in eqn. (1) may be written,

ϕ_{i} = ζ_{i} + b_{i}(ζ_{1}, …, ζ_{p}),  (10)

where b_{i}(ζ_{1}, …, ζ_{p}) is a multilinear function of the variables ζ_{1}, …, ζ_{p} with the coefficient of each term being 1 or −1. This holds for p = 1. Suppose it is true also for p = k. Then eqn. (1) ensures that the new values of ϕ_{1}, …, ϕ_{k+1} are suitable multilinear functions of the variables ζ_{1}, …, ζ_{k+1}.

Consider a subset AR (p) with only one parameter ϕ_{h} = 0 constrained, where h is a fixed value 0 < h ≤ p. Using (10), an explicit solution of the equation ϕ_{h} = 0 may be found,

ζ_{h} = g_{h}(ζ^{(−h)}),  (11)

where ζ^{(−h)} denotes the vector (ζ_{1}, …, ζ_{p}) with the h-th element removed. We can subsitute for ζ_{h} in each of the p − 1 equations ϕ_{i} = B_{i}(ζ_{1}, …, ζ_{p}) to obtain an explicit solution ϕ_{i} = B_{i,h}(ζ^{(−h)}) which defines the required transformation.

In the more general case, there are p − m constraints. Denote the indices of those parameters ϕ_{1}, … ϕ_{p} which are constrained to zero by j_{1}, …, j_{p−m}. The p − m constraint equations in p − m unknowns may be written 0 = ζ_{j_{k}} + b_{j_{k}}(ζ_{1}, …, ζ_{p}), k = 1, …, (p − m). Starting with the equation corresponding to ϕ_{j_{1}} = 0, the solution for ζ_{j_{1}} can be found and substituted in the remaining p − 1 equations. The next equation, ϕ_{j_{2}} = 0, is by the multilinear property, a quadratic with two solutions for ζ_{j_{2}}. Continuing this process, there are (p − m)! solutions to the constraint equations. The theorem of Barndorff-Neilsen and Schou (1973) guarantees that at most only one of these solutions is admissible. □

Denote the admissible boundaries for the parameters (ϕ_{i,1}, …, ϕ_{i,m}) and (ζ_{i,1}, …, ζ_{i,m}) by ∂_{ϕ_{i,1},...,ϕ_{i,m}} and ∂_{ζ_{i,1},...,ζ_{i,m}} respectively.

Theorem 3. ∂_{ϕ_{i,1},...,ϕ_{i,m}} = Ḃ_{i_{1},...,i_{m}}(∂_{ζ_{i,1},...,ζ_{i,m}})

Proof. Since Ḃ_{i_{1},...,i_{m}} is a one-to-one, continuous and differentiable transformation, Theorem 3 follows making the necessary changes to the argument given by Theorem 1. □

Theorem 2 indicates that the parameter space of the subset autoregression is spanned by the corresponding partial autocorrelations. This suggests an alternative parameterization for the subset autoregessive model in which some partial autocorrelations are set to zero and the others are estimated. This new approach to subset autoregression offers many advantages and is developed in McLeod and Zhang (2003, in press).

ACKNOWLEDGEMENTS

The authors would like to thank Dr. André Boivin for his suggestions and comments on Theorem 1.

REFERENCES Barndorff-Nielsen, O. and Schou G. (1973). On the Parametrization of Autoregressive Models by Partial Autocorrelations Journal of Multivariate Analysis 3, 408-419. Brockwell, P.J. and Davis, R.A. (1991). Time Series: Theory and Methods. (2nd edn.). New York: Springer-Verlag. Cryer, J. D. and Ledolter, J. (1981). Small-sample properties of the maximum likelihood estimator in the first-order moving average model. Biometrika 68, 691-694. Davis, R.A. and Dunsmuir, W.T.M. (1996). Maximum likelihood estimation for MA(1) processes with a root on or near the unit circle. Econometric Theory 12, 1–29. McLeod, A.I. and Zhang, Ying (2003). Subset Autoregression: A New Approach. Submitted for publication. Monahan, J.F. (1984). A note on enforcing stationarity in autoregressivemoving average models, Biometrika 71, 403-404. Zhang, Y. (2002). Topics in Autoregression. Ph.D. Thesis, The University of Western Ontario.

Table 1. Estimated probability, π̂, of the maximum likelihood estimator of the parameters in a series of length n being on the boundary for the MA(2) model with true parameters randomly chosen within the invertible region.

n = 15 n = 30 n = 50 n = 100 n = 200
π 0.592 0.425 0.319  0.207  0.139
Est.Sd(π) 0.016 0.016 0.015  0.013  0.011

Figure 1: Admissible region, z_{t} = ϕ_{1}z_{t−1} + ϕ_{4}z_{t−4} for reparameterized model with ζ_{1} and ζ_{4}.

Figure 2: Admissible region, z_{t} = ϕ_{1}z_{t−1} + ϕ_{4}z_{t−4}.

14