The Bias of Yule Walker Estimates in AR(2)

Introduction

For AR(2), the Burg estimates may be as

\(TraditionalForm\`\(\(\ \)\(\(ϕ\&-\)\_2 = \ \ \(\((\(∑\+\(t = 1\)\)\&n x\_t\^2)\)\ \((\(∑\+\(t = 1\)\)\&\(n - 2\)\(\(x\_t\)\_\(+2\)\) x\_t)\) - \((\(∑\+\(t = 1\)\)\&\(n - 1\)\(\(x\_t\)\_\(+1\)\) x\_t)\)\^2\)\/\(\((\(∑\+\(t = 1\)\)\&n x\_t\^2)\)\^2 - \((\(∑\+\(t = 1\)\)\&\(n - 1\)\(\(x\_t\)\_\(+1\)\) x\_t)\)\^2\)\)\)\)

Then Yule Walker estimates can be written as

\(TraditionalForm\`\(ϕ\&-\)\_2 = \(x\ y - z\^2\)\/\(\(\(x\)\(\ \)\)\^2 - z\^2\)\)

where \(TraditionalForm\`x = \(\(∑\+\(t = 1\)\)\&n x\_t\^2 = 𝔖\_\(0, 0, 1\)\)\); \(TraditionalForm\`\(y = \(\(∑\+\(t = 1\)\)\&\(n - 2\)\(\(x\_t\)\_\(+2\)\) x\_t = \(\(∑\+\(t = 3\)\)\&n\(\( x\_t\)\_\(-2\)\) 
                  x = 𝔖\_\(2, 0, 3\)\)\);\)\) \(TraditionalForm\`z = \(\(∑\+\(t = 1\)\)\&\(n - 1\)\(\(x\_t\)\_\(+1\)\) 
              x\_t = \(\(∑\+\(t = 2\)\)\&n\( x\_\(t - 1\)\) 
                x\_t = 𝔖\_\(1, 0, 2\)\)\)\);

Mathematica Codes

Autocovariance  of AR(2) with two unequal characteristic  roots,:

R[
          t_] := ((((Subscript[z, 1]) (1 - ((((Subscript[z, 2])))^2)))/(((Subscript[z, 1]) - (Subscript[z, 2])) (1 + (Subscript[z, 1]) (Subscript[z, 2])))) ((((Subscript[z, 1])))^t) - (((Subscript[z, 2]) (1 - ((((Subscript[z, 1])))^2)))/(((Subscript[z, 1]) - (Subscript[z, 2])) (1 + (Subscript[z, 1]) (Subscript[z, 2])))) ((((Subscript[z, 2])))^t)) ((1 + (Subscript[z, 1]) (Subscript[z, 2]))/((1 - (Subscript[z, 1]) (Subscript[z, 2])) (1 - ((((Subscript[z, 1])))^2)) (1 - ((((Subscript[z, 2])))^2)))) // Simplify;

where t ≥  0.  And γ(t) = γ(-t) when t < 0.

LCoV[p_, q_] := 
      Module[{a, b, c, d, e}, 
          a = Sum[R[-h] R[-(h - p + q)] + 
                R[-(h + q)] 
                  R[-(h - 
                          p)], {h, -∞, -q}]; 
b = 
            Sum[R[-h] R[-(h - p + q)] + 
                R[(h + q)] R[-(h - p)], {h, -q + 1, 
                0}]; 
c = 
            Sum[R[h] R[-(h - p + q)] + 
                R[(h + q)] R[-(h - p)], {h, 1, 
                p - q}]; 
d = 
            Sum[R[h] R[(h - p + q)] + 
                R[(h + q)] R[-(h - p)], {h, p - q + 1, 
                p}]; 
e = 
            Sum[R[h] R[(h - p + q)] + R[(h + q)] R[(h - p)], {h, 
                p + 1, ∞}]; 
a + b + c + d + e] // 
        Simplify

Expectation  of (Subscript[𝔖, m, k, i])

Expect[𝔖[m_, k_, i_]] = R[Abs[m - k]]

Large Sample Results of x, y and z

\(TraditionalForm\`lim\ \((n\ \ Bias {𝔖\_\(m, k, i\)})\) = \(-\(\(|\)\(i - 
            1\)\(|\)\(\ \)\(γ(
              p) - \(∑\&∞\)\+\(h = \(-∞\)\)γ(
                h)\)\)\), \ \ \ \ p = \(\(|\)\(m - 
          k\)\(|\)\)\)

Large sample biases of x, y, z, u, v and w : lim  (n Bias ( x)),  
      lim  (n Bias ( y)),  ... ,  
      and lim  (n Bias ( z))

LB[p_, 
          i_] := -(i - 1) R[
                p] - (Sum[R[-t], {t, -∞, 0}] + 
                Sum[R[t], {t, 1, ∞}]) // Simplify;

where \(TraditionalForm\`x = \(\(∑\+\(t = 1\)\)\&n x\_t\^2 = 𝔖\_\(0, 0, 1\)\)\); \(TraditionalForm\`\(y = \(\(∑\+\(t = 1\)\)\&\(n - 2\)\(\(x\_t\)\_\(+2\)\) x\_t = \(\(∑\+\(t = 3\)\)\&n\(\( x\_t\)\_\(-2\)\) 
                  x = 𝔖\_\(2, 0, 3\)\)\);\)\) \(TraditionalForm\`z = \(\(∑\+\(t = 1\)\)\&\(n - 1\)\(\(x\_t\)\_\(+1\)\) 
              x\_t = \(\(∑\+\(t = 2\)\)\&n\( x\_\(t - 1\)\) 
                x\_t = 𝔖\_\(1, 0, 2\)\)\)\);

FirstExp = { LB[0, 1], LB[2, 3], LB[1, 2]} // Simplify

Large sample variances of x, y, z, u, v and w : lim  (n V a r( x)),  
      lim  (n V a r( y)),  ... ,  
      and lim  (n V a r( z))

ExpSquareX = LCoV[0, 0]

ExpSquareY =  LCoV[2, 2]

ExpSquareZ = LCoV[1, 1]

ExpSquare = {ExpSquareX, ExpSquareY, ExpSquareZ} // Simplify

The large sample covariances  of x,  y,   ... ,  z: lim  (n C o v( x y)),  
      lim  (n C o v (x z)),  ...

ExpCrossX = {LCoV[2, 0], LCoV[1, 0]}

ExpCrossY = LCoV[2, 1]

The expectations  of x,  y,  and z:

(Subscript[x, 0]) = Expect[𝔖[0, 0, 1]]; 
    (Subscript[z, 0]) = Expect[𝔖[1, 0, 2]]; 
    (Subscript[y, 0]) = Expect[𝔖[2, 0, 3]];

Bias of Yule Walker Estimate of (Subscript[ϕ, 2]) in AR(2)

The Yule Walker estimate of (Subscript[ϕ, 2]) is

CoeAR2[x_, y_, z_]:=((x y - (z^2))/(((x )^2) - (z^2)))

The vectors of the first derivatives of CoeAR2[x, y, z] with the respective to x, y and z:

FirstTerm = (D[CoeAR2[x, y, z], #] & /@ {x, y, 
                z}) /. {x -> (Subscript[x, 0]), y ->  (Subscript[y, 0]), 
            z ->  (Subscript[z, 0])} // Simplify

FirstTerm . FirstExp // Simplify

The vectors of the second derivatives of CoeAR2[x, y, z] with the respective to x, y, w and z:

SqTerm = (D[CoeAR2[x, y, z], #, #] & /@ {x, y, 
                z}) /. {x -> (Subscript[x, 0]), y ->  (Subscript[y, 0]), 
            z ->  (Subscript[z, 0])} // Simplify

a = {x, y, z}

CrossTermX = (Table[
              D[CoeAR2[x, y, z], a[[1]], 
                a[[j]]],  {j, 2, 
                3}]) /. {x -> (Subscript[x, 0]), y ->  (Subscript[y, 0]), 
            z ->  (Subscript[z, 0])} // Simplify

CrossTermY = 
      D[CoeAR2[x, y, z], a[[2]], 
            a[[3]]] /. {x -> (Subscript[x, 0]), 
            y ->  (Subscript[y, 0]), z ->  (Subscript[z, 0])} // Simplify

The large sample bias of the Burg estimate of (Subscript[ϕ, 2]), (lim_(n ->  ∞))n(
          E((Subscript[(Overscript[ϕ, –]), 2]) - (Subscript[ϕ, 2]))), is

LSBiasϕ2 = 
      FirstTerm . FirstExp + (1/2) SqTerm . ExpSquare + 
          CrossTermX . ExpCrossX + (CrossTermY)*( ExpCrossY) // 
        FullSimplify

Expand[
      2 (-1 + (Subscript[z, 2]^2) + (Subscript[z, 1]^2) (1 + (Subscript[z, 2]^2)) + 
            (Subscript[z, 1]^3) (Subscript[z, 2]) (-2 + 3 (Subscript[z, 2]^2)) + 
            (Subscript[z, 1]) (3 (Subscript[z, 2]) - 2 (Subscript[z, 2]^3)))]

((2  (Subscript[z, 1]) (Subscript[z, 2]) (((1 + (Subscript[z, 1]) (Subscript[z, 2])))^2))/((1 - ((((Subscript[z, 1])))^2)) (1 - ((((Subscript[z, 2])))^2)))) + (4  (Subscript[z, 1]) (Subscript[z, 2]) - 
          2)

In (3.7) of T & P, the bias is

((2  (Subscript[z, 1]) (Subscript[z, 2]) (((1 + (Subscript[z, 1]) (Subscript[z, 2])))^2))/((1 - ((((Subscript[z, 1])))^2)) (1 - ((((Subscript[z, 2])))^2)))) + (4  (Subscript[z, 1]) (Subscript[z, 2]) - 
          2)

Expand[
      2  (Subscript[z, 1]) (Subscript[z, 2]) (((1 + (Subscript[z, 1]) (Subscript[z, 2])))^2) + (4  (Subscript[z, 1]) (Subscript[z, 2]) - 
              2) (1 - ((((Subscript[z, 1])))^2)) (1 - ((((Subscript[z, 2])))^2))]