The Bias of Least Squares Estimates in AR(2)

Introduction

For AR(2), the LS estimates may be as

\(TraditionalForm\`\(\(\ \)\(\(ϕ\&-\)\_2 = \ \ \(\((\(∑\+\(t = 2\)\)\&\(n - 1\)x\_t\^2)\)\ \((\(∑\+\(t = 1\)\)\&\(n - 2\)\(\(x\_t\)\_\(+2\)\) x\_t)\) - \((\(∑\+\(t = 1\)\)\&\(n - 2\)\(\(x\_t\)\_\(+1\)\) x\_t)\) \((\(∑\+\(t = 2\)\)\&\(n - 1\)\(x\_t\) x\_\(t + 1\))\)\)\/\(\((\(∑\+\(t = 2\)\)\&\(n - 1\)x\_t\^2)\) \((\(∑\+\(t = 1\)\)\&\(n - 2\)x\_t\^2)\) - \((\(∑\+\(t = 1\)\)\&\(n - 2\)\(\(x\_t\)\_\(+1\)\) x\_t)\)\^2\)\)\)\)

Then it can be written as

\(TraditionalForm\`\(ϕ\&-\)\_2 = \(x\ y - z\ w\)\/\(x\ u - z\^2\)\)

where \(TraditionalForm\`x = \(\(∑\+\(t = 2\)\)\&\(n - 1\)x\_t\^2 = \(\(∑\+\(t = 3\)\)\&n x\_\(t - 1\)\^2 = 𝔖\_\(1, 1, 3\)\)\)\); \(TraditionalForm\`\(y = \(\(∑\+\(t = 1\)\)\&\(n - 2\)\(\(x\_t\)\_\(+2\)\) x\_t = \(\(∑\+\(t = 3\)\)\&n\(\( x\_t\)\_\(-2\)\) 
                  x\_t = 𝔖\_\(2, 0, 3\)\)\);\)\) \(TraditionalForm\`w = \(\(∑\+\(t = 2\)\)\&\(n - 1\)\(x\_t\) 
              x\_\(t + 1\) = 𝔖\_\(1, 0, 3\)\); \ 
      u = \(\(∑\+\(t = 1\)\)\&\(n - 2\)x\_t\^2 = 𝔖\_\(2, 2, 3\)\); 
      z = \(\(∑\+\(t = 1\)\)\&\(n - 2\)\(\(x\_t\)\_\(+1\)\) 
              x\_t = \(\(∑\+\(t = 2\)\)\&n\( x\_\(t - 1\)\) 
                x\_t = 𝔖\_\(1, 2, 3\)\)\);\)

Mathematica Codes

Autocovariance  of AR(2) with two unequal characteristic  roots,:

R[
          t_] := ((((Subscript[z, 1]) (1 - ((((Subscript[z, 2])))^2)))/(((Subscript[z, 1]) - (Subscript[z, 2])) (1 + (Subscript[z, 1]) (Subscript[z, 2])))) ((((Subscript[z, 1])))^t) - (((Subscript[z, 2]) (1 - ((((Subscript[z, 1])))^2)))/(((Subscript[z, 1]) - (Subscript[z, 2])) (1 + (Subscript[z, 1]) (Subscript[z, 2])))) ((((Subscript[z, 2])))^t)) ((1 + (Subscript[z, 1]) (Subscript[z, 2]))/((1 - (Subscript[z, 1]) (Subscript[z, 2])) (1 - ((((Subscript[z, 1])))^2)) (1 - ((((Subscript[z, 2])))^2)))) // Simplify;

where t ≥  0.  And γ(t) = γ(-t) when t < 0.

The large  sampel covariances  of those  (Subscript[𝔖, m, k, i]) for known mean case

(lim_(n ->  ∞))n Cov ((Subscript[𝔖, m, k, i]), (Subscript[𝔖, f,  g,  j]) )=(Underscript[((∑^∞)), h = -∞])[γ(
                  h) γ(h - p + q) + γ(
                  h + q) γ(h - p)]

where p = |m - k| and q = |f - g|.  When p ≥  q ≥  0, those large sample covariances may be obtained by Mathematica function, LCoV[p_,q_], as follows:

LCoV[p_, q_] := 
      Module[{a, b, c, d, e}, 
          a = Sum[R[-h] R[-(h - p + q)] + 
                R[-(h + q)] 
                  R[-(h - 
                          p)], {h, -∞, -q}]; 
b = 
            Sum[R[-h] R[-(h - p + q)] + 
                R[(h + q)] R[-(h - p)], {h, -q + 1, 
                0}]; 
c = 
            Sum[R[h] R[-(h - p + q)] + 
                R[(h + q)] R[-(h - p)], {h, 1, 
                p - q}]; 
d = 
            Sum[R[h] R[(h - p + q)] + 
                R[(h + q)] R[-(h - p)], {h, p - q + 1, 
                p}]; 
e = 
            Sum[R[h] R[(h - p + q)] + R[(h + q)] R[(h - p)], {h, 
                p + 1, ∞}]; 
a + b + c + d + e] // 
        Simplify

Expectation  of (Subscript[𝔖, m, k, i])

Expect[𝔖[m_, k_, i_]] = R[Abs[m - k]]

Large Sample bias of x, y, w, u and z for unknown case

\(TraditionalForm\`lim\ \((n\ \ Bias {𝔖\_\(m, k, i\)})\) = \(-\(\(|\)\(i - 
            1\)\(|\)\(\ \)\(γ(
              p) - \(∑\&∞\)\+\(h = \(-∞\)\)γ(
                h)\)\)\), \ \ \ \ p = \(\(|\)\(m - 
          k\)\(|\)\)\)

Large sample biases of x, y, w, u and z : lim  (n Bias ( x)),  
      lim  (n Bias ( y)),  ... ,  
      and lim  (n Bias ( w))

LB[p_, 
          i_] := -(i - 1) R[
                p] - (Sum[R[-t], {t, -∞, 0}] + 
                Sum[R[t], {t, 1, ∞}]) // Simplify;

where \(TraditionalForm\`x = \(\(∑\+\(t = 2\)\)\&\(n - 1\)x\_t\^2 = \(\(∑\+\(t = 3\)\)\&n x\_\(t - 1\)\^2 = 𝔖\_\(1, 1, 3\)\)\)\); \(TraditionalForm\`\(y = \(\(∑\+\(t = 1\)\)\&\(n - 2\)\(\(x\_t\)\_\(+2\)\) x\_t = \(\(∑\+\(t = 3\)\)\&n\(\( x\_t\)\_\(-2\)\) 
                  x\_t = 𝔖\_\(2, 0, 3\)\)\);\)\) \(TraditionalForm\`w = \(\(∑\+\(t = 2\)\)\&\(n - 1\)\(x\_t\) 
              x\_\(t + 1\) = 𝔖\_\(1, 0, 3\)\); \ 
      u = \(\(∑\+\(t = 1\)\)\&\(n - 2\)x\_t\^2 = 𝔖\_\(2, 2, 3\)\); 
      z = \(\(∑\+\(t = 1\)\)\&\(n - 2\)\(\(x\_t\)\_\(+1\)\) 
              x\_t = \(\(∑\+\(t = 3\)\)\&n\( x\_\(t - 2\)\) 
                x\_\(t - 1\) = 𝔖\_\(2, 1, 3\)\)\);\)

Compute  {lim n Bias[X], lim n Bias[Y], lim n Bias[W], 
        lim n Bias[U], lim n Bias[Z]}

FirstExp = { LB[0, 3], LB[2, 3], LB[1, 3], LB[0, 3], LB[1, 3]} // 
        FullSimplify

Large sample variances of x, y, w, u and z : lim  (n V a r( x)),  
      lim  (n V a r( y)),  ... ,  
      and lim  (n V a r( z))

ExpSquareX = LCoV[0, 0]

ExpSquareY =  LCoV[2, 2]

ExpSquareW = LCoV[1, 1]

ExpSquareU = LCoV[0, 0]

ExpSquareZ = LCoV[1, 1]

ExpSquare = {ExpSquareX, ExpSquareY, ExpSquareW, ExpSquareU,  
          ExpSquareZ} // Simplify

The large sample covariances  of x,  y,   ... ,  z: lim  (n C o v( x y)),  
      lim  (n C o v (x z)),  ... ,  
      and lim  (n C o v (u z))

ExpCrossX = {LCoV[2, 0], LCoV[1, 0], LCoV[0, 0], LCoV[1, 0]}

ExpCrossY = {LCoV[2, 1], LCoV[2, 0],  LCoV[2, 1]}

ExpCrossW = {LCoV[1, 0], LCoV[1, 1]}

ExpCrossU = LCoV[1, 0]

The expectations  of x,  y,  ...  and z:

where \(TraditionalForm\`x = \(\(∑\+\(t = 2\)\)\&\(n - 1\)x\_t\^2 = \(\(∑\+\(t = 3\)\)\&n x\_\(t - 1\)\^2 = 𝔖\_\(1, 1, 3\)\)\)\); \(TraditionalForm\`\(y = \(\(∑\+\(t = 1\)\)\&\(n - 2\)\(\(x\_t\)\_\(+2\)\) x\_t = \(\(∑\+\(t = 3\)\)\&n\(\( x\_t\)\_\(-2\)\) 
                  x\_t = 𝔖\_\(2, 0, 3\)\)\);\)\) \(TraditionalForm\`w = \(\(∑\+\(t = 2\)\)\&\(n - 1\)\(x\_t\) 
              x\_\(t + 1\) = 𝔖\_\(1, 0, 3\)\); \ 
      u = \(\(∑\+\(t = 1\)\)\&\(n - 2\)x\_t\^2 = 𝔖\_\(2, 2, 3\)\); 
      z = \(\(∑\+\(t = 1\)\)\&\(n - 2\)\(\(x\_t\)\_\(+1\)\) 
              x\_t = \(\(∑\+\(t = 3\)\)\&n\( x\_\(t - 1\)\) 
                x\_\(t - 2\) = 𝔖\_\(2, 1, 3\)\)\);\)

(Subscript[x, 0]) = Expect[𝔖[0, 0, 3]]; 
    (Subscript[y, 0]) = Expect[𝔖[2, 0, 3]]; 
    (Subscript[w, 0]) = Expect[𝔖[1, 0, 3]]; 
    (Subscript[u, 0]) = Expect[𝔖[2, 2, 3]]; 
    (Subscript[z, 0]) = Expect[𝔖[2, 1, 3]];

Bias of LS Estimate of (Subscript[ϕ, 2]) in AR(2)

The LS estimate of (Subscript[ϕ, 2]) is

\(TraditionalForm\`\(ϕ\&-\)\_2 = \(x\ y - z\ w\)\/\(x\ u - z\^2\)\)

CoeAR2[x_, y_, w_, u_, z_]:=((x y - z w)/(x u - (z^2)))

The vectors of the first derivatives of CoeAR2[x, y, z, v, u, w] with the respective to x, y, w, z, v and u:

FirstTerm = (D[CoeAR2[x, y, w, u, z], #] & /@ {x, y, w, u, 
                z}) /. {x -> (Subscript[x, 0]), y ->  (Subscript[y, 0]), w ->  (Subscript[w, 0]), 
            u ->  (Subscript[u, 0]), z ->  (Subscript[z, 0])} // Simplify

The vectors of the second derivatives of CoeAR2[x, y, z, v, u, w] with the respective to x, y, w, u and z:

SqTerm = (D[CoeAR2[x, y, w, u, z], #, #] & /@ {x, y, w, u, 
                z}) /. {x -> (Subscript[x, 0]), y ->  (Subscript[y, 0]), w ->  (Subscript[w, 0]), 
            u ->  (Subscript[u, 0]), z ->  (Subscript[z, 0])} // Simplify

a = {x, y, w, u, z}

CrossTermX = (Table[
              D[CoeAR2[x, y, w, u, z], 
                a[[1]], 
                a[[j]]],  {j, 2, 
                5}]) /. {x -> (Subscript[x, 0]), y ->  (Subscript[y, 0]), 
            w ->  (Subscript[w, 0]), u ->  (Subscript[u, 0]), z ->  (Subscript[z, 0])} // 
        Simplify

CrossTermY = (Table[
              D[CoeAR2[x, y, w, u, z], 
                a[[2]], 
                a[[j]]], {j, 3, 
                5}]) /. {x -> (Subscript[x, 0]), y ->  (Subscript[y, 0]), 
            w ->  (Subscript[w, 0]), u ->  (Subscript[u, 0]), z ->  (Subscript[z, 0])} // 
        Simplify

CrossTermW = (Table[
              D[CoeAR2[x, y, w, u, z], 
                a[[3]], 
                a[[j]]], {j, 4, 
                5}]) /. {x -> (Subscript[x, 0]), y ->  (Subscript[y, 0]), 
            w ->  (Subscript[w, 0]), u ->  (Subscript[u, 0]), z ->  (Subscript[z, 0])} // 
        Simplify

CrossTermU = 
      D[CoeAR2[x, y, w, u, z], a[[4]], 
            a[[5]]] /. {x -> (Subscript[x, 0]), 
            y ->  (Subscript[y, 0]), w ->  (Subscript[w, 0]), u ->  (Subscript[u, 0]), 
            z ->  (Subscript[z, 0])} // Simplify

The large sample bias of the LS estimate of (Subscript[ϕ, 2]), (lim_(n ->  ∞))n(
          E((Subscript[(Overscript[ϕ, –]), 2]) - (Subscript[ϕ, 2]))), is

LSBiasϕ2 = 
      FirstTerm . FirstExp + (1/2) SqTerm . ExpSquare + 
          CrossTermX . ExpCrossX + CrossTermY . ExpCrossY + 
          CrossTermW . ExpCrossW + CrossTermU ExpCrossU // 
        FullSimplify