Derivation  of Bias of Burg Estimator

Purpose

This notebook demonstrates how to obtain the large sample biases of the Burg estimates in AR(2) and AR(1), for both the known mean or unknown mean cases, by symbolically implementing the large sample properties of the estimated mean and covariance.

The Large Sample Properties

Estimation of the Mean

By Corollary 6.1.1.1 and Corollary 6.1.1.2 of Fuller (1996, pp. 309), for a stationary time series with absolutely summable covariance function, we have

\(TraditionalForm\`\(Lim\+\(n -> ∞\)\) 
        n\ Var {\(x\&-\)\_n} = \(\(\(∑\+\(h = \(-∞\)\)\)\&∞\)\(γ(h)\)\(\ \)\)\)

and

\(TraditionalForm\`Lim\+\(n -> ∞\)\ Var {\(x\&-\)\_n} = 
      0\)

Estimation of the Autocovariance

If the mean is known, without loss of generality, taken to be zero,  the unbiased estimator of γ(
          m - k) = E {(Subscript[x, t - m]) (Subscript[x, t - k])}  may be written as

(Overscript[γ,  ∼]) (
        h) = (1/(n - (i - 1))) (Overscript[((∑_(t  =  i))), n]) 
                  (Subscript[x, t - m]) 
              (Subscript[x, t - k]) = (Subscript[𝔖, m, k, i])

where m, k,  and i are integers.  If the mean is unknown, one of estimators of γ(h) may be written as

(Overscript[γ, ⋀])(
        h) = (1/n) (Overscript[((∑_(t  =  i))), n])((Subscript[x, t - m]) - 
                  (Subscript[(Overscript[x, –]), n])) ((Subscript[x, t - k]) - 
                  (Subscript[(Overscript[x, –]), n])) = (Subscript[(Overscript[𝔖, –]), m, 
              k, i]).

Thus the large sample covariance of (Subscript[𝔖, m, k, i]) or (Subscript[(Overscript[𝔖, –]), m, k, i]) is satisfied with Theorem 1 or Theorem 2 below.

Theorem 1.  Let the time series {(Subscript[x, t])} be the two-sided moving average,

\(TraditionalForm\`x\_t = \(∑\+\(j = \(-∞\)\)\)\&∞\( α\_j\) e\_\(t - j\)\)

where the sequence {(Subscript[α, j])} is absolutely summable and the (Subscript[e, t]) are independent N(0,  (σ^2)) random variables.  Then, for fixed integers m,  k,  f,  
      g,  i and j (i ≤  j),

\(TraditionalForm\`\(\(lim\ \((n\ \ Cov {𝔖\_\(m, k, i\), 𝔖\_\(f, g, j\)})\) = \(∑\&∞\)\+\(p = \(-∞\)\)T\_p\)\(,\)\)\)

where

(Subscript[T, p]) = {γ(
                g - k + i - j + p) γ(
                f - m + i - j + p) + γ(
                f - k + i - j + p) γ(
                g - m + i - j + p)}.

The proof needs the Lemma (Fuller, Page 314, 1996).

Lemma 1.  Let \(TraditionalForm\`\({δ\_j}\_\(j = \(-∞\)\)\)\&∞\) and  \(TraditionalForm\`\({c\_j}\_\(j = \(-∞\)\)\)\&∞\) be two absolutely summable sequences.  Then, for fixed integers r,  h,  and d, d ≥  0,

\(TraditionalForm\`\(Lim\+\(n -> \ ∞\)\) \(1\/n\) \(\(∑\+\(s = \ 1\)\)\&\(n + d\)\(\(∑\+\(t\  = \ 1\)\)\&n δ\_\(s - t + r\)\ c\_\(s - t + h\)\)\) = \(∑\+\(p = \(-∞\)\)\)\&∞ δ\_\(p + r\)\ c\_\(p + h\)\)

Proof  of Theorem 1.

\(TraditionalForm\`E(\(𝔖\_\(m, k, \ 
              i\)\) 𝔖\_\(f, g, j\)) = \(1\/\(n - \((i - 
                  1)\)\)\) \(1\/\(n - \((j - 
                  1)\)\)\) \(\(∑\+\(s\  = \ i\)\)\&n\(\(∑\+\(t\  = \ j\)\)\&n E\ \(\({\(x\_\(s - m\)\) x\_\(s - k\)\ \(x\_\(t - f\)\) 
                    x\_\(t - g\)}\)\(.\)\)\)\)\)

Using the well-known fourth moment result for products of random variables, we have

E {(Subscript[x, s + m]) (Subscript[x, s + k]) (Subscript[x, t + f]) 
          (Subscript[x, t + g])}

=γ[-f + g] γ[-k + 
              m] + γ[g - k + s - t] γ[
            f - m + s - t] + γ[
            f - k + s - t] γ[
              g - m + s - t].

This can also be computed using our Mathematica Function, ParPow[ ], computes the expectation of any stationary time series products.

Therefore,

Cov((Subscript[𝔖, m, k,  
            i]) (Subscript[𝔖, f, g, j]))

\(\(=\)\(\(1\/\(n - \((i - 1)\)\)\) \(1\/\(n - \((j - 
                      1)\)\)\) \(\(∑\+\(s\  = \ i\)\)\&n\(\(∑\+\(t\  = \ j\)\)\&n\(\({γ[g - k + s - t]\ γ[f - m + s - t] + γ[
                          f - k + s - t]\ γ[
                          g - m + s - t]}\)\(.\)\)\)\)\)\)

Applying Lemma 6.2.1 of Fuller (1996, pp. 314) to the first term of the formula above, we obtain that

\(TraditionalForm\`\(Lim\+\(n -> \ ∞\)\) \(1\/\(n - \((j - 
                1)\)\)\) \(\(∑\+\(s\  = \ i\)\)\&n\(\(∑\+\(t\  = \ j\)\)\&n{\(γ(g - k + s - t)\)\ \(γ(
                f - m + s - t)\)}\)\)\)

\(TraditionalForm\`\(\(=\)\(\(Lim\+\(n -> \ ∞\)\) \(1\/\(n - \((j - 1)\)\)\) \(\(∑\+\(s\  = \ 1\)\)\&\(n - \((i - 1)\)\)\(\(∑\+\(t\  = \ 1\)\)\&\(n - \((j - 1)\)\){\(γ(
                  g - k + i - j + s - t)\)\ \(γ(
                  f - m + i - j + s - t)\)}\)\)\)\)\)

\(TraditionalForm\`\(\(=\)\(\(∑\+\(p = \(-∞\)\)\)\&∞{\(γ(g - k + i - j + p)\)\ \(γ(
              f - m + i - j + p)\)}\)\)\)

since i ≤  j.

Similarly,

\(TraditionalForm\`\(Lim\+\(n -> \ ∞\)\) \(1\/\(n - \((j - 
                1)\)\)\) \(\(∑\+\(s\  = \ i\)\)\&n\(\(∑\+\(t\  = \ j\)\)\&n{\(γ(f - k + s - t)\)\ \(γ(
                g - m + s - t)\)}\)\)\)

\(TraditionalForm\`\(\(=\)\(\(∑\+\(p = \(-∞\)\)\)\&∞\(\({\(γ(f - k + i - j + p)\)\ \(γ(
                g - m + i - j + p)\)}\)\(.\)\)\)\)\)

Thus

\(TraditionalForm\`\(\(\ \)\(\(Lim\+\(n -> \ ∞\)\)\(\ \)\(n\)\(\ \)\(Cov(\(𝔖\_\(m, k, \ 
              i\)\) 𝔖\_\(f, g, j\))\)\(\ \)\)\)\)

\(TraditionalForm\`\(\(=\)\(\(∑\+\(p = \(-∞\)\)\)\&∞\(\({\(γ(g - k + i - j + p)\)\ \(γ(
                  f - m + i - j + p)\) + \(γ(
                  f - k + i - j + p)\)\ \(γ(
                  g - m + i - j + p)\)}\)\(.\)\)\)\)\)

Theorem 2. Let a time series {(Subscript[x, t])} satisfying the assumption of Theorem 1, then

\(TraditionalForm\`\(\(\(Lim\+\(n -> \ ∞\)\) 
        n\ E\ \((𝔖\&–\_\(m, k, i\) - γ(
              m - k))\) = \(-\(\(|\)\(i - 
            1\)\(|\)\(γ(
              m - k) - \(∑\+\(h = \(-∞\)\)\)\&∞ γ(h)\)\)\)\)\(,\)\)\)

\(TraditionalForm\`\(\(lim\ \((n\ \ Cov {𝔖\&–\_\(m, k, i\), 𝔖\&–\_\(f, g, j\)})\) = \(∑\&∞\)\+\(p = \(-∞\)\)T\_p\)\(,\)\)\)

where

(Subscript[T, p]) = {γ(
                g - k + i - j + p) γ(
                f - m + i - j + p) + γ(
                f - k + i - j + p) γ(
                g - m + i - j + p)}.

Proof of Theorem 2.

It is true that (Fuller, 1996, pp. 316)

\(TraditionalForm\`E\ \((𝔖\&–\_\(m, k, i\) - γ(m - k))\)\)

=-((|i - 
                    1|)/n) γ(
            m - k) - ((n - (i - 1))/n) 
          Var {(Subscript[(Overscript[x, –]), n])} + O(
            (n^(-2))).

By (1) and (2), it is turned out that

\(TraditionalForm\`\(Lim\+\(n -> \ ∞\)\) 
        n\ E\ \((𝔖\&–\_\(m, k, i\) - γ(
              m - k))\) = \(-\(\(|\)\(i - 
            1\)\(|\)\(γ(
              m - k) - \(∑\+\(h = \(-∞\)\)\)\&∞\(\( γ(h)\)\(.\)\)\)\)\)\)

Since Var {((Subscript[(Overscript[x, –]), n])^2)} = O((n^(-2))), the second conclusion follows (Fuller, 1996, pp. 317).

Let p=|m - k| andq = |f - g|.  Then (6) or (9) can be written as

\(TraditionalForm\`lim\ \((n\ \ Cov {𝔖\_\(m, k, i\), 𝔖\_\(f, g, j\)})\)\)

\(TraditionalForm\`\(\(\(=\)\(\(∑\&∞\)\+\(h = \(-∞\)\)\(γ(h)\) \(γ(t - p + q)\) + \(γ(
            h)\) \(γ(t - p + q)\)\)\)\(,\)\)\)

where γ(h) is an autocovariance at the lag h.  For the simplicity, we use (10) in the followlng computing codes.

Mathematica Codes

Off[General::"spell"]; Off[General::"spell1"];

Expectation of the stationary time series  product

𝒫[a__] := 
      Module[{k}, 𝒫[{i_, j_}] := γ[
            i - j]; (∑_(k = 1)^(Length[a] - 1))𝒫[{a[[1]], 
                a[[1 + 
                    k]]}] 𝒫 [
              Drop[Drop[a, {1}], {k}]]]

Pow[ω_ a__]:=ω Pow[Times[a]]/;Coe[ω]
Literal[Pow[Times[a__]]]:=Flatten[Map[Pow,List[a]]]
Pow[x[k_]^p_]:=Table[k,{p}]
Pow[x[k_]]:={k}
Protect[Pow];

Coe[c_Integer (ρ^k)_] := True;
Coe[(ρ^k)_] := True;
Coe[c_Integer] := True;
Coe[c_Integer ρ] := True;
Coe[ρ] := True;
ParPow[ω_ a__] := ω ParPow[Times[a]] /; 
        Coe[ω]
ParPow[a__] := 𝒫[Pow[a]]

ParPow[x[s - m] x[s - k] x[t - f] x[t - g]]

Autocovariance  of AR(2) with two unequal characteristic  roots:

R[
          t_] := ((((Subscript[z, 1]) (1 - ((((Subscript[z, 2])))^2)))/(((Subscript[z, 1]) - (Subscript[z, 2])) (1 + (Subscript[z, 1]) (Subscript[z, 2])))) ((((Subscript[z, 1])))^t) - (((Subscript[z, 2]) (1 - ((((Subscript[z, 1])))^2)))/(((Subscript[z, 1]) - (Subscript[z, 2])) (1 + (Subscript[z, 1]) (Subscript[z, 2])))) ((((Subscript[z, 2])))^t)) ((1 + (Subscript[z, 1]) (Subscript[z, 2]))/((1 - (Subscript[z, 1]) (Subscript[z, 2])) (1 - ((((Subscript[z, 1])))^2)) (1 - ((((Subscript[z, 2])))^2))));

where t ≥  0, and γ(t) = γ(-t) when t < 0.

The order (n^(-1)) covariances  of those  (Subscript[𝔖, m, k, i]) or  (Subscript[(Overscript[𝔖, –]), m, k, 
          i])

(lim_(n ->  ∞))n Cov ((Subscript[𝔖, m, k, i]), (Subscript[𝔖, f,  g,  j]) )

\(TraditionalForm\`\(\(=\)\(lim\ \((n\ \ Cov {𝔖\&–\_\(m, k, i\), 𝔖\&–\_\(f, g, j\)})\)\)\)\)

\(TraditionalForm\`\(\(=\)\(∑\+\(n = \(-∞\)\)\%∞\([\(γ(h)\) \(γ(h - p + q)\) + \(γ(h + q)\) \(γ(
                h - p)\)]\)\)\)\)

where p = |m - k| and q = |f - g|.  When p ≥  q ≥  0, those order (n^(-1)) covariances may be obtained by Mathematica function, LCoV[p_,q_] as follows:

LCoV[p_, q_] := 
      Module[{a, b, c, d, e}, 
          a = Sum[R[-h] R[-(h - p + q)] + 
                R[-(h + q)] 
                  R[-(h - 
                          p)], {h, -∞, -q}]; 
b = 
            Sum[R[-h] R[-(h - p + q)] + 
                R[(h + q)] R[-(h - p)], {h, -q + 1, 
                0}]; 
c = 
            Sum[R[h] R[-(h - p + q)] + 
                R[(h + q)] R[-(h - p)], {h, 1, 
                p - q}]; 
d = 
            Sum[R[h] R[(h - p + q)] + 
                R[(h + q)] R[-(h - p)], {h, p - q + 1, 
                p}]; 
e = 
            Sum[R[h] R[(h - p + q)] + R[(h + q)] R[(h - p)], {h, 
                p + 1, ∞}]; 
a + b + c + d + e] // 
        Simplify

Expectation  of (Subscript[𝔖, m, k, i]) in a zero mean case

\(TraditionalForm\`E\ \((𝔖\_\(m, k, i\))\) = \(E\ \((\(1\/\(n - \((i - 1)\)\)\) \(\(∑\+\(t\  = \ i\)\)\&n\( x\_\(t - m\)\) 
                  x\_\(t - k\)\))\) = γ(\(|\)\(m - 
              k\)\(|\))\)\)

Expect[𝔖[m_, k_, i_]] = R[Abs[m - k]]

Order (n^(-1)) bias of (Subscript[(Overscript[𝔖, –]), m, k, i]) in an unknown mean case:

\(TraditionalForm\`\(Lim\+\(n -> \ ∞\)\) 
        n\ E\ \((𝔖\&–\_\(m, k, i\) - γ(
              m - k))\) = \(-\(\(|\)\(i - 
            1\)\(|\)\(γ(
              m - k) - \(∑\+\(h = \(-∞\)\)\)\&∞\(\( γ(h)\)\(.\)\)\)\)\)\)

LB[p_, 
          i_] := -(i - 1) R[
                p] - (Sum[R[-t], {t, -∞, 0}] + 
                Sum[R[t], {t, 1, ∞}]) // Simplify;

The Bias of Burg Estimates in AR(2)

Introduction

For a zero mean AR(2) case, the Burg estimates may be written as

\(TraditionalForm\`\(\(\ \)\(\(ϕ\&-\)\_2 = \ \ \(2 \(\(∑\+\(t = 3\)\)\&n\((x\_t - \(ϕ\&–\_11\) x\_\(t - 1\))\) \((x\_\(t - 2\) - \(ϕ\&–\_11\) x\_\(t - 1\))\)\)\)\/\(\(∑\+\(t = 3\)\)\&n\([\((x\_t - \(ϕ\&–\_11\) x\_\(t - 1\))\)\^2 + \((x\_\(t - 2\) - \(ϕ\&–\_11\) x\_\(t - 1\))\)\^2]\)\)\)\)\)

and

\(TraditionalForm\`\(\(\(ϕ\&-\)\_1 = \ \ ϕ\&–\_11 - ϕ\&–\_2\ ϕ\&–\_11\)\(,\)\)\)

where

\(TraditionalForm\`\(ϕ\&-\)\_11 = \(\(\(2 \(\(∑\_\(t = 2\)\)\&n\( x\_t\) 
                  x\_\(t - 1\)\)\)\/\(\(∑\_\(t = 2\)\)\&n\((x\_t\^2 + \(x\_\(t - 1\)\)\&2)\)\)\)\(.\)\)\)

The estimates can be written as

\(TraditionalForm\`\(ϕ\&-\)\_2 = 
      1 - \(x\ y\^2 - 2  z\ y\^2\)\/\(x\ y\^2 + 8 \( u\^2\) v - 4  u\ w\ y\)\)

and

\(TraditionalForm\`\(\(\(ϕ\&-\)\_1 = \(\(ϕ\&-\)\_11\)(
        1 - \(ϕ\&-\)\_2)\)\(,\)\)\)

where \(TraditionalForm\`x = \(\(1\/\(n - 
                  2\)\) \(\(∑\+\(t = 3\)\)\&n\((\(x\^2\)\_t + \(x\^2\)\_\(t - 2\))\)\) = 𝔖\_\(0, 0, \ 3\) + 𝔖\_\(2, 2, \ 3\)\)\), \(TraditionalForm\`y = \(\(1\/\(n - 
                  1\)\) \(\(∑\+\(t = 2\)\)\&n\((\(x\^2\)\_t + \(x\^2\)\_\(t - 1\))\)\) = 𝔖\_\(0, 0, \ 2\) + 𝔖\_\(1, 1, \ 2\)\)\), z = (1/(n - 
                  2)) (Overscript[((∑_(t = 3))), n]) (Subscript[x, t]) 
                (Subscript[x, t - 2]) = (Subscript[𝔖, 0, 2,  3]), \(TraditionalForm\`\(\(u = \(\(1\/\(n - 
                  1\)\) \(\(∑\_\(t = 2\)\)\&n\( x\_t\) 
                x\_\(t - 1\)\) = 𝔖\_\(0, 1, \ 2\)\)\)\(,\)\)\) \(TraditionalForm\`v = \(\(1\/\(n - 
                  2\)\) \(\(∑\_\(t = 3\)\)\&n\( x\^2\)\_\(t - 1\)\) = 𝔖\_\(1, 1, \ 3\)\)\) and \(TraditionalForm\`w = \(\(1\/\(n - 
                  2\)\) \(\(∑\+\(t = 3\)\)\&n\((\(x\_t\) 
                    x\_\(t - 1\) + \(x\_\(t - 2\)\) 
                    x\_\(t - 1\))\)\) = 𝔖\_\(0, 1, \ 3\) + 𝔖\_\(2, 1, \ 3\)\)\).

Let π=(x, y, z, u, v, w).  Thus the Burg estimators are the ratio functions of π.  The moments of \(\(ϕ\&-\)\_1\)(π) and  \(TraditionalForm\`\(ϕ\&-\)\_2\)(π) can be expanded to order (n^(-1)) around (Subscript[π, 0])=((Subscript[x, 0]), (Subscript[y, 0]), (Subscript[z, 0]), (Subscript[u, 0]), (Subscript[v, 0]), (Subscript[w, 0]))=(2  γ(0),  
        2  γ(0),  γ(2),  γ(1),  γ(0),  
        2  γ(1)).  For a known mean case, π is an unbiased estimator of (Subscript[π, 0]).  Hence

\(E\ \((\(ϕ\&-\)\_2 - ϕ\_2)\)\)=(1/2){((E(x-(Subscript[x, 0])))^2)·(((∂^2)\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/((∂^2) x))+...+((E(w - (Subscript[w, 0])))^2)(((∂^2)\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/((∂^2) w))}+

{E[(x-(Subscript[x, 0]))(y-(Subscript[y, 0]))]·(((∂^2)\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/(∂x ∂y))+...+E[(v - (Subscript[v, 0])) (w - (Subscript[w, 0]))](((∂^2)\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/(∂v ∂w))}+O((n^(-2))).

The above evaluation requires to compute the expectations of all of the squared terms and cross terms of π, that is, all the variances and covariances of π.  There are totally six variances and fifteen covariances.  The order (n^(-1)) bias component may be computed as follows:

\(TraditionalForm\`\(\(\ \)\(lim\ \((n\ \ E\ \((\(ϕ\&-\)\_2 - ϕ\_2)\))\)\)\)\)

=(1/2){lim (n Var(x))·(((∂^2)\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/((∂^2) x))+...+lim (n Var (w))(((∂^2)\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/((∂^2) w))}+

{lim (n Cov (x y))·(((∂^2)\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/(∂x ∂y))+...+lim (n Cov (v w))(((∂^2)\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/(∂v ∂w))}

=(1/2){lim (n Var(x)), ..., lim (n Var (w))}.{(((∂^2)\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/((∂^2) x)),...,(((∂^2)\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/((∂^2) w))}+

{lim (n Cov (x y)), ..., lim (n Cov (v w))}.{(((∂^2)\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/(∂x ∂y)),...,(((∂^2)\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/(∂v ∂w))}.

The bias coefficient to order (n^(-1)) can be considered as a sum of two inner products: one is related to the limiting variance vector of π to order (n^(-1)) and another is related to the limiting covariance vector of π to order (n^(-1)).  For an unknown mean case, (Subscript[𝔖, m, k, i]) will be replaced by (Subscript[(Overscript[𝔖, –]), m, k, i]).  Similarly, the bias to order (n^(-1)) becomes

\(TraditionalForm\`\(\(\ \)\(lim\ \((n\ \ E\ \((\(ϕ\&-\)\_2 - ϕ\_2)\))\)\)\)\)

=lim(n Bias(x)) ((∂\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/(∂x))+...+lim(n Bias(w)) ((∂\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/(∂w))+

(1/2){lim (n Var(x))(((∂^2)\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/((∂^2) x))+...+lim (n Var (w))(((∂^2)\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/((∂^2) w))}+

{lim (n Cov (x y))(((∂^2)\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/(∂x ∂y))+...+lim (n Cov (v w))(((∂^2)\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/(∂v ∂w))}

={lim(n Bias(x)) ,   ... , lim(n Bias(w))}.{ ((∂\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/(∂x)),...,  ((∂\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/(∂w))}+

(1/2){lim (n Var(x)), ..., lim (n Var (w))}.{(((∂^2)\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/((∂^2) x)),...,(((∂^2)\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/((∂^2) w))}+

{lim (n Cov (x y)), ..., lim (n Cov (v w))}.{(((∂^2)\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/(∂x ∂y)),...,(((∂^2)\(\(ϕ\&-\)\_2\)((Subscript[π, 0])))/(∂v ∂w))}.

The bias coefficient to order (n^(-1)) in unknow mean cases includes one more inner product term than in zero mean cases.  This extra inner product term is related to the bias vector of π to order (n^(-1)) and the other two remain the same as in zero mean cases since Theorem 1 and 2.  Equations (11) and (12) will be used to compute the order (n^(-1))biases of the Burg estimators in a known mean case or an unknown mean case.

Large Sample Results of x, y, z, u, v, and w

lim  (n V a r( x)),  
      lim  (n V a r( y)),  ... ,  
      and lim  (n V a r( w))

ExpSquareX = ExpSquareY = 4  LCoV[0, 0]

ExpSquareW = 4 LCoV[1, 1]

ExpSquareZ = LCoV[2, 2]

ExpSquareV = LCoV[0, 0]

ExpSquareU = LCoV[1, 1]

ExpSquare = {ExpSquareX, ExpSquareY, ExpSquareW, ExpSquareZ, 
          ExpSquareV, ExpSquareU} // Simplify

lim  (n C o v( x y)),  
      lim  (n C o v (x z)),  ... ,  
      and lim  (n C o v (v w))

ExpCrossX = {4  LCoV[0, 0], 4  LCoV[1, 0], 2  LCoV[2, 0], 
        2  LCoV[0, 0], 2  LCoV[1, 0]}

ExpCrossY = {4  LCoV[1, 0], 2  LCoV[0, 2], 2  LCoV[0, 0], 
        2  LCoV[1, 0]}

ExpCrossW = {2  LCoV[2, 1], 2  LCoV[1, 0], 2  LCoV[1, 1]}

ExpCrossZ = {LCoV[2, 0], LCoV[2, 1]}

ExpCrossV = LCoV[1, 0]

lim  (n Bias ( x)),  
      lim  (n Bias ( y)),  ... ,  
      and lim  (n Bias ( w))

FirstExp = {2 LB[0, 3], 2  LB[0, 2], 2  LB[1, 3], LB[2, 3], LB[0, 3], 
          LB[1, 2]} // Simplify

The expectations  of x,  y,  z,  u,  v and w:

(Subscript[x, 0]) = 
        Map[Expect, 𝔖[0, 0, 3] + 𝔖[2, 2, 
              3]];

(Subscript[z, 0]) = Expect[𝔖[0, 2, 3]];

(Subscript[y, 0]) = 
        Map[Expect, 𝔖[0, 0, 2] + 𝔖[1, 1, 
              2]];

(Subscript[v, 0]) = Expect[𝔖[1, 1, 3]];

(Subscript[u, 0]) = Expect[𝔖[0, 1, 2]];

(Subscript[w, 0]) = 
        Map[Expect, 𝔖[0, 1, 3] + 𝔖[2, 1, 
              3]];

Some Derivatives  of the Burg Estimator of (Subscript[ϕ, 2])

The Burg estimator of (Subscript[ϕ, 2]) is

CoeAR2[x_, y_, z_, v_, u_, w_] := 
      1 - ((x (y^2) - 2 z (y^2))/(x (y^2) + 8 (u^2) v - 4  u w y))

The vector of the first derivatives of CoeAR2[x, y, z, v, u, w] with the respective to x, y, w, z, v and u:

FirstTerm = (D[CoeAR2[x, y, z, v, u, w], #] & /@ {x, y, w, z, v, 
                u}) /. {x -> (Subscript[x, 0]), y ->  (Subscript[y, 0]), w ->  (Subscript[w, 0]), 
            z ->  (Subscript[z, 0]), v -> (Subscript[v, 0]), u -> (Subscript[u, 0])} // 
        Simplify

The vectors of the second derivatives of CoeAR2[x, y, z, v, u, w]  with the respective to x, y, w, z, v and u:

SqTerm = (D[CoeAR2[x, y, z, v, u, w], #, #] & /@ {x, y, w, z, v, 
                u}) /. {x -> (Subscript[x, 0]), y ->  (Subscript[y, 0]), w ->  (Subscript[w, 0]), 
            z ->  (Subscript[z, 0]), v -> (Subscript[v, 0]), u -> (Subscript[u, 0])} // 
        Simplify

a = {x, y, w, z, v, u}

CrossTermX = (Table[
              D[CoeAR2[x, y, z, v, u, w], 
                a[[1]], 
                a[[j]]],  {j, 2, 
                6}]) /. {x -> (Subscript[x, 0]), y ->  (Subscript[y, 0]), 
            w ->  (Subscript[w, 0]), z ->  (Subscript[z, 0]), v -> (Subscript[v, 0]), 
            u -> (Subscript[u, 0])} // Simplify

CrossTermY = 
      Table[D[CoeAR2[x, y, z, v, u, w], 
              a[[2]], 
              a[[j]]],  {j, 3, 
              6}] /. {x -> (Subscript[x, 0]), y ->  (Subscript[y, 0]), w ->  (Subscript[w, 0]), 
            z ->  (Subscript[z, 0]), v -> (Subscript[v, 0]), u -> (Subscript[u, 0])} // 
        Simplify

CrossTermW = 
      Table[D[CoeAR2[x, y, z, v, u, w], 
              a[[3]], 
              a[[j]]],  {j, 4, 
              6}] /. {x -> (Subscript[x, 0]), y ->  (Subscript[y, 0]), w ->  (Subscript[w, 0]), 
            z ->  (Subscript[z, 0]), v -> (Subscript[v, 0]), u -> (Subscript[u, 0])} // 
        Simplify

CrossTermZ = 
      Table[D[CoeAR2[x, y, z, v, u, w], 
              a[[4]], 
              a[[j]]],  {j, 5, 
              6}] /. {x -> (Subscript[x, 0]), y ->  (Subscript[y, 0]), w ->  (Subscript[w, 0]), 
            z ->  (Subscript[z, 0]), v -> (Subscript[v, 0]), u -> (Subscript[u, 0])} // 
        Simplify

CrossTermV = 
      Table[D[CoeAR2[x, y, z, v, u, w], 
              a[[5]], 
              a[[6]]]] /. {x -> 
              (Subscript[x, 0]), y ->  (Subscript[y, 0]), w ->  (Subscript[w, 0]), z ->  (Subscript[z, 0]), 
            v -> (Subscript[v, 0]), u -> (Subscript[u, 0])} // Simplify

Bias Coefficient of the Burg Estimator of (Subscript[ϕ, 2]) to Order (n^(-1)) in Zero Mean Cases

By (11), the order (n^(-1)) bias of the Burg estimate of (Subscript[ϕ, 2]) in zero mean cases, (lim_(n ->  ∞))n(
          E((Subscript[(Overscript[ϕ, –]), 2]) - (Subscript[ϕ, 2])))

ZeroMeanLSBiasϕ2 = (1/2) SqTerm . ExpSquare + 
          CrossTermX . ExpCrossX + CrossTermY . ExpCrossY + 
          CrossTermW . ExpCrossW + CrossTermZ . ExpCrossZ + 
          CrossTermV ExpCrossV // Simplify

Bias Coefficient  of the Burg Estimator of (Subscript[ϕ, 2]) to Order (n^(-1)) in Unknown Mean Cases

By (12), the order (n^(-1))bias of the Burg estimate of (Subscript[ϕ, 2]) in unknown mean cases, (lim_(n ->  ∞))n(
          E((Subscript[(Overscript[ϕ, –]), 2]) - (Subscript[ϕ, 2]))), is

UnknownMeanLSBiasϕ2 = 
      FirstTerm . FirstExp + (1/2) SqTerm . ExpSquare + 
          CrossTermX . ExpCrossX + CrossTermY . ExpCrossY + 
          CrossTermW . ExpCrossW + CrossTermZ . ExpCrossZ + 
          CrossTermV ExpCrossV // Simplify

Bias Coefficient of the Burg Estimator of (Subscript[ϕ, 1]) to Order (n^(-1)) in Zero Mean Cases

Since

\(TraditionalForm\`\(ϕ\&-\)\_1 = \(\(ϕ\&-\)\_11\)(
        1 - \(ϕ\&-\)\_2)\)

where

\(TraditionalForm\`\(\(\(ϕ\&-\)\_11 = \(2 \(\(∑\_\(t = 2\)\)\&n\( x\_t\) x\_\(t - 1\)\)\)\/\(\(∑\_\(t = 2\)\)\&n\((x\_t\^2 + \(x\_\(t - 1\)\)\&2)\)\)\)\(,\)\)\)

thus

\(TraditionalForm\`lim\+\(n -> \ ∞\)\ 
        n\ E\ \((ϕ\&–\_12 - ϕ\_1)\)\  = \((1 + \(z\_1\) 
                z\_2)\)\ \(lim\+\(n -> \ ∞\)\ {n\ E\ \((ϕ\&–\_11 - ρ(1))\)\ }\) - \(\(z\_1 + z\_2\)\/\(1 + 
                z\_1\ z\_2\)\) \(lim\+\(n -> \ ∞\){\ 
              n\ E\ \((ϕ\&–\_22 - ϕ\_2)\)}\) + 
        O(1\/\(\(n\^2\)\(\ \)\))\)

where ρ(1) is the first order autocorrelation.

\(TraditionalForm\`lim\+\(n -> \ ∞\)n\ E\ \((ϕ\&–\_11 - ρ(1))\)\) can be computed using the same approach as the large sample bias of the Burg estimate of (Subscript[ϕ, 2]).

Since

\(TraditionalForm\`\(\(\(ϕ\&-\)\_11 = \(\(2 \(\(∑\_\(t = 2\)\)\&n\( x\_t\) x\_\(t - 1\)\)\)\/\(\(∑\_\(t = 2\)\)\&n\((x\_t\^2 + \(x\_\(t - 1\)\)\&2)\)\) = \(2  u\)\/y\)\)\(,\)\)\)

by (11), thus

\(TraditionalForm\`lim\+\(n -> \ ∞\)n\ E\ \((ϕ\&–\_11 - ρ(1))\)\  = \(1\/2\) \(lim\+\(n -> \ ∞\)\ 
            n\ \(Var(
                  u)\)·∂\^2\(\(\( ϕ\&-\)\_11\)(u, y)\)\/∂\^2 u\) + \(1\/2\) \(lim\+\(n -> \ ∞\)\ 
            n\ \(Var(
                  y)\)·∂\^2\(\(\( ϕ\&-\)\_11\)(u, y)\)\/∂\^2 y\) + 
        lim\+\(n -> \ ∞\)\ 
          n\ \(Cov(u, 
              y)\) \(\(∂\^2\(\(\( ϕ\&-\)\_11\)(u, 
                      y)\)\/\(∂u ∂y\)\)\(.\)\)\)

ARϕ11[u_, y_] := (2  u)/y

Lnvarianceϕ11 = {LCoV[1, 1],  4  LCoV[0, 0]}

LnCovarianceϕ11 = 2  LCoV[1, 0]

SqrDtermϕ11 = {D[ARϕ11[u, y], u, u], 
            D[ARϕ11[u, y], y, y]} /. {u ->  (Subscript[u, 0]), 
            y ->  (Subscript[y, 0])} // Simplify

CrossDtermϕ11 = 
      D[ARϕ11[u, y], u, y] /. {u ->  (Subscript[u, 0]), y ->  (Subscript[y, 0])} // 
        Simplify

\(TraditionalForm\`lim\+\(n -> \ ∞\)\ \ {n\ E\ \((ϕ\_\(\(11\)\(\ \)\) - ρ(1))\)}\) may be obtained by

ZeroMeanLSBiasϕ11 = (1/2) 
            SqrDtermϕ11 . Lnvarianceϕ11 + 
          CrossDtermϕ11*LnCovarianceϕ11 // Simplify

Therefore, the large sample bias of Burg estimate of (Subscript[ϕ, 1 ]) may be obtained by

LSBiasϕ1 = (1 + (Subscript[z, 1]) 
                  (Subscript[z, 2])) ZeroMeanLSBiasϕ11 - (((Subscript[z, 1]) + (Subscript[z, 2]))/(1 + 
                  (Subscript[z, 1]) (Subscript[z, 2]))) ZeroMeanLSBiasϕ2 // Simplify

Bias of the Burg Estimate of (Subscript[ϕ, 1]) to Order (n^(-1)) in Unknown Mean Cases

Let's consider the term associated with the bais of u and y.

LnFirstExpϕ11 = {LB[1, 2], 2  LB[0, 2]} // Simplify

The vector of the first derivatives of ARϕ11 with the respective to u and y.

FirstTermϕ11 = {D[ARϕ11[u, y], u], 
            D[ARϕ11[u, y], y]} /. {u ->  (Subscript[u, 0]), y ->  (Subscript[y, 0])} // 
        Simplify

In unknown mean cases, lim n E ((Subscript[ϕ, 11 ]) - ρ(
              1)) may be obtained by (12), that is,

UnknownMeanLSBiasϕ11 = 
      LnFirstExpϕ11 . FirstTermϕ11 + (1/2) 
            SqrDtermϕ11 . Lnvarianceϕ11 + 
          CrossDtermϕ11*LnCovarianceϕ11 // Simplify

The bais of the  Burg estimate of (Subscript[ϕ, 1])to order (n^(-1)) in unknown mean cases is

UnknownMeanLSBiasϕ1 = (1 + (Subscript[z, 1]) 
                  (Subscript[z, 2])) UnknownMeanLSBiasϕ11 - (((Subscript[z, 1]) + 
                  (Subscript[z, 2]))/(1 + (Subscript[z, 1]) (Subscript[z, 2]))) UnknownMeanLSBiasϕ2 // 
        Simplify

The Bias of Burg Estimates in AR(1)

Considering AR(1) as

(Subscript[x, t]) - ρ (Subscript[x, t - 1]) = (Subscript[a, t]),  t = 1, 
    2,  ... ,  n,

where |ρ| < 1 and {(Subscript[a, t ])} is a sequence of independently and normally distributed random variables with zero mean and unit variance.  The Burg estimate can be written as \(TraditionalForm\`\(ϕ\&-\)\_11\) in AR(2), that is

\(TraditionalForm\`ρ\&– = \(\(2 \(\(∑\_\(t = 2\)\)\&n\( x\_t\) x\_\(t - 1\)\)\)\/\(\(∑\_\(t = 2\)\)\&n\((x\_t\^2 + \(x\_\(t - 1\)\)\&2)\)\) = \(\(\(2  u\)\/y\)\(.\)\)\)\)

We can use the same computing functions used in AR(2).  The only thing that need to modifiy is to change the AR(2) autocovariance into the AR(1)'s.

R[t_] := (ρ^t)/(1 - (ρ^2))

AR1ρ[u_, y_] := (2  u)/y

Expect[𝔖[m_, k_, i_]] = R[Abs[m - k]]

(Subscript[y, 0]) = 
      Map[Expect, 𝔖[0, 0, 2] + 𝔖[1, 1, 2]];  
    (Subscript[u, 0]) = Expect[𝔖[0, 1, 2]];

Lnvarianceρ = {LCoV[1, 1],  4  LCoV[0, 0]}

LnCovarianceρ = 2  LCoV[1, 0]

FirstDTermρ = {D[AR1ρ[u, y], u], 
          D[AR1ρ[u, y], y]} /. {u ->  (Subscript[u, 0]), 
          y ->  (Subscript[y, 0])}

LnBiasρ = {LB[1, 2], 2  LB[0, 2]}

SqrDtermρ = {D[AR1ρ[u, y], u, u], 
          D[AR1ρ[u, y], y, y]} /. {u ->  (Subscript[u, 0]), 
          y ->  (Subscript[y, 0])}

CrossDtermρ = 
      D[AR1ρ[u, y], u, y] /. {u ->  (Subscript[u, 0]), 
          y ->  (Subscript[y, 0])}

By (11), in a zero mean case, \(TraditionalForm\`lim\ n\ E\ \((\(ρ\&–\_\ \) - ρ)\)\) may be obtained by

ZeroMeanLSBiasρ = (1/2) (SqrDtermρ . 
                Lnvarianceρ) + CrossDtermρ*LnCovarianceρ // 
        Simplify

By (12), in an unknown mean case, \(TraditionalForm\`lim\ n\ E\ \((\(ρ\&–\_\ \) - ρ)\)\) may be obtained by

UnknownMeanLSBiasρ = 
      FirstDTermρ . 
            LnBiasρ + (1/2) (SqrDtermρ . Lnvarianceρ) + 
          CrossDtermρ*LnCovarianceρ // Simplify