“Developments in Statistical Computing for Time Series Analysis” by Ian McLeod

November 13, 2012.  Systems Design Engineering, University of Waterloo

http://www.stats.uwo.ca/faculty/aim/2012/UWSeminar/

Mathematica slideshow also available as PDF and CDF (Computable Document Format)

Copyright © A.I. McLeod, 2012

QPE and Statistical Software

QPE

free: R, python

$$$: Mathematica, MatLab, etc

Menu style

free: MHTS, JMulti, Weka

$$$: SAS, SPSS, etc

MHTS Package

ARIMA(p,d,q) models are a type of linear time series model,

(Subscript[z, t])-(Subscript[ϕ, 1])(Subscript[z, t-1])-…-(Subscript[ϕ, p])(Subscript[z, t-p])=ζ+(Subscript[a, t])-(Subscript[θ, 1])(Subscript[a, t-1])-…-(Subscript[θ, q])(Subscript[a, t-q]), (Subscript[a, t])~IID(0,(Subscript[σ, a]^2))

Extensions: regression, transfer-function-noise, seasonal models, intervention analysis
estimation, diagnostic checking, forecasting and simulation, Granger causality, trend analysis, and spectral analysis.

Time Series Modelling of Water Resources and Environmental Systems. Keith Hipel & Ian McLeod

McLeod-Hipel Time Series Package

Box-Jenkins Iterative Model Building

ToTimes[x_]:=Style[x,FontFamily->"Helvetica",FontSize->14,FontColor->RGBColor[0,0,1]];
ToTimesLarger[x_]:=Style[x,FontFamily->"Times",FontSize->20, FontColor->RGBColor[1,0.3,1]];
fz=16;Show[Graphics[
{
{AbsoluteThickness[5],Yellow,Disk[{0,0},{3,1}],Disk[{0,4},{3,1}],Disk[{0,8},{3,1}]},
Text[Style["Model Criticism", fz, "Helvetica", Blue, Bold],{0,0}],
Text[Style["Model Calibration", fz, "Helvetica", Blue, Bold],{0,4}],
Text[Style["Model Selection", fz, "Helvetica", Blue, Bold],{0,8}],
Line[{{-3,0},{-4,0}}],Line[{{-3,8},{-4,8}}],
{Arrowheads[Large],Arrow[{{0,7},{0,5}}],
Arrow[{{0,3},{0,1}}],
Arrow[{{-3,0},{-4,0}}],
Arrow[{{-4,8},{-3,8}}],
Arrow[{{-4,0},{-4,4}}],
Arrow[{{-4,4},{-4,8}}]}},
AspectRatio->Automatic,PlotLabel->Style["Iterative Model Building", 22, "Helvetica", "Bold",Green]
]
]

R Programming Language

CRAN: http://cran.r-project.org/

http://probability.ca/cran/

Although a relatively new QPE, built on solid foundations. Generally very high quality. Elementary to advanced statistics.

Rich ecosystems: refereed journal (jss), reproducible research, books, blogs, consulting

Interface with SPSS, Excel, Mathematica  (Version 9), Weka, C, Java, Fortran, etc.

Built-in support for parallel and cluster computing. 64 bit versions.

Time Series Analysis with R

“Time Series Analysis with R” (2012), McLeod, Yu and Krougly

http://www.stats.uwo.ca/faculty/aim/tsar/default.htm

Time Series Analysis Textbooks with R Packages

Venables and Ripley, 2002

Lutkepohl and Kratzig, 2004

Cryer and Chan, 2008

Chan, 2010

Tsay, 2010

Shumway and Stoffer, 2011

Time Series Plots

lynx series, Beveridge wheat prices, (Subscript[CO, 2])

IBM returns

annual temperatures

Airline Passengers

Creatine Clearances

VAR and SVAR Models

often environmental and economic time series are multivariate

(Subscript[y, t]) is a k-dimensional time series

(Subscript[y, t])=δ (Subscript[d, t])+(Subscript[Φ, 1])(Subscript[y, t-1])+…+(Subscript[Φ, p])(Subscript[y, t-p])+(Subscript[e, t]) (VAR)

A (Subscript[y, t])=δ (Subscript[d, t])+(Subscript[Φ, 1])(Subscript[y, t-1])+…+(Subscript[Φ, p])(Subscript[y, t-p])+B (Subscript[e, t]) (SVAR)

Mahdi & McLeod, Improved Multivariate Portmanteau Test, JTSA, March 2012.

Kalman Filter

includes ARMA, multivariate ARMA and many other models

estimation, optimal smoothing, interpolation and forecasting

Nile riverflow example: random walk with additive noise:

(Subscript[y, t])=(Subscript[θ, t])+(Subscript[v, t]), (Subscript[v, t])~NID(0, V)

(Subscript[θ, t])=(Subscript[θ, t-1])+(Subscript[w, t]), (Subscript[w, t])~NID(0, W)

Unit Root Testing: The Null Hypothesis

Consider the model (Subscript[z, t])=c+ϕ (Subscript[z, t-1])+(Subscript[a, t]), (Subscript[a, t])~NID(0, (Subscript[σ, a]^2)).  (Subscript[ℌ, 0]):ϕ=1 vs (Subscript[ℌ, a]):ϕ<1.

If  (∇^d)(Subscript[z, t]) is stationary for d=1,2,… it is called difference-stationary.

More general (Subscript[ℌ, 0]) states that (Subscript[z, t]) is difference-stationary and alternate is that (Subscript[z, t]) is stationary.

Large research literature including research monographs.

Advanced probability theory used to derive asymptotic distributions but practical answer provided by using Response Surface Regression methods using large-scale simulations to obtain more exact p-values.

Model Building Approach to Unit Roots

∇ (Subscript[z, t])=(Subscript[c, t])+τ (Subscript[z, t-1])+(∑_(i=1)^p)(Subscript[ϕ, i]) ∇(Subscript[z, t-i])+(Subscript[a, t])

(Subscript[c, t])=(Piecewise[{{0, none}, {(Subscript[β, 0]), drift}, {(Subscript[β, 0])+(Subscript[β, 1])t, trend}}])

(Subscript[H, 0]):τ=0  and the alternative is (Subscript[H, a]): τ<0.

Co-integration

http://nobelprize.org/nobel_prizes/economics/laureates/2003/granger-lecture.html

Looks for deep relationships between econometric/financial series that are assumed to be difference-stationary.

(Subscript[x, t]) and (Subscript[y, t]) are co-integrated if for some β≠0, (Subscript[y, t])-β (Subscript[x, t]) is stationary.

consumption and income

wages and prices

short and long term interest rates

VECH Model and Test for Co-integration

(Subscript[y, t]) is k-dimensional time series, t=1,2,… then VEC (vector-error-correction) model,

∇(Subscript[y, t])=Π (Subscript[y, t])+(Subscript[Γ, 1])∇(Subscript[y, t-1])+…+(Subscript[Γ, p])∇(Subscript[y, t-p])+(Subscript[a, t]), (Subscript[a, t])~NID(0, (Subscript[Σ, a]))

co-integration exists if 0<rank Π<k

model building procedure used to construct suitable VEC model

Nonlinear time series

generalized linear models and generalized additive models: Car Fatalities IA

multi-adaptive regression splines (MARS)

neural nets

support vector machines

GARCH/ARMA

GARCH(r,s) provides a model for the innovations, (Subscript[a, t])~NID(0, (Subscript[σ, t]^2))

(Subscript[σ, t]^2)=(Subscript[α, 0])+(Subscript[α, 1])(Subscript[a, t-1]^2)+…+(Subscript[α, r])(Subscript[a, t-r]^2)+(Subscript[β, 1])(Subscript[σ, t-1]^2)+…+(Subscript[β, s])(Subscript[σ, t-s]^2)

Theorem: (Subscript[a, t])~GARCH(r,s) ⟺ (Subscript[a, t]^2)~ARMA(r,s).

Simulation of GARCH(1,1) with (Subscript[α, 0])=(10^(-6)),(Subscript[α, 1])=0.2,(Subscript[β, 1])=0.7 with GED error distribution with skewness coefficient 1.25 and shape parameter 4.8.  R-Script for Simulated GARCH.

Fitting ARMA/GARCH model to U.S. GNP deflator for 1947 to 2010 (quarterly). Let (Subscript[r, t])=log((Subscript[z, t]))-log((Subscript[z, t-1])) then the final model is given by

(Subscript[r, 1])=0.103+0.369 (Subscript[r, t-1])+0.223 (Subscript[r, t-2])+0.248 (Subscript[r, t-3])+(Subscript[ϵ, t])

(Subscript[σ, t]^2)=0.004+0.269 (Subscript[ϵ, t-1]^2)+0.396(Subscript[r, t-1])+0.223 (Subscript[r, t-2])+0.248 (Subscript[r, t-3])

R-Script for GNP Deflator Example.

Threshold Autoregression (TAR)

Tsay (2010) fits a two-regime TAR model to the monthly U.S. unemployment data, (Subscript[z, t]), January 1948 to March 2009 (n=735).

(Subscript[w, t])=0.983 (Subscript[w, t-2])+0.158(Subscript[w, t-3])+0.0118(Subscript[w, t-4])-0.180 (Subscript[w, t-12])+(Subscript[a, 1,t]), if (Subscript[w, t-1])≤0.01

(Subscript[w, t])=0.421 (Subscript[w, t-2])+0.239(Subscript[w, t-3])-0.127(Subscript[w, t-12])+(Subscript[a, 2,t]), if (Subscript[w, t-1])>0.01

where (Subscript[w, t])=∇(Subscript[z, t]).  This model fits better than SARIMA.  See R graphics plot.

Wavelet Methods for Time Series

Best books are Percival & Walden (2000), Wavelet Methods for Time Series Analysis and an applied version of this book by Gencay et al. An Introduction to Wavelets and Other Filtering Methods in Finance and Economics.  Several packages available in R. I have implemented WMTSA in Mathematica for my research and teaching.

Wavelet variance, denoising, smoothing, multi-resolution analysis.

Recent research uses wavelet methods for forecasting riverflow.

R-Script: denoised mean Annual Nile Riverflow

Stochastic Differential Equations

SDE’s used in financial mathematics and many other areas

in math finance simulation of SDE’s used to get answers

recent book by Iacus (2009) and R package provides estimation and simulation

d x(t)=(5-11x(t)+6((x(t))^2)-((x(t))^3))dt+W(t)

simulation using Shoji-Ozaki algorithm: R-Script

Mathematica

high level advanced programming language

superb graphics, numerics, symbolics, technical typesetting

curated datasets, interface to Wolfram|Alpha, C and R

Mathematica Demonstrations

ARIMA Explorer

Mathematica Demonstrations Project

Visualizing the Admissible Region

AR(4) is a more complex 4-dimensional region.

Symbolic Computing and Time Series

Derivation of asymptotic bias in Yule-Walker, LS and Burg

Derivation of asymptotic distribution of a new unit root test

Derivation of closed form expression of the SDF of FNG

WolframAlpha Pro

The free version is amazing:

replaces need for mathematical tables and integrals

does many symbolic computations

natural language processing interface to the world’s computational knowledge

available on mobile devices: Android, iOS

Pro version has many more amazing features:

can output data in csv format

input data for automated statistical analysis

Sonification

“Listening to uncertainty: Information that sings”, Ethan Brown & Nick Bearman (2012).  Significance, October 2012.