Multiple Linear Regression

A.I. McLeod, University of Western Ontario
Copyright ©, A.I. McLeod, 1999.

This notebook illustrates the use of the Mathematica Standard Package, Linear Regression.  We analyze the famous hills datasets discussed in the Splus book Modern Applied Statistics with Splus, Venables and Ripley (1997)

Needs["Statistics`LinearRegression`"];

Here is a list of the data file.

!! "\\p\\math\\1998\\jsm\\hills.txt"

Input the data and form separate lists of the variables.

hills = 
      ReadList["c:\\p\\math\\1998\\jsm\\hills.dat", {Word, Real, Real, 
          Real}, WordSeparators  ->  {"\n"}];

Short[hills, 2]

{race, dist, climb, time} = Transpose[hills];

hills2 = Transpose[{climb,  time,  dist}];

Fit a multiple linear regression using the Regress function.

Regress[hills2, {CLIMB, DIST}, {CLIMB,  DIST}]

At first glance, the model is a bit unsatisfactory since the intercept term would expected to be zero.  There are many options for the output.

?RegressionReport

RegressionReportValues[Regress]

We refit and save the output for diagnostic checking.

hillmodel = 
      Regress[hills2, {1, CLIMB, TIME}, {CLIMB,  TIME}, 
        RegressionReport -> {CookD, FitResiduals, StudentizedResiduals}];

Get the Cook distances and studentized residuals for diagnostics,

cd = CookD /. hillmodel;

stres = StudentizedResiduals /. hillmodel;

For convenience we are going to reset the default options to ListPlot,

SetOptions[ListPlot, PlotRange -> All, Frame -> True, Axes -> False, 
      PlotStyle -> {RGBColor[0, 0, 1], AbsolutePointSize[5]}];

ListPlot[cd];

i = 
      Select[Range[Length[cd]], 
        cd[[#]] > 1&]

race[[i]]

ListPlot[Transpose[{stres, cd}]];

threepts = 
      Transpose[{race, stres, cd}][[i
        ]];

threepts

Text[#[[1
            ]], {#[[2
              ]], #[[3
              ]]}]&/@threepts

ListPlot[Transpose[{stres, cd}], 
      Epilog -> {
          Text[#[[1
                  ]], {#[[2
                    ]], #[[3
                    ]]}, {0, 1}]&/@threepts}];

threepts = 
      Transpose[{race, dist, stres}][[i
        ]];

ListPlot[Transpose[{dist, stres}], 
      Epilog -> {
          Text[#[[1
                  ]], {#[[2
                    ]], #[[3
                    ]]}, {0, 1}]&/@threepts}];

threepts = 
      Transpose[{race, climb, stres}][[i
        ]];

ListPlot[Transpose[{climb, stres}], 
      Epilog -> {
          Text[#[[1
                  ]], {#[[2
                    ]], #[[3
                    ]]}, {0, 1}]&/@threepts}];

<< "c:\\p\\math\\1998\\jsm\\eiss.m"

NPlot[stres];

new = Complement[Range[35], i];

hillmodel = 
      Regress[hills2[[new]], {CLIMB, 
          DIST, DIST^2}, {CLIMB,  DIST}, 
        Weights -> dist[[new]], 
        RegressionReport -> {ParameterTable, CookD, FitResiduals, 
            StudentizedResiduals}];

cd = CookD /. hillmodel;

ListPlot[cd];

stres = StudentizedResiduals /. hillmodel;

ListPlot[
      Transpose[{dist[[new]], stres}]];

ListPlot[
      Transpose[{climb[[new]], stres}]];

NPlot[stres];

Lets refit without the intercept term,

hillmodel = 
      Regress[hills2[[new]], {CLIMB, 
          DIST, DIST^2}, {CLIMB,  DIST}, 
        Weights -> dist[[new]], 
        RegressionReport -> {ParameterTable}, IncludeConstant -> False]