The purpose of these timings is to demonstrate that in certain circumstances
the widely held view that you can always dramatically improve on the cpu time required for
lengthy computations by using compiled C or Fortran code instead of advanced QPEs such as
* Mathematica* is wrong.

It is frequently stated that interpretative languages are not as useful as
compiled languages such as C and Fortran for numerically intensive
computations. It is obviously the case that an well written C program can
be expected to outperform * Mathematica*, *R, *S-Plus or MATLAB but there are two caveats that need to be
understood. First, if the C program is not efficiently programmed using
the best possible algorithm then in
fact it may take longer. Second, by using the * Mathematica *byte-code
compiler the speed can often be within a factor of three of compiled C code.
So if the C code only runs about 3 times as fast, but takes 20 as much time to
program and debug as *Mathematica* code, there is a strong incentive to use
*Mathematica* at least in many situations. The factor 20 was not made
up, but is based on a study done by IBM comparing productivity of Cobol
programmers with APL programmers and I believe this would be similar to
comparing * Mathematica* with C or C++, at least in many situations. A byte-code compiler is presently
being developed for R as well. MATLAB already has a byte-code compiler.

The test problem was chosen in order to demonstrate the compiled C or Fortran code does always result in significantly improved timings. Consider the problem of inverting the covariance matrix of a first-order autoregressive process with dimension n=1000 and correlation parameter 0.5 and unit innovation variance. The (i,j) entry of this matrix is 0.5^abs(i-j).

The notebook MatrixInverseTiming.nb
contains the compiled and un-compiled version of the test program. The
compiler used in * Mathematica* is an internal byte-code compiler which optimizes
the computation but does not produce native binary code. In the test
example chosen the compiler only produces about a 30% reduction is cpu time but
in other situations a much more dramatic reduction can often be possible.
For example, in the article by McLeod
and Quenneville (2000, Mean likelihood estimators. * Statistics and Computing* 11, 57-65),
it was shown that the speed exact computation of the MA(1) likelihood for
samples of size n=50 could be improved by a factor of more than 50 by using the *Mathematica*
compiler.

To get the timing under MATLAB just put the script file, test.m,
on the MATLAB path and run the commands,

profile on

test(0.5, 1000)

profile report

Just save the * R* file MatrixInverseTime.R
and to run simply input it to *R* with the *R* source function.

The Fortran source code is in test0.for.
An MS Windows executable that was built with Compaq Fortran 6.0 using the
compiler settings for No debug and Full Optimizations is in test0.exe
. This program can also be compiled and run on linux with f77. We
also experimented with using the LAPACK library available on linux. The
program test1.f uses a
general matrix inverse routine while test2.f
uses a specialized subroutine for symmetric positive definite
matrices. The linux PC executables are also available as test1
and test2.

Windows source code comprises two files: test.cpp and test.h. The MS Windows console executable test.exe was built using Visual Studio 6.0 with compiler settings for No Debug and Maximum Optimizations.

A similar program was created for linux and compiled with g++, test. The source code for this program is in test.C and nrutil.h

For S-Plus windows, source the file MatInvTime.S and for S-Plus Linux, MatInvTimeUnix.S

We present the timings on various MS Windows and Linux PCs.

PC with a 1 GHz Xeon processor running Red Hat linux with 4 GB RAM.

Program |
CPU Time in Seconds |

Mathematica, uncompiled | 47 |

Mathematica, compiled | 36 |

R V. 1.3 | 55 |

C++ | 31 |

Fortran | 31^{(a)} 46^{(b) }29^{(c)
}9^{(d)} |

(a) compiling with f77 -fomit-frame-pointer -O3 -funroll-loops
test.f

(b) compiling with f77 -O3 test.f

(c) using LAPACK general matrix inverse

(d) using LAPACK symmetric matrix inverse

PC with a 800 MHz processor running Red Hat linux with 1 GB RAM.

Program |
CPU Time in Seconds |

Mathematica, uncompiled | 100 |

Mathematica, compiled | 71 |

R V. 1.4.1 | 135 |

S-Plus, V.6 Linux | 253 |

C++ | 76 |

Fortran | 67(a) 24(b) |

(a) using LAPACK general matrix inverse

(b) using LAPACK symmetric matrix inverse

PC with a 400 MHz Pentium II processor running WinNT with 256 MB RAM..

Program |
CPU Time in Seconds |

Mathematica, uncompiled | 127 |

Mathematica, compiled | 99 |

R | 77 |

MatLab V.5.3 R11 | 48 |

S-Plus Version 5.2 | 109 |

C++ | 98 |

Fortran | 79 |

*Remark: *The MatLab timing seems amazing. On a slightly more powerful
system, in the table following, the MatLab timing is approximately the
same as for *Mathematica*.

PC with a 732 MHz Pentium III processor running Win2K with 256 MB RAM.

Program |
CPU Time in Seconds |

Mathematica, uncompiled | 51 |

Mathematica, compiled | 36 |

MATLAB V5.3 R11 | 34 |

R | 47 |

S-Plus Version 5.2 | 66 |

C++ | 66 |

Fortran | 54 |

PC with 2 Xeon CPUS at 2 GHZ with 2 Gig RAM

Program |
CPU Time in Seconds |

Mathematica, uncompiled | 21.9 |

Mathematica, compiled | 15.8 |

MATLAB V5.3 R11 | |

R | 99.9 |

S-Plus Version 6.1 | 97.2 |

C++ | 12 |

Fortran | 10.1 |

Last update: Wednesday, June 26, 2002 12:18:31 PM