The purpose of these timings is to demonstrate that in certain circumstances the widely held view that you can always dramatically improve on the cpu time required for lengthy computations by using compiled C or Fortran code instead of advanced QPEs such as Mathematica is wrong.
It is frequently stated that interpretative languages are not as useful as compiled languages such as C and Fortran for numerically intensive computations. It is obviously the case that an well written C program can be expected to outperform Mathematica, R, S-Plus or MATLAB but there are two caveats that need to be understood. First, if the C program is not efficiently programmed using the best possible algorithm then in fact it may take longer. Second, by using the Mathematica byte-code compiler the speed can often be within a factor of three of compiled C code. So if the C code only runs about 3 times as fast, but takes 20 as much time to program and debug as Mathematica code, there is a strong incentive to use Mathematica at least in many situations. The factor 20 was not made up, but is based on a study done by IBM comparing productivity of Cobol programmers with APL programmers and I believe this would be similar to comparing Mathematica with C or C++, at least in many situations. A byte-code compiler is presently being developed for R as well. MATLAB already has a byte-code compiler.
The test problem was chosen in order to demonstrate the compiled C or Fortran code does always result in significantly improved timings. Consider the problem of inverting the covariance matrix of a first-order autoregressive process with dimension n=1000 and correlation parameter 0.5 and unit innovation variance. The (i,j) entry of this matrix is 0.5^abs(i-j).
The notebook MatrixInverseTiming.nb contains the compiled and un-compiled version of the test program. The compiler used in Mathematica is an internal byte-code compiler which optimizes the computation but does not produce native binary code. In the test example chosen the compiler only produces about a 30% reduction is cpu time but in other situations a much more dramatic reduction can often be possible. For example, in the article by McLeod and Quenneville (2000, Mean likelihood estimators. Statistics and Computing 11, 57-65), it was shown that the speed exact computation of the MA(1) likelihood for samples of size n=50 could be improved by a factor of more than 50 by using the Mathematica compiler.
To get the timing under MATLAB just put the script file, test.m,
on the MATLAB path and run the commands,
profile on
test(0.5, 1000)
profile report
Just save the R file MatrixInverseTime.R and to run simply input it to R with the R source function.
The Fortran source code is in test0.for.
An MS Windows executable that was built with Compaq Fortran 6.0 using the
compiler settings for No debug and Full Optimizations is in test0.exe
. This program can also be compiled and run on linux with f77. We
also experimented with using the LAPACK library available on linux. The
program test1.f uses a
general matrix inverse routine while test2.f
uses a specialized subroutine for symmetric positive definite
matrices. The linux PC executables are also available as test1
and test2.
Windows source code comprises two files: test.cpp and test.h. The MS Windows console executable test.exe was built using Visual Studio 6.0 with compiler settings for No Debug and Maximum Optimizations.
A similar program was created for linux and compiled with g++, test. The source code for this program is in test.C and nrutil.h
For S-Plus windows, source the file MatInvTime.S and for S-Plus Linux, MatInvTimeUnix.S
We present the timings on various MS Windows and Linux PCs.
PC with a 1 GHz Xeon processor running Red Hat linux with 4 GB RAM.
| Program | CPU Time in Seconds |
| Mathematica, uncompiled | 47 |
| Mathematica, compiled | 36 |
| R V. 1.3 | 55 |
| C++ | 31 |
| Fortran | 31(a) 46(b) 29(c) 9(d) |
(a) compiling with f77 -fomit-frame-pointer -O3 -funroll-loops
test.f
(b) compiling with f77 -O3 test.f
(c) using LAPACK general matrix inverse
(d) using LAPACK symmetric matrix inverse
PC with a 800 MHz processor running Red Hat linux with 1 GB RAM.
| Program | CPU Time in Seconds |
| Mathematica, uncompiled | 100 |
| Mathematica, compiled | 71 |
| R V. 1.4.1 | 135 |
| S-Plus, V.6 Linux | 253 |
| C++ | 76 |
| Fortran | 67(a) 24(b) |
(a) using LAPACK general matrix inverse
(b) using LAPACK symmetric matrix inverse
PC with a 400 MHz Pentium II processor running WinNT with 256 MB RAM..
| Program | CPU Time in Seconds |
| Mathematica, uncompiled | 127 |
| Mathematica, compiled | 99 |
| R | 77 |
| MatLab V.5.3 R11 | 48 |
| S-Plus Version 5.2 | 109 |
| C++ | 98 |
| Fortran | 79 |
Remark: The MatLab timing seems amazing. On a slightly more powerful system, in the table following, the MatLab timing is approximately the same as for Mathematica.
PC with a 732 MHz Pentium III processor running Win2K with 256 MB RAM.
| Program | CPU Time in Seconds |
| Mathematica, uncompiled | 51 |
| Mathematica, compiled | 36 |
| MATLAB V5.3 R11 | 34 |
| R | 47 |
| S-Plus Version 5.2 | 66 |
| C++ | 66 |
| Fortran | 54 |
PC with 2 Xeon CPUS at 2 GHZ with 2 Gig RAM
| Program | CPU Time in Seconds |
| Mathematica, uncompiled | 21.9 |
| Mathematica, compiled | 15.8 |
| MATLAB V5.3 R11 | |
| R | 99.9 |
| S-Plus Version 6.1 | 97.2 |
| C++ | 12 |
| Fortran | 10.1 |
Last update: Wednesday, June 26, 2002 12:18:31 PM