A Tour of Morfa

Benchmarks

A subset of benchmarks (taken from the The Computer Language Benchmarks Game ) has been used to benchmark performance of Morfa programs. The benchmarking process followed the idea of The Computer Language Benchmarks Game and same caution at jumping to conclusions should be exercised with respect to the following results.

name Data csharp gcc java julia morfa python
binarytrees cpu(s) 10.51 14.697 6.353 192.835 333.169 224.467
mem(KB) 192628 131628 472800 1021548 145392 835820
size(B) 733 894 688 638 991 724
chameneosredux cpu(s) 3.184 151.862
elapsed(s) 1.563 56.937
mem(KB) 31788 4392
size(B) 1967 2783
fasta cpu(s) 5.093 1.618 2.645 31.334 7.088 51.634
mem(KB) 10888 356 26656 322368 4940 5780
size(B) 1350 2443 1679 889 1376 1647
knucleotide cpu(s) 53.253 12.064 15.047 232.284 148.12 134.22
mem(KB) 541692 190688 179764 1732276 756124 161532
size(B) 1516 2724 1955 874 1806 2011
nbody cpu(s) 10.239 4.728 5.193 8.242 12.792 460.513
mem(KB) 13064 428 25652 61188 4184 5828
size(B) 1471 1336 1540 1329 1600 1334
spectralnorm cpu(s) 8.933 8.548 5.23 11.381 4.206 376.444
mem(KB) 10944 428 25684 56640 4960 6640
size(B) 597 473 776 540 781 498

Judging based on these particular results, it seems that for the majority benchmarks Morfa delivers performance comparable with Mono C# and sometimes C/Java level in terms of CPU usage. In terms of memory usage, Morfa performs very well except for the gc-intensive knucleotide benchmark.

The code

Code from morfa-examples repository was used, see benchmarks/bencher/programs dir. Therein you can check the versions of codes from The Computer Language Benchmarks Game which were used to compare.

Notes

  • the above results may not be 100% fair for all languages, because their benchmark implementations represent a fairly uneven level of perfection. Treat them as a coarse approximation of the order of magnitue of performance metrics.
  • performance of fasta and nbody should be further improved after an issue with for loops is solved. It is estimated, that having done this the performance of these two benchmarks should level with C.
  • performance in binarytrees is weak due to heavy use of GC, a known weakness to be improved in the future.

Tests were done using:

  • Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
  • 16.0 GM RAM DDR3 @ 800MHz
  • Windows 8.1 64bit running a Ubuntu 14.04 LTS guest system on VirtualBox virtual machine
  • Mono C# compler / Mono JIT compiler version 3.2.8 (Debian 3.2.8+dfsg-4ubuntu1.1)
  • gcc (Ubuntu/Linaro 4.7.3-12ubuntu1) 4.7.3
  • Java: -- java version "1.7.0_75" -- OpenJDK Runtime Environment (IcedTea 2.5.4) (7u75-2.5.4-1~trusty1) -- OpenJDK 64-Bit Server VM (build 24.75-b04, mixed mode)
  • Morfa revision c7a9682f5edd
  • Python 3.4.0
  • Julia Version 0.3.10, Generic Linux Binaries with system image built to sys.so
  • bencher script of The Computer Language Benchmarks Game to perform measurements as well as N (task size) and general approach taken from there