wiki:Nofib

Version 2 (modified by batterseapower, 4 years ago) (diff)

--

Nofib is the standard Haskell benchmark suite. It is described in the paper The nofib Benchmark Suite of Haskell Programs.

Getting nofib

From your GHC tree, run:

./darc-all --nofib get

It will be pulled into the a "nofib" subdirectory.

Benchmarking an optimisation

To test your whizzy new optimisation with nofib, you will want to:

  1. Build the inplace stage2 compiler with a flag that controls whether it runs or not
  2. Run the nofib suite with that flag on and then off, and compare the two runs:
cd nofib
make clean && make boot && make -k >& log1
make clean && make boot && make -k EXTRA_HC_OPTS=-fenable-cool-optimisation >& log2
nofib-analyse/nofib-analyse log1 log2

The output of the nofib-analyse tool is quite readable, with two provisios:

  • Missing values in the output typically mean that the benchmark crashed and may indicate a problem with your optimisation
  • If a difference between the two modes is displayed as an absolute quantity instead of a percentage, it means that the difference was below the threshold at which the analyser considers it significant

If the comparison identifies any particularly bad benchmark results, you can run them individually by changing into their directory and running something like:

EXTRA_HC_OPTS="-fenable-cool-optimisation -ddump-simpl" make

You can add whatever dumping flags you need to see the output and understand what is going wrong.