Extend nofib with benchmarks focused on compiler performance
GHC seems to be missing a set of benchmarks focused on the compiler performance that would contain some realistic examples and make it easy to test even small changes in the compiler (by having a simple "before and after" comparison). The two main use cases would be:
- Anyone hacking on GHC could quickly check even for small performance differences (running the benchmark with and without their change).
- Tracking performance of GHC over time (potentially as part of perf.haskell.org)
We already have two collections of performance tests/benchmarks:
-
tests/perf/compiler
which contains actual tests (part ofvalidate
) and seems to be focused on larger regressions (if the bounds are too tight, it can generate a fair amount of noise). -
nofib
which is mostly focused on benchmarking the code produced by GHC (the vast majority of the modules compile within 400ms).
The current idea is to extend nofib
with some modules that are only checking the performance of GHC (e.g., might be a library module that doesn't contain main
).
Please see: https://mail.haskell.org/pipermail/ghc-devs/2016-December/013323.html for an initial discussion about this.
Also related:
- Ticket focused on improvements to
tests/perf/compiler
#12758 (closed) - Proposal of having a similar tests but hosted in a separate repo on github: https://github.com/ghc-proposals/ghc-proposals/pull/26
Trac metadata
Trac field | Value |
---|---|
Version | |
Type | Task |
TypeOfFailure | OtherFailure |
Priority | normal |
Resolution | Unresolved |
Component | NoFib benchmark suite |
Test case | |
Differential revisions | |
BlockedBy | |
Related | |
Blocking | |
CC | |
Operating system | |
Architecture |