Use `+RTS -G1` for more stable residency measurements
Tests that measure peak_megabytes_allocated
and max_bytes_used
can be very sensitive to small changes in, well, just about anything: order of command line flags, what should be irrelevant compiler patches, minor changes in the test file to compile.
From Note [residency]
in testsuite/tests/perf/compiler/all.T
:
# Residency (peak_megabytes_allocated and max_bytes_used) is sensitive
# to when the major GC runs, which makes it inherently inaccurate.
# Sometime an innocuous change somewhere can shift things around such
# that the samples occur at a different time, and the residency
# appears to change (up or down) when the underlying profile hasn't
# really changed.
I propose setting +RTS -G1
for these tests. This seems to make them more robust to small changes, as can be seen in the following two graphs, adapted from the test for #9675:
https://ghc.haskell.org/trac/ghc/attachment/ticket/9675/T9675-peak_megabytes_allocated.png https://ghc.haskell.org/trac/ghc/attachment/ticket/9675/T9675-max_bytes_used.png
This shows the peak_megabytes_allocated
and max_bytes_used
as a function of the number fields when compiling a single record:
data Foo = Foo
{ field1 :: Int -> Int
, field2 :: Int -> Int
...
The default GHC -O
line is wiggling up and down, while the GHC -O +RTS -G1
line is nice and smooth, which is what we want.
I don't know if making this change defeats the purpose of some tests, which is why I'm opening this ticket.
Trac metadata
Trac field | Value |
---|---|
Version | 7.10.1 |
Type | Task |
TypeOfFailure | OtherFailure |
Priority | normal |
Resolution | Unresolved |
Component | Test Suite |
Test case | |
Differential revisions | |
BlockedBy | |
Related | |
Blocking | |
CC | ezyang, thoughtpolice |
Operating system | |
Architecture |