Testsuite: compile_timeout_multiplier is fragile
I'm taking a sweep through the testsuite, trying to eliminate failures in a DEBUG
compiler. But two tests are stymieing me: pmcheck/should_compile/T11{276,374}
. Both of these use the compile_timout_multiplier
option in their all.T
entries. This option means that, instead of the normal 300s allotted to the test, these tests get only 3s. In my setup, with a DEBUG
(read: unoptimized) stage2 compiler, these tests fail when I'm running the testsuite in parallel. Then, when I run the tests individually, they succeed.
This poses two problems:
- I can only see the failure when my machine is stressed, and it's disconcerting to have a failure appear and disappear depending on how I look.
- There is (sometimes) a failure in
DEBUG
mode.
What is the goal of compile_timeout_multiplier
? If we're really testing performance (which #11276 (closed) and #11374 (closed) indicate), then these tests should be in perf/compiler
and be "stat" tests, where some wibbling in the numbers is expected (and where DEBUG
compilers are exempted). I would imagine that compile_timeout_multiplier
should only ever get values greater than 1, when a non-performance test is so expensive to run that it takes more than 300s.
As a local solution here, I think these two tests should move to perf/compiler
. Other pattern-match tests also have compile_timeout_multiplier
, too, and I can move these as well. Any objections?
Trac metadata
Trac field | Value |
---|---|
Version | 8.4.3 |
Type | Bug |
TypeOfFailure | OtherFailure |
Priority | normal |
Resolution | Unresolved |
Component | Compiler |
Test case | |
Differential revisions | |
BlockedBy | |
Related | |
Blocking | |
CC | |
Operating system | |
Architecture |