Test suite should show reason for skipping a test
I want to add a GHCi test, but it's a bit tricky to run the test so I use a
make rule and use run_command
, like this:
test('UnsafeReenterGhci',
[only_ways(['ghci']), exit_code(1), extra_files(['UnsafeReenter.hs', 'UnsafeReenterC.c'])],
run_command,
['$MAKE -s --no-print-directory UnsafeReenterGhci'])
The important part is run_command
and only_ways(['ghci'])
. The problem is
that it's impossible to run this test, because by default a run_command
test
can only be run in normal way (see test_common_work() in testlib.py), but
there's no way to know this without reading the source code.
Having better documentation might help, but I think the test driver should
print why a test is skipped (maybe only in verbose mode -- if such a mode
exists). In our case it should print something like "Test is only run in
'normal' way" (because run_command
tests only run in 'normal' way).
Correct version of this test is:
test('UnsafeReenterGhci',
[extra_ways(['ghci']), only_ways(['ghci']), exit_code(1), extra_files(['UnsafeReenter.hs', 'UnsafeReenterC.c'])],
run_command,
['$MAKE -s --no-print-directory UnsafeReenterGhci'])
The extra_ways
makes the test driver consider running the test in ghci
way,
only_ways
is to avoid running it in normal
way (the default and only way
for run_command
).
Trac metadata
Trac field | Value |
---|---|
Version | 8.5 |
Type | Task |
TypeOfFailure | OtherFailure |
Priority | normal |
Resolution | Unresolved |
Component | Test Suite |
Test case | |
Differential revisions | |
BlockedBy | |
Related | |
Blocking | |
CC | |
Operating system | |
Architecture |