Opened 2 months ago

Last modified 5 weeks ago

#16042 new bug

T13031 doesn't get run

Reported by: bgamari Owned by:
Priority: normal Milestone:
Component: Compiler Version: 8.6.3
Keywords: Cc: alpmestan
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: None/Unknown Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s):
Wiki Page:

Description

Simon wrote noting that two tests in stranal/should_compile didn't appear to be run in his testing. I have been able to reproduce this.

A few observations:

  • There is a setTestOpts( only_ways(['optasm']) ) at the top of stranal/should_compile/test.T
  • in a validate build T13031 is skipped when run with:
    • make test TEST=T13031 WAY=normal
    • make test TEST=T13031 WAY=optasm
    • make slowtest TEST=T13031 WAY=normal
    • make slowtest TEST=T13031 WAY=optasm
  • When the setTestOpts call is commented out T13031 is
    • skipped with make test TEST=T13031 WAY=optasm
    • run with make test TEST=T13031 WAY=normal
  • optasm is in the config.run_ways list

I suspect what is happening here is that the optasm way is not being run

To be continued...

Change History (3)

comment:1 Changed 2 months ago by bgamari

I see the problem. In test_common_work we have,

def test_common_work(watcher, name, opts, func, args):
    try:
        t.total_tests += 1
        setLocalTestOpts(opts)

        package_conf_cache_file_start_timestamp = get_package_cache_timestamp()

        # All the ways we might run this test
        if func == compile or func == multimod_compile:
            all_ways = config.compile_ways
        elif func == compile_and_run or func == multimod_compile_and_run:
            all_ways = config.run_ways
        elif func == ghci_script:
            if 'ghci' in config.run_ways:
                all_ways = ['ghci']
            else:
                all_ways = []
        else:
            all_ways = ['normal']

    ...

and we then look at all_ways to determine whether the test is supposed to run.

When I call make slowtest TEST=T13031 WAY=optasm I find that all_ways is ['normal'] yet getTestOpts().only_ways is ['optasm']. Consequently I suspect the problem here is that T13031 is hits the else case of the if, being a run_command test.

The simplest fix for this would be to simply drop the only_way modifier on the test. However, I worry that issue this will inevitably recur elsewhere. I'm going to sleep on it to see what we can do to improve this situation.

comment:2 Changed 5 weeks ago by osa1

There's another problem in test_common_work: if I define a run_command test with only_ways(['optasm']) my test will never run, because if func == ... always sets all_ways = ['normal'] when func is run_command.

I wonder if we have any tests that never run because of this..

comment:3 Changed 5 weeks ago by alpmestan

Cc: alpmestan added
Note: See TracTickets for help on using tickets.