Opened 3 years ago

Closed 3 years ago

#9619 closed bug (fixed)

HPC Code Coverage complains when two exactly the same mix files are on the path

Reported by: Kasper Owned by:
Priority: lowest Milestone: 7.10.1
Component: Code Coverage Version: 7.8.3
Keywords: Cc:
Operating System: Linux Architecture: x86_64 (amd64)
Type of failure: Incorrect result at runtime Test Case: libraries/hpc/tests/simple/tixs/T9619
Blocked By: Blocking:
Related Tickets: Differential Rev(s): Phab:D704
Wiki Page:

Description

Cabal's --enable-library-coverage flag generates the .mix files in the dist/mix directory that cabal creates during compilation. I have two test suites, integration tests and unit tests. Those test suites calll tests for moduleA in moduleATest. Unfortunately, in a first version, the integration tests and unit tests for moduleA were both present in moduleATest. Therefore, the unit test suite (in UnitTests.hs) and the integration test suite (in IntegrationTests.hs) both 'use' the moduleATest by linking in tests from that module. This leads to cabal generating a directory integration-tests and a directory unit-tests with .mix files, but the moduleATest.mix file is present in both directories. When performing hpc sum and hpc markup to get the total result of the test runs I need to specify the directories where the mix files are present (so dist/hpc/mix/unit-tests and dist/hpc/mix/integration-tests) and it will then complain that it finds moduleATest.mix file twice.

This issue is not present when using the -fhpc flag, as the directory structure with integration-tests and unit-tests is not present in the .hpc directory at that moment. That being said, I don't think it's a cabal issue as it's more or less expected what they're doing, generating the mix files necessary to instrument integration-tests and unit-tests separately.

Priority put to lowest because I can work around it by diving moduleATest up in moduleATest and moduleAIntegrationTest, which is saner in the end anyway. But I guess somebody will come up with a use case where a split up isn't possible, in the future.

Change History (9)

comment:1 Changed 3 years ago by thomie

Thank you for the report. Could you add attach the files needed to reproduce the problem, with instructions on how to run them, and the current and expected output.

Please note that hpc hasn't seen any updates in a long time, so it might take a while for someone to take a look.

comment:2 Changed 3 years ago by thomie

Status: newinfoneeded

@Kasper: I need your help here. I can't reproduce your problem.

Here's what I've tried. Source code:

cat > ./T9619.cabal <<EOF
name:                T9619
version:             0.1.0.0
build-type:          Simple
cabal-version:       >=1.2

executable T9619
    main-is: T9619.hs
    build-depends: base

test-suite IntegrationTests
    type: exitcode-stdio-1.0
    main-is: IntegrationTests.hs
    ghc-options: -main-is IntegrationTests
    other-modules: ModuleATest
    build-depends: base

test-suite UnitTests
    type: exitcode-stdio-1.0
    main-is: UnitTests.hs
    ghc-options: -main-is UnitTests
    other-modules: ModuleATest
    build-depends: base
EOF

cat > ./Setup.hs <<EOF
import Distribution.Simple
main = defaultMain
EOF

cat > ./T9619.hs <<EOF
main = return ()
EOF

cat > ./IntegrationTests.hs <<EOF
module IntegrationTests(main) where
import ModuleATest
main :: IO ()
main = print moduleATest
EOF

cat > ./UnitTests.hs <<EOF
module UnitTests(main) where
import ModuleATest
main :: IO ()
main = print moduleATest
EOF

cat > ./ModuleATest.hs <<EOF
module ModuleATest where
moduleATest = "hello"
EOF
$ cabal configure --enable-tests --enable-coverage
$ cabal test

$ find dist/hpc/vanilla/mix/
dist/hpc/vanilla/mix/
dist/hpc/vanilla/mix/IntegrationTests
dist/hpc/vanilla/mix/IntegrationTests/ModuleATest.mix
dist/hpc/vanilla/mix/IntegrationTests/IntegrationTests.mix
dist/hpc/vanilla/mix/UnitTests
dist/hpc/vanilla/mix/UnitTests/ModuleATest.mix
dist/hpc/vanilla/mix/UnitTests/UnitTests.mix

$ hpc sum dist/hpc/vanilla/tix/IntegrationTests/IntegrationTests.tix dist/hpc/vanilla/tix/UnitTests/UnitTests.tix --union
Tix [TixModule "IntegrationTests" 1693426709 3 [1,1,1],TixModule "ModuleATest" 897400108 2 [2,2],TixModule "UnitTests" 1821344755 3 [1,1,1]]

$ cabal --version
cabal-install version 1.23.0.0
using version 1.23.0.0 of the Cabal library

$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 7.8.4

It all seems to work to me. The ModuleATest.mix file is indeed generated twice, but it doesn't seem to bother hpc sum.

You mention "When performing hpc sum and hpc markup to get the total result of the test runs I need to specify the directories where the mix files are present". What is the command you use to run hpc sum. My understanding is that hpc sum only takes .tix files, not .mix files.

I must be missing something, so please clarify.

Last edited 3 years ago by thomie (previous) (diff)

comment:3 Changed 3 years ago by thomie

Ok, it's not hpc sum that fails, only hpc markup:

$ hpc markup dist/hpc/vanilla/tix/IntegrationTests/IntegrationTests.tix --hpcdir=dist/hpc/vanilla/mix/IntegrationTests/ --hpcdir=dist/hpc/vanilla/mix/UnitTests/
hpc: found 2 instances of ModuleATest in ["./.hpc","./dist/hpc/vanilla/mix/UnitTests/","./dist/hpc/vanilla/mix/IntegrationTests/"]

comment:4 Changed 3 years ago by Kasper

Ok, sorry, bad bug report, I did indeed do a combination of hcp markup and hpc sum. I moved away from this approach right now, so I don't really have the problem anymore, clarification might have been very difficult.

comment:5 Changed 3 years ago by thomie

Differential Rev(s): Phab:D704
Milestone: 7.12.1
Status: infoneededpatch

A potential fix can be found here (pending validation): https://github.com/ghc/packages-hpc/tree/wip/T9619

comment:6 Changed 3 years ago by Thomas Miedema <thomasmiedema@…>

In 41e8400a57620978681663e9c804fee405da26d5/ghc:

Update submodule hpc (includes fix for #9619)

Reviewers: austin

Differential Revision: https://phabricator.haskell.org/D704

comment:7 Changed 3 years ago by thomie

Resolution: fixed
Status: patchclosed
Test Case: libraries/hpc/tests/simple/tixs/T9619

This will be fixed in 7.12.

The relevant commit in the hpc library: https://github.com/ghc/packages-hpc/commit/f601495ac5f93f24cbcaa95f45b1bc26ad644ac9

comment:8 Changed 3 years ago by hvr

Milestone: 7.12.17.10.1
Status: closedmerge

Since this landed into GHC HEAD a bit too soon, we're gonna have to cherry pick this into 7.10 to avoid git branching in the hpc submodule :-/

comment:9 Changed 3 years ago by hvr

Status: mergeclosed
Note: See TracTickets for help on using tickets.