Opened 9 years ago

Last modified 4 months ago

#1853 new bug

hpc mix files for Main modules overwrite each other

Reported by: guest Owned by:
Priority: lowest Milestone:
Component: Code Coverage Version: 6.8.1
Keywords: Cc: chris@…, andygill@…, kostmo
Operating System: Linux Architecture: x86_64 (amd64)
Type of failure: None/Unknown Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s):
Wiki Page:

Description

I have several programs, and hence several files that define Main modules, in the same directory. I build each one with a ghc --make -o Progname. When The hpc mix files describing the compiled modules are dumped in .hpc, the current Main.mix overwrites any previous Main.mix. As a result, I can only get an hpc report from Progname.tix if Progname was the most recent binary to be compiled.

Change History (28)

comment:1 Changed 9 years ago by igloo

  • difficulty set to Unknown
  • Milestone set to 6.10 branch

comment:2 Changed 9 years ago by AndyGill

  • Owner changed from andy@… to AndyGill
  • Status changed from new to assigned

comment:3 Changed 7 years ago by igloo

  • Milestone changed from 6.10 branch to 6.12 branch

comment:4 Changed 6 years ago by igloo

  • Milestone changed from 6.12 branch to 6.12.3

comment:5 Changed 6 years ago by igloo

  • Milestone changed from 6.12.3 to 6.14.1
  • Priority changed from normal to low

comment:6 Changed 6 years ago by igloo

  • Milestone changed from 7.0.1 to 7.0.2

comment:7 Changed 6 years ago by igloo

  • Milestone changed from 7.0.2 to 7.2.1

comment:8 Changed 5 years ago by igloo

  • Milestone changed from 7.2.1 to 7.4.1

comment:9 Changed 5 years ago by igloo

  • Milestone changed from 7.4.1 to 7.6.1
  • Priority changed from low to lowest

comment:10 Changed 4 years ago by igloo

  • Milestone changed from 7.6.1 to 7.6.2

comment:11 Changed 4 years ago by morabbin

  • Type of failure set to None/Unknown

Bump; was this looked at? Does it still happen?

comment:12 follow-up: Changed 4 years ago by andygill

This one is tricky to fix, because of a baked in assumption. If anyone wants to help, or if I can find a willing student, I can give guidance about how to fix it.

comment:13 Changed 2 years ago by thoughtpolice

  • Milestone changed from 7.6.2 to 7.10.1

Moving to 7.10.1.

comment:14 Changed 21 months ago by thoughtpolice

  • Milestone changed from 7.10.1 to 7.12.1

Moving to 7.12.1 milestone; if you feel this is an error and should be addressed sooner, please move it back to the 7.10.1 milestone.

comment:15 Changed 21 months ago by thoughtpolice

Moving to 7.12.1 milestone; if you feel this is an error and should be addressed sooner, please move it back to the 7.10.1 milestone.

comment:16 in reply to: ↑ 12 Changed 19 months ago by thomie

Replying to andygill:

This one is tricky to fix, because of a baked in assumption. If anyone wants to help, or if I can find a willing student, I can give guidance about how to fix it.

@andygill. I am currently looking at hpc.

Do you think this issue is worth fixing? Multiple programs in the same directory can be nicely handled using -main-is. The following doesn't overwrite any .mix files:

$ cat ProgramA.hs
module ProgramA where
main = return ()

$ cat ProgramB.hs
module ProgramB where
main = return ()

$ ghc -fhpc -main-is ProgramA ProgramA
$ ghc -fhpc -main-is ProgramB ProgramB

$ ls .hpc
ProgramA.mix ProgramB.mix

comment:17 Changed 13 months ago by thoughtpolice

  • Milestone changed from 7.12.1 to 8.0.1

Milestone renamed

comment:18 follow-up: Changed 12 months ago by mgsloan

This is also troublesome for using hpc combine --union to merge tix files which come from different programs. My current hack is to preprocess the tix files and remove all of the executable modules, like this: https://gist.github.com/mgsloan/e44e8775516c50f10edf

I think the ideal resolution would be to prefix all of the executable modules with some better identification. How about having a new ghc flag like --hpc-package-prefix prefix, where the conventional value for prefix is the cabal package:component? Then, the tix files will use this prefix on module names, and the mix files will get outputted to a subdirectory instead of the root of the hpcdir.

This convention gets a little weird due to the usage of : when passing --include and --exclude to the hpc program (see https://downloads.haskell.org/~ghc/latest/docs/html/users_guide/hpc.html). However, with care, : should be a valid thing to use in PACKAGE.

comment:19 Changed 12 months ago by thomie

  • Owner AndyGill deleted

comment:20 Changed 12 months ago by thomie

  • Cc andygill@… added

@AndyGill: any comments on the proposal from comment:18?

@mgsloan: I'm still curious what Andy (the creator of hpc) had in mind in comment:12, but if your proposal solves the problem, and you don't hear from Andy, please go ahead. Sorry for not giving real feedback. I don't really know much about hpc at all, and I don't think anyone else does either (*cough* technical debt *cough*), but I trust you to do the right thing and will make sure your patches get reviewed/merged.

comment:21 in reply to: ↑ 18 Changed 12 months ago by andygill

Replying to mgsloan:

I would support this change, or in general, adding some type of way of designating the location and/or naming of the .tix file.

comment:22 Changed 12 months ago by andygill

I've not touched hpc for years, but am happy to try help. Perhaps we could work on a concrete failure, and decide where to go from there? Can we create a small github repo with the failures, for example. The solution would need to integrate with cabal, for example.

comment:23 Changed 11 months ago by mgsloan

Hi, thanks for the response Andy!

Handily, I already had a minimal repro for this from back when I initially started working on coverage support in stack: https://github.com/mgsloan/multi-test-suite

In particular:

mgsloan@computer:~/fpco/multi-test-suite$ cabal test --ghc-options -fhpc
# ...mgsloan@computer:~/fpco/multi-test-suite$ hpc report multi-test-suite-test.tix
hpc: can not find Main in ["./.hpc"]
mgsloan@computer:~/fpco/multi-test-suite$ hpc report multi-test-suite-test-2.tix
 50% expressions used (1/2)
100% boolean coverage (0/0)
     100% guards (0/0)
     100% 'if' conditions (0/0)
     100% qualifiers (0/0)
100% alternatives used (0/0)
100% local declarations used (0/0)
100% top-level declarations used (1/1)

This issue will also affect cabal files that have other stanzas like executables and benchmarks. I added a folder called sub-package to multi-test-repo, actually debug / test out a different stack coverage related issue.

The only reasonable way I can think of to fix this is to add a flag to GHC, which takes the name to use for the mix file folder. This won't directly fix the problem with current Cabal, but it would allow future version of Cabal to pass the info so that this issue is resolved.

I'm not sure what to call the flag - it doesn't necessarily need to be specific to HPC - since there might be other places in GHC that benefit from knowing the cabal component. So, I'm thinking either -component-name, or -hpc-component-name.

I took a look at Cabal's current invocations of GHC to see if there is any way for GHC to know that it's building a particular executable component. Unfortunately, I don't see any arguments that would make a good source for this info:

/usr/local/bin/ghc --make -no-link -fbuilding-cabal-package -O -j8 -static -outputdir dist/build/multi-test-suite-test/multi-test-suite-test-tmp -odir dist/build/multi-test-suite-test/multi-test-suite-test-tmp -hidir dist/build/multi-test-suite-test/multi-test-suite-test-tmp -stubdir dist/build/multi-test-suite-test/multi-test-suite-test-tmp -i -idist/build/multi-test-suite-test/multi-test-suite-test-tmp -itest -idist/build/autogen -Idist/build/autogen -Idist/build/multi-test-suite-test/multi-test-suite-test-tmp -optP-include -optPdist/build/autogen/cabal_macros.h -hide-all-packages -package-db dist/package.conf.inplace -package-id base-4.8.1.0-4f7206fd964c629946bb89db72c80011 -package-id multi-test-suite-0.1.0.0-inplace -XHaskell2010 test/Spec.hs -threaded -rtsopts '-with-rtsopts=-N' -fhpc

(Sorry, the naming isn't all that clear, multi-test-suite-test is the name of the component)

comment:24 Changed 9 months ago by ezyang

Hello all, I was recently looking at the Cabal code which is responsible for setting hpcdir, and I am a little confused by this ticket: why is it NOT sufficient to just specify different hpc directories for each main module? Does this have to do with wanting to combine tix files together?

comment:25 Changed 9 months ago by mgsloan

There's a pretty good reason not to vary --hpcdir. You can only pass in one --hpcdir to the hpc program, and it's treated as a relative path to each of the --srdirs. So, if you want to generate a report which spans multiple packages, --hpcdir needs to be kept constant.

We might consider improving the hpc program's CLI. This particular issue is discussed further in https://ghc.haskell.org/trac/ghc/ticket/10951

The tricky thing about combining tix files together is that module names within the tix file are not sufficiently disambiguated. For packages, they are prefixed by their package key / id. So, you expect to find the mix file for TixModule "cycli_6S2W1a0WjTe91AyXmPMq2j/Cyclic" ... at some HPCDIR/cycli_6S2W1a0WjTe91AyXmPMq2j/Cyclic.mix. However, for modules from the non-library components, there is no folder. Instead it's just TixModule "Main" ...

comment:26 Changed 9 months ago by ezyang

So, in some sense, you are actually completely screwed, because at runtime the executable producing Tix files has no idea if it was compiled from Main1.hs or Main2.hs: the symbol names for the two executables live in the same namespace, etc.

There is actually a very cunning way this problem could be fixed:

  1. -main-is could be extended to work with modules which are not in the main package.
  2. Executables could then be compiled while setting -package-name to something that is not main.

Then the symbol names would be different and you could merge them reliably.

The alternative is that HPC can be told explicitly that a mix file is for a specific file name (extending it's model to not be profiling a pile of Modules, but a pile of Modules or files).

comment:27 Changed 8 months ago by thomie

  • Milestone 8.0.1 deleted

comment:28 Changed 4 months ago by thomie

  • Cc kostmo added

Perhaps relevant to the discussion here, @kostmo points out in #12065:

specifying HPCTIXFILE as an environment variable when running the executable program does change the output path of the .tix file.

Note: See TracTickets for help on using tickets.