Opened 3 years ago

Last modified 22 months ago

#9176 new bug

GHC not generating dyn_hi files

Reported by: heatsink Owned by:
Priority: normal Milestone:
Component: Compiler Version: 7.9
Keywords: dynamic Cc:
Operating System: Unknown/Multiple Architecture: x86_64 (amd64)
Type of failure: Other Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s):
Wiki Page:

Description (last modified by heatsink)


In dynamic-too compilation, a module is compiled the normal way, then the backend runs twice to generate normal (.o, .hi) and dynamic (.dyn_o, .dyn_hi) outputs. Compilation uses the .hi files of imported modules. Effectively, the compiler assumes that all the information it gets from a .hi file, used by the normal output, is also true of the corresponding .dyn_hi file, used by the dynamic output. GHC has a few checks to verify this, but they don’t always produce desirable results.

  • When importing a module from a package in --make mode, GHC checks that the normal and dynamic interfaces match, in checkBuildDynamicToo in findAndReadIface. If they don’t match, GHC silently disables dynamic-too, so that no dynamic interface file is produced. No errors or warnings are reported.
  • When importing a module (either from a package, or standalone) in one-shot mode with -c, GHC starts out the same way as above: it checks the imported interfaces, finds out they don't match, and disables dynamic-too output. After generating output the normal way, the compiler pipeline runs again in the dynamic way to generate the dynamic output. This seems to be the "right" thing to do.
  • When importing a standalone module in --make mode, GHC does not examine the dynamic interface at all. It generates both normal and dynamic output.

Of the two --make cases, GHC uses the weaker checks when building a package and stricter checks when using the installed package. The weaker checks allow a package with mismatched normal and dynamic interface files to build and install without errors. After it's installed, the stronger checks suppress the creation of .dyn_hi files whenever the mismatched module is imported. Thus, a package installs fine, but importing from the package prevents dynamic-too compilation from working properly.


The attached code sets up an inconsistent pair of module interfaces and runs both kinds of imports. To compile the first two files, build the cabal package in testcase9176/. To compile the last file, install the package, then run the makefile in user/.

  • Imported.hs is compiled with different optimization flags, setting up a situation where the normal and dyamic module interfaces do not match.
  • User.hs importing directly Imported.hs loads only the static module interface. The dynamic interface is ignored.
  • User.hs importing from the installed package Imported.hs compares the static and dynamic interfaces, turns off dynamic-too compilation, and does not generate a .dyn_hi file. No errors or warnings are produced. This file is a copy of the other User.hs, just compiled differently.


GHC with -dynamic-too is expected to generate a .dyn_hi file, but it does not when compiling the third file.

When compiling a module, GHC should read the interfaces of any modules that it imports. However, when compiling the second file, GHC does not read the .dyn_hi file describing the imported dynamic module, only the .hi file describing the normal module.

The compiler does not treat mismatched module interfaces consistently in all situations, making it hard to find the root cause of errors.

I'm not sure what the correct behavior should be.


This bug appeared on my system in a Parsec module. Parsec installed with no apparent problems, but other modules importing from Parsec would not compile to .dyn_hi files.

GHC was built from source, commit 9e10963e394680dbb1b964c66cb428a2aa03df09, compiled by GHC 7.6.3 with XCode 5.1.1 on OS X 10.9.3.

Attachments (1)

testcase9176-root.tar.gz (2.2 KB) - added by heatsink 3 years ago.

Download all attachments as: .zip

Change History (12)

comment:1 Changed 3 years ago by heatsink

Description: modified (diff)

comment:2 Changed 3 years ago by heatsink

Description: modified (diff)

comment:3 Changed 3 years ago by heatsink

The dyn_hi file goes missing because GHC detects a bad module interface hash and disables dynamic-too. It's disabled in findAndReadIface by assigning False to canGenerateDynamicToo. That flag stops hscWriteIface from writing a dyn_hi file.

GHC will print diagnostic messages if it's run with -ddump-if-trace, so you can see this happening.

IMO, this condition should be an error because GHC does not generate the file it was told to generate.

comment:4 Changed 3 years ago by heatsink

Description: modified (diff)

Changed 3 years ago by heatsink

Attachment: testcase9176-root.tar.gz added


comment:5 Changed 3 years ago by heatsink

Description: modified (diff)

comment:6 Changed 3 years ago by heatsink

Description: modified (diff)

comment:7 Changed 3 years ago by heatsink

Description: modified (diff)

comment:8 Changed 3 years ago by heatsink

Description: modified (diff)

comment:9 Changed 3 years ago by thomie

Operating System: MacOS XUnknown/Multiple

Thank you for the elaborate report. This is my current understanding of the problems you pointed out, and the steps I took to reproduce them:

$ cd testcase9176

$ make

$ ghc --show-iface dist/build/Testcase9176/Imported.hi | grep ABI
ABI hash: 2dc4674632fe6d892f3f2ff2cfa3b411

$ ghc --show-iface dist/build/Testcase9176/Imported.dyn_hi | grep ABI
ABI hash: 786d9e1369def1ee16d1bf9865cce2e0

First observation. The ABI hashes are different, as was the intention. Imported.o is compiled with -O2, whereas Imported.dyn_o is compiled with the default -O1. (Remove this difference, and they have the same ABI.)

$ ghc --show-iface dist/build/Testcase9176/User.hi | grep 'Imported '
import  -/  Testcase9176.Imported 2dc4674632fe6d892f3f2ff2cfa3b411

$ ghc --show-iface dist/build/Testcase9176/User.dyn_hi | grep 'Imported '
import  -/  Testcase9176.Imported 2dc4674632fe6d892f3f2ff2cfa3b411

This shows the first problem (2nd on your list of problems). User.dyn_hi depends on Imported.hi (2dc..), while it should depend on Imported.dyn_hi (786..). GHC never examines the dynamic interface of Imported. This problem still exists in HEAD (ghc-20141113).


# Change extension `.dynlib` to `.so` in `Makefile`.
$ make clean
$ cabal install
$ cd ../user
$ cat Makefile 
User.o User.dyn_o : User.hs
	ghc --make -static -dynamic-too User.hs
$ make
$ ls
Makefile  User.dyn_o  User.hi  User.hs  User.o

This shows the second problem (1st on your list of problems). User.dyn_hi is not generated, even though we request for -dynamic-too. GHC does not show any errors or warnings about this. I am not able to reproduce this problem with HEAD, since cabal install --with-ghc=ghc-7.9.20141113 fails for me with "Bad interface file" (which might be good news for all I know).

Last edited 3 years ago by thomie (previous) (diff)

comment:10 in reply to:  9 Changed 3 years ago by heatsink

Replying to thomie:

Thanks for reproducing this bug. That looks like what I experienced, except for the package not installing with HEAD. The "Bad interface file" message looks like it's refusing to install a package with inconsistent ABI hashes, which may be an improvement.

I don't think anyone has worked out what properties are intended to hold for interface files, such as whether having mismatched ABI hashes is allowed. I brought this up on the mailing and got replies about a few use cases that should be supported. I's not clear to me what this means for the implementation.

comment:11 Changed 22 months ago by ezyang

So, let's see if we can get this straight:

  1. In principle, it would be better if we didn't have any dyn_hi files. We should only have an hi file which everyone uses for typechecking. Because -dynamic does not affect type-checking/core optimization (it only affects the backend), any dyn_hi file you generate SHOULD be the same as hi. I'm pretty sure there isn't anything that is affected by this, and if there IS something we should kill it.
  1. In practice, if -dynamic-too does not work (as claimed on Windows), you have to run the compiler twice (once with -dynamic and once without) to actually get both dynamic and normal object files. Now, GHC is nondeterministic, so we're not guaranteed to get the same interface each time. That's trouble. Additionally, if you don't call GHC with exactly the same flags as the first time, you will end up with different interface files easily.

The moral of the story is that you should not be allowed compile static objects, and THEN compile dynamic objects in a separate GHC invocation. Then the nondeterminism problem goes away and we can kill dyn_hi files with fire.

It seems to me that -dynamic-too, even for platforms which "don't support it". I am not exactly sure how -dynamic-too is implemented, but in the worst case scenario it should be sufficient to just run the entire backend pipeline twice.

Note: See TracTickets for help on using tickets.