In ghci object code loader, linking against the previous temp dylib is not enough on OS X
joelteon encountered this issue in a rather complicated program, and we worked out the cause over IRC.
Suppose that ghci needs to sequentially load three modules A, B and C, where B refers to symbols from A and C refers to symbols from both A and B. (For example modules B, C and another module D all contain Template Haskell that refers to the previous module(s).) The object code linker currently works like this:
- Link the module
A.o
into a dylibghc_1.dylib
- Link the module
B.o
againstghc_1.dylib
into a new dylibghc_2.dylib
- Link the module
C.o
againstghc_2.dylib
into a new dylibghc_3.dylib
As a result, ghc_2.dylib
ends up with a NEEDED (or whatever it's called in Mach-O) entry for ghc_1.dylib
, and ghc_3.dylib
ends up with a NEEDED entry for ghc_2.dylib
.
However, this apparently does not satisfy the OS X dlopen
implementation, which complains about a missing symbol _A_foo
referenced by ghc_3.dylib
and which is defined in ghc_1.dylib
. Apparently the dynamic loader only checks the direct dependencies when trying to resolve undefined symbols.
(The Linux dynamic loader seems to be perfectly happy to use an indirect dependency to resolve an undefined symbol. But I found out that the linker gold has the same sort of behavior as the OS X dynamic loader. I don't know whether there is any standard here, but it seems that we cannot rely on the Linux dynamic loader's behavior.)
Presumably the fix is to keep track of all the previous temporary dylibs (rather than just one in last_temp_so
) and link against all of them when building a new temporary dylib. I'm slightly worried about quadratic behavior here, but I think it's unlikely to be an issue in practice.
I have a reproducer at http://lpaste.net/808723564239781888 (which I'll add to the test suite next) and oherrala ran the test on an OS X system with the following results:
=====> Last(ghci) 1167 of 4480 [0, 0, 0]
cd ./ghci/scripts && HC="/Users/oherrala/gits/ghc/inplace/bin/ghc-stage2" HC_OPTS="-dcore-lint -dcmm-lint -dno-debug-output -no-user-package-db -rtsopts -fno-warn-tabs -fno-ghci-history " "/Users/oherrala/gits/ghc/inplace/bin/ghc-stage2" -dcore-lint -dcmm-lint -dno-debug-output -no-user-package-db -rtsopts -fno-warn-tabs -fno-ghci-history --interactive -v0 -ignore-dot-ghci +RTS -I0.1 -RTS <Last.script > Last.run.stdout 2> Last.run.stderr
Actual stderr output differs from expected:
--- /dev/null 2015-04-18 17:23:26.000000000 +0300
+++ ./ghci/scripts/Last.run.stderr 2015-04-18 17:23:34.000000000 +0300
@@ -0,0 +1,9 @@
+ghc-stage2: panic! (the 'impossible' happened)
+ (GHC version 7.11.20150418 for x86_64-apple-darwin):
+ Loading temp shared object failed: dlopen(/var/folders/64/90jfy8lj65bcm1k02syxz_l80000gn/T/ghc18812_0/libghc18812_12.dylib, 5): Symbol not found: _LastA_a_closure
+ Referenced from: /var/folders/64/90jfy8lj65bcm1k02syxz_l80000gn/T/ghc18812_0/libghc18812_12.dylib
+ Expected in: flat namespace
+ in /var/folders/64/90jfy8lj65bcm1k02syxz_l80000gn/T/ghc18812_0/libghc18812_12.dylib
+
+Please report this as a GHC bug: http://www.haskell.org/ghc/reportabug
+
Actual stdout output differs from expected:
--- ./ghci/scripts/Last.stdout 2015-04-18 16:26:55.000000000 +0300
+++ ./ghci/scripts/Last.run.stdout 2015-04-18 17:23:34.000000000 +0300
@@ -1,3 +1,2 @@
3
4
-7
*** unexpected failure for Last(ghci)
Trac metadata
Trac field | Value |
---|---|
Version | 7.10.1 |
Type | Bug |
TypeOfFailure | OtherFailure |
Priority | high |
Resolution | Unresolved |
Component | Compiler |
Test case | |
Differential revisions | |
BlockedBy | |
Related | |
Blocking | |
CC | |
Operating system | |
Architecture |