wiki:DynamicGhcPrograms

Version 3 (modified by igloo, 9 months ago) (diff)

--

Dynamic GHC programs

Currently, GHCi doesn't use the system linker to load libraries, but instead uses our own "GHCi linker". Unfortunately, this is a large blob of unpleasant code that we need to maintain, it already contains a number of known bugs, and new problems have a tendency to arise as new versions of OSes are released. We are therefore keen to get rid of it!

There is some benefit (in terms of both bugs fixed and code removed) to removing it for a particular platform (e.g. Linux(elf)/x86), more benefit to removing it for a particular OS (e.g. Linux(elf)), but the most benefit is gained by removing it entirely.

Our solution is to switch GHCi from using the "static way", to using the "dynamic way". GHCi will then use the system linker to load the .dll for the library, rather than using the GHCi linker to load the .a.

This is controlled by the DYNAMIC_GHC_PROGRAMS build system variable. If it is YES, then we build ghci the dynamic way. If it is NO then we build it the static way.

Build times, and -dynamic-too

The default way, as used by ghc Foo.hs for example, will remain the vanilla (static) way. But GHCi uses dynamic libraries, so we need to build all the libraries both ways (both in the GHC build, and when the user cabal-installs 3rd party libraries). This would double the compilation time needed.

We have therefore added a -dynamic-too flag. This allows a single invocation of GHC to build both the vanilla and dynamic ways. The GHC build system uses this if DYNAMIC_TOO is YES, and Cabal HEAD uses it if the compiler supports it.

Current status

Unix-like platforms

Both DYNAMIC_GHC_PROGRAMS and DYNAMIC_TOO work on unix-like platforms. They are enabled by default provided the platform supports dynamic libraries (except on FreeBSD, where it is disabled for now, while we wait for a patch to rtld(1) to filter through to the FreeBSD release; #7819).

Windows

On Windows, DYNAMIC_GHC_PROGRAMS works, but DYNAMIC_TOO doesn't. Currently, DYNAMIC_GHC_PROGRAMS is disabled, although it could be enabled at the expense of longer compilation times. Linking time is also significantly higher for the dynamic way, but we aren't aware of any way to improve that.

Other platforms

We do not know the situation with other platforms, such as iOS and Android. We do not know whether they have a system loader that they can use, or whether it would be useful to keep the GHCi linker around for them.

Bugs

As well as the ticket for implementing dynamic GHCi (#3658), the table below lists the related tickets and the platforms that they affect. Most, if not all, of these would be immediately fixed by switching to dynamic GHCi.

TicketAffects OS X x86_64?Affects OS X x86?Affects Linux x86_64?Affects Linux x86?Affects Windows x86_64?Affects Windows x86?Affects other platforms?
#781 GHCi on x86_64, cannot link to static data in shared libsnonoYESnononono
#1883 GHC can't find library using "short" namenonononoprobablyYESno
#2283 WIndows: loading objects that refer to DLL symbolsnonononoprobablyYESno
#3242 ghci: can't load .so/.DLL for: m (addDLL: could not load DLL)nonononoprobablyYESno
#3654 Mach-O GHCi linker lacks support for a range of relocation entriesYESYESnonononono
#4244 Use system linker in GHCi to support alpha, ia64, ppc64nonononononoYES
#5062 Patch: Debug output for OS X linker and coding standard upgradesYESYESnonononono
#5197 Support static linker semantics for archives and weak symbolsYESYESYESYESYESYESYES
#5435 GHCi linker should run constructors for linked librariesYESYESYESYESYESYESYES
#6107 GHCi runtime linker cannot link with duplicate common symbolsYESYESYESYESYESYESYES
#7043 32-bit GHC ceiling of negative float SEGFAULT: 11noYESnonononono
#7056 GHCi loadArchive "libiconv.a":failed Unknown PEi386 section name `.drectve'nonononoprobablyYESno
#7072 GHC interpreter does not find stat64 symbol on LinuxnonoYESnononono
#7097 linker fails to load package with binding to foreign librarynonononoprobablyYESno
#7103 Compiler panic, when loading wxc in GHCinonononoprobablyYESno
#7134 ghc-7.6.0.20120810-x86_64-windows.exe -> internal error R_X86_64_PC32nonononoYESnono
#7207 linker fails to load package with binding to foreign library (win64)nonononoYESnono
#7299 threadDelay broken in ghci, Mac OS XYESYESnonononono
#7357 GHC.exe gives an internal error while linking vector's Monadic.hsnonononoYESnono

Other issues

Cabal support

Currently released versions of Cabal/cabal-install don't handle dynamic GHCi well. In particular, if a library uses Template Haskell, then Cabal will build the ways in the wrong order, so compilation will fail. Cabal in HEAD handles it properly.

Other approaches

In #3658 there is some discussion of related design decisions etc.