Opened 5 years ago

Last modified 7 months ago

#4022 new bug

GHC Bindist is Broken on FreeBSD/amd64

Reported by: pgj Owned by: pgj
Priority: lowest Milestone: 7.12.1
Component: Core Libraries Version: 6.13
Keywords: GMP, integer-gmp, sharedlibs Cc:
Operating System: FreeBSD Architecture: x86_64 (amd64)
Type of failure: Building GHC failed Test Case:
Blocked By: Blocking:
Related Tickets: #8156 Differential Revisions:

Description

After correcting the name of the FreeBSD/amd64 platform in PlatformSupportsSharedLibs it turned out that shared libraries do not build on that platform at all. Log shows that the problem is with the integer-gmp library where the sources are not compiled with the -fPIC flag.

Excerpt from the build log:

...
"inplace/bin/ghc-stage1" libraries/integer-gmp/dist-install/build/GHC/Integer.dyn_o libraries/integer-gmp/dist-install/build/GHC/Integer/GMP/Internals.dyn_o libraries/integer-gmp/dist-install/build/GHC/Integer/Type.dyn_o libraries/integer-gmp/dist-install/build/cbits/gmp-wrappers.dyn_o libraries/integer-gmp/dist-install/build/cbits/cbits.dyn_o libraries/integer-gmp/gmp/objs/*.o `/usr/bin/find libraries/integer-gmp/dist-install/build -name "*_stub.dyn_o" -print` -shared -dynamic -dynload deploy -dylib-install-name /usr/local/lib/ghc-6.13.20100427/`basename "libraries/integer-gmp/dist-install/build/libHSinteger-gmp-0.2.0.0-ghc6.13.20100427.so" | sed 's/^libHS//;s/[-]ghc.*//'`/`basename "libraries/integer-gmp/dist-install/build/libHSinteger-gmp-0.2.0.0-ghc6.13.20100427.so"` -no-auto-link-packages -packageghc-prim-0.2.0.0 -o libraries/integer-gmp/dist-install/build/libHSinteger-gmp-0.2.0.0-ghc6.13.20100427.so
/usr/bin/ld: libraries/integer-gmp/gmp/objs/abs.o: relocation R_X86_64_32 can not be used when making a shared object; recompile with -fPIC
libraries/integer-gmp/gmp/objs/abs.o: could not read symbols: Bad value
...

According to Simon Marlow, it looks like it is building the in-tree gmp library rather than using a local one, and perhaps there is no shared version, so gcc picks up the static one and complains. There should be a shared gmp library when shared Haskell libraries are being built.

Attachments (2)

patch-libraries_integer-gmp_gmp_ghc.mk (951 bytes) - added by pgj 5 years ago.
A fix for the problem
integer-gmp-fix.diff (1.9 KB) - added by pgj 5 years ago.
(Re)Add support for setting GMP include and library dirs

Download all attachments as: .zip

Change History (33)

comment:1 Changed 5 years ago by pgj

  • Architecture changed from Unknown/Multiple to x86_64 (amd64)

Fix architecture.

comment:2 Changed 5 years ago by pgj

  • Keywords x86_64 FreeBSD removed

Remove some keywords which can be inferred from other fields.

comment:3 Changed 5 years ago by pgj

  • Status changed from new to assigned

Over to me.

comment:4 Changed 5 years ago by igloo

  • Milestone set to 6.12.3

Changed 5 years ago by pgj

A fix for the problem

comment:5 Changed 5 years ago by pgj

  • Status changed from new to patch

I have a patch that solves the problem.

comment:6 Changed 5 years ago by pgj

  • Owner changed from pgj to igloo

Ian, would you validate and push it? :)

comment:7 Changed 5 years ago by simonmar

The patch doesn't look right to me, it shouldn't be a platform-specific condition. Perhaps we should be building a shared version of the GMP library?

comment:8 Changed 5 years ago by pgj

I experimented with the approach I found in libffi/ghc.mk, but there were some problems with that.

This patch seemed to be a safer and a smaller one, and since the ghc.mk files already contains platform-specific conditions, I did not think it very rude :)

I wanted to apply a quick fix to unbreak the FreeBSD/amd64 builds (since GHC 6.12.3 is almost here), but I need more time to learn how things going your Makefiles if you want a nicer solution.

comment:9 Changed 5 years ago by pgj

Stop! I think I found the real cause: --with-gmp-includes=/usr/local/include and --with-gmp-libs=/usr/local/lib should be passed to the configure script in the build slaves. Hopefully no patching needed... I am just testing it.

Changed 5 years ago by pgj

(Re)Add support for setting GMP include and library dirs

comment:10 Changed 5 years ago by pgj

Okay... It seems to problem is that the support for setting "custom" include and library directories for GMP disappeared (as far as I recall, you can set it for GHC 6.10.4). I created a patch to handle --with-gmp-includes and --with-gmp-libraries so integer-gmp picks up the local GMP. It should work everywhere, no more platform-dependent hacks.

However, that also implies the compilation of the bundled GMP library is broken for 64 bits. I hope you will like this patch :P Note to Ian: so, for the GHC build slaves it requires these flags to be added for the configure script as well.

comment:11 Changed 5 years ago by simonmar

The patch looks ok to me.

I think the problem is not that the in-tree GMP library doesn't work for 64-bit, but that it doesn't work with shared libraries. It is just a coincidence that the first platform that tried to use the in-tree GMP with shared libraries happened to be an x86_64.

We probably need to use --enable-shared for GMP when building shared libraries, but I imagine there are related build system and installation fixes required.

comment:12 follow-up: Changed 5 years ago by pgj

No, this is not true. Please check out a recent build log for the FreeBSD/i386 build slave:

http://darcs.haskell.org/ghcBuilder/builders/pgj/45/8.html

It clearly shows that the in-tree GMP is picked up (search for the line "Configuring integer-gmp") as the local GMP cannot be found (since it is installed to /usr/local) just because the missing --with-gmp-* options in the configure script.

To me, the reason why the in-tree GMP actually works is pure luck. Simply adding "--enable-shared" will not help (I tried it in many different combination, but I reserve the right to be wrong), I think it needs a ghc.mk file similar to libffi's.

comment:13 in reply to: ↑ 12 ; follow-up: Changed 5 years ago by simonmar

Replying to pgj:

No, this is not true. Please check out a recent build log for the FreeBSD/i386 build slave:

http://darcs.haskell.org/ghcBuilder/builders/pgj/45/8.html

It clearly shows that the in-tree GMP is picked up (search for the line "Configuring integer-gmp") as the local GMP cannot be found (since it is installed to /usr/local) just because the missing --with-gmp-* options in the configure script.

So it's surprising that it works there. As you say, probably just lucky.

To me, the reason why the in-tree GMP actually works is pure luck. Simply adding "--enable-shared" will not help (I tried it in many different combination, but I reserve the right to be wrong), I think it needs a ghc.mk file similar to libffi's.

Exactly - that's what I meant by "I imagine there are related build system and installation fixes required".

So the point remains: the build system for the in-tree GMP library needs to be fixed for shared libraries.

comment:14 in reply to: ↑ 13 ; follow-up: Changed 5 years ago by pgj

Replying to simonmar:

the point remains: the build system for the in-tree GMP library needs to be fixed for shared libraries.

Yes, sorry for my misunderstanding :) By the way: what is your intention with the in-tree GMP?

comment:15 in reply to: ↑ 14 ; follow-up: Changed 5 years ago by simonmar

Replying to pgj:

By the way: what is your intention with the in-tree GMP?

It is necessary for platforms that don't provide a GMP library, e.g. Windows, Solaris(?), so we have to keep it.

comment:16 in reply to: ↑ 15 ; follow-ups: Changed 5 years ago by pgj

Replying to simonmar:

It is necessary for platforms that don't provide a GMP library, e.g. Windows, Solaris(?), so we have to keep it.

As far as I concerned, GNU MP is part of the recent MSYS (mingw) distribution you (theoretically) use for building GHC on Windows. GNU MP has been also ported to Solaris, however Solaris has its own MP library (though I do not know whether it supports bignums). Even Microsoft .NET has its wrapper for GNU MP. GCC (4.3+) also requires GNU MP (transparently via MPFR) to compile.

Simon, please do not get me wrong: I am happy with that in-tree GMP (and I also would be happy to fix it up), but I have the intuition that GNU MP can be taken available on any platform where GHC might be compiled (due to its UNIX-ish nature -- at least I have not seen my .vcsproj and .sln files appearing in the repository :P).

comment:17 in reply to: ↑ 16 Changed 5 years ago by pgj

Replying to pgj:

I have not seen my .vcsproj and .sln files appearing in the repository

s/my/any/

comment:18 in reply to: ↑ 16 ; follow-up: Changed 5 years ago by simonmar

  • Owner igloo deleted
  • Status changed from patch to new

Replying to pgj:

As far as I concerned, GNU MP is part of the recent MSYS (mingw) distribution you (theoretically) use for building GHC on Windows.

Right, but we not only require GMP when building GHC, we also have to ensure it is available when the user installs GHC. We don't currently ask the user to install anything in addition to GHC, we provide everything - including GMP. We can't use the MSYS GMP because presumably it depends on other MSYS bits and we don't want to end up shipping chunks of MSYS with GHC (also, I'm not sure it works to use the MSYS GMP, we need a native version).

GNU MP has been also ported to Solaris, however Solaris has its own MP library (though I do not know whether it supports bignums).

Sure - we know GMP works on Solaris, because we build it. The issue here is just making it easier for people to build and install GHC, that's why we have the in-tree GMP. Solaris doesn't have the equivalent of apt-get install libgmp-dev.

Even Microsoft .NET has its wrapper for GNU MP. GCC (4.3+) also requires GNU MP (transparently via MPFR) to compile.

Again, I don't think these help - we have to think about the install environment as well as the build environment.

Simon, please do not get me wrong: I am happy with that in-tree GMP (and I also would be happy to fix it up), but I have the intuition that GNU MP can be taken available on any platform where GHC might be compiled (due to its UNIX-ish nature -- at least I have not seen my .vcsproj and .sln files appearing in the repository :P).

We could add do that, but it would make GHC harder to build and install, which is why we include GMP currently.

The patch has been pushed, so changing the ticket state back to "new", as the underlying problem with the in-tree GMP still needs to be fixed, it's perhaps not as urgent as before though.

comment:19 in reply to: ↑ 18 Changed 5 years ago by pgj

Replying to simonmar:

We don't currently ask the user to install anything in addition to GHC, we provide everything - including GMP.

So as perl and libiconv? Perl is no longer part of the base system on FreeBSD, so it cannot be taken available. The iconv library is also a requirement on some platforms (like FreeBSD) that you do not have in-tree, at least as far as I know.

Of course the Ports Collection can handle this issues, but I think you are referring to the case when the user wants to install GHC from sources (or from vanilla bindists) without being aware of the ports.

The patch has been pushed, so changing the ticket state back to "new", as the underlying problem with the in-tree GMP still needs to be fixed, it's perhaps not as urgent as before though.

Ok, thank you.

comment:20 Changed 5 years ago by pgj

  • Owner set to pgj

I would like to work with this ticket.

comment:21 Changed 5 years ago by igloo

  • Milestone changed from 6.12.3 to 6.14.1
  • Priority changed from normal to low

comment:22 Changed 5 years ago by igloo

  • Milestone changed from 7.0.1 to 7.0.2

comment:23 Changed 4 years ago by igloo

  • Milestone changed from 7.0.2 to 7.2.1

comment:24 Changed 4 years ago by igloo

  • Milestone changed from 7.2.1 to 7.4.1

comment:25 Changed 3 years ago by igloo

  • Milestone changed from 7.4.1 to 7.6.1
  • Priority changed from low to lowest

comment:26 Changed 3 years ago by igloo

  • Milestone changed from 7.6.1 to 7.6.2

comment:27 Changed 2 years ago by errge

  • difficulty set to Unknown

comment:28 Changed 13 months ago by thoughtpolice

  • Milestone changed from 7.6.2 to 7.10.1

Moving to 7.10.1.

comment:29 Changed 10 months ago by thoughtpolice

  • Component changed from libraries (other) to Core Libraries

Moving over to new owning component 'Core Libraries'.

comment:30 Changed 7 months ago by thoughtpolice

  • Milestone changed from 7.10.1 to 7.12.1

Moving to 7.12.1 milestone; if you feel this is an error and should be addressed sooner, please move it back to the 7.10.1 milestone.

comment:31 Changed 7 months ago by thoughtpolice

Moving to 7.12.1 milestone; if you feel this is an error and should be addressed sooner, please move it back to the 7.10.1 milestone.

Note: See TracTickets for help on using tickets.