Opened 13 months ago

Closed 12 months ago

Last modified 12 months ago

#7819 closed bug (fixed)

FreeBSD without system libffi: Shared object "libffi.so.6" not found

Reported by: igloo Owned by: pgj
Priority: high Milestone: 7.8.1
Component: Build System Version: 7.6.2
Keywords: Cc: kazu@…, pho@…
Operating System: FreeBSD Architecture: Unknown/Multiple
Type of failure: None/Unknown Difficulty: Unknown
Test Case: Blocked By:
Blocking: Related Tickets:

Description

In #7806, kazu-yamamoto reported that on FreeBSD without a system libffi:

GHC can be built:

% make maintainer-clean
% perl boot
% ./configure --prefix=/ghc-head \
              --with-iconv-includes=/usr/local/include \
              --with-iconv-libraries=/usr/local/lib \
              --with-gmp-includes=/usr/local/include \
              --with-gmp-libraries=/usr/local/lib \
              --with-gcc=/usr/local/bin/gcc47 \
              --with-gcc-4.2=/usr/local/bin/gcc47
% gmake -j10

But installation fails:

% gmake install
Installing library in /ghc-head/lib/ghc-7.7.20130323/haskell2010-1.1.1.0
"/ghc-head/lib/ghc-7.7.20130323/bin/ghc-pkg" --force --global-package-db "/ghc-head/lib/ghc-7.7.20130323/package.conf.d" update rts/package.conf.install
Shared object "libffi.so.6" not found, required by "libHSrts-ghc7.7.20130323.so"
gmake[1]: *** [install_packages] Error 1
gmake: *** [install] Error 2

Change History (20)

comment:1 Changed 13 months ago by igloo

  • Operating System changed from Unknown/Multiple to FreeBSD
  • Owner set to pgj

pgj, would you be able to take a look at this please?

comment:2 Changed 13 months ago by pgj

Sure, I am hoping fix this in the coming days.

comment:3 Changed 13 months ago by kazu-yamamoto

  • Cc kazu@… added

comment:4 Changed 13 months ago by PHO

  • Cc pho@… added

comment:5 Changed 12 months ago by pgj

Well, regarding libffi I believe the following patch could solve the situation for you (please test):

diff --git a/rts/ghc.mk b/rts/ghc.mk
index b7651b0..a1e771e 100644
--- a/rts/ghc.mk
+++ b/rts/ghc.mk
@@ -186,7 +186,7 @@ else
 ifneq "$$(UseSystemLibFFI)" "YES"
 LIBFFI_LIBS = -Lrts/dist/build -lffi 
 ifeq "$$(TargetElf)" "YES"
-LIBFFI_LIBS += -optl-Wl,-rpath -optl-Wl,'$$$$ORIGIN'
+LIBFFI_LIBS += -optl-Wl,-rpath -optl-Wl,'$$$$ORIGIN' -optl-Wl,-zorigin
 endif

On FreeBSD, one needs to invoke ld(1) with -z origin in order to enable resolution of $ORIGIN in RPATH.

However, I guess we shall also have to face a more generic problem here: dynamic library paths are not linked as relative on FreeBSD, because ghc-pkg is not linked with other dependent dynamic libraries in its RPATH. I tried to use the Linux approach, i.e. enabled linking with relative paths in rules/distdir-way-opts.mk by applying the following (trivial) patch:

diff --git a/rules/distdir-way-opts.mk b/rules/distdir-way-opts.mk
index 872e527..fd9f5e5 100644
--- a/rules/distdir-way-opts.mk
+++ b/rules/distdir-way-opts.mk
@@ -129,6 +129,11 @@ ifeq "$$(TargetOS_CPP)" "linux"
 $1_$2_$3_GHC_LD_OPTS += \
     -fno-use-rpaths \
     $$(foreach d,$$($1_$2_TRANSITIVE_DEPS),-optl-Wl$$(comma)-rpath -optl-Wl$$(comma)'$$$$ORIGIN/../$$d')
+else ifeq "$$(TargetOS_CPP)" "freebsd"
+$1_$2_$3_GHC_LD_OPTS += \
+    -fno-use-rpaths \
+    $$(foreach d,$$($1_$2_TRANSITIVE_DEPS),-optl-Wl$$(comma)-rpath -optl-Wl$$(comma)'$$$$ORIGIN/../$$d') \
+    -optl-Wl,-zorigin
 else ifeq "$$(TargetOS_CPP)" "darwin"
 $1_$2_$3_GHC_LD_OPTS += -optl-Wl,-headerpad_max_install_names
 endif

Unfortunately the chances are high that there is rtld(1) bug in handling $ORIGIN substitutions and can only work with path sizes lesser than 1024. When I modified rtld(1) to work with larger path sizes, it worked, but I am now waiting for an official fix.

The builder clients work because they have the absolute paths embedded inside and installing them to a different location does not make any difference in that case. But when somebody tries to install the vanilla binary tarball they generate it will fail in a similar way.

Ian, do you have any recommendations how to work this problem around?

comment:6 follow-up: Changed 12 months ago by kazu-yamamoto

The build and installation works well with your first patch (only).
But the installed GHC refers to the dynamic libraries in the build directory.
So, after "gmake maintainer-clean", the installed GHC cannot find
necessary dynamic libraries.

comment:7 in reply to: ↑ 6 Changed 12 months ago by pgj

Replying to kazu-yamamoto:

The build and installation works well with your first patch (only).

Great!

But the installed GHC refers to the dynamic libraries in the build directory.
So, after "gmake maintainer-clean", the installed GHC cannot find
necessary dynamic libraries.

Yes, as I noted in my previous comment, this is because of a bug in rtld(1). If you want to see this working, you can experiment with a fix (written by Kostik Belousov). Download the diff, and rebuild your ld-elf.so.1 from the base system source code. This should be under /usr/src/libexec/rtld-elf, where you should just apply the patch and reinstall the dynamic loader by issuing the following commands below.

# make
# make install

comment:8 follow-up: Changed 12 months ago by kazu-yamamoto

Please merge these patches.

comment:9 in reply to: ↑ 8 Changed 12 months ago by pgj

Replying to kazu-yamamoto:

Please merge these patches.

Note that due to the referenced bug in the rtld(1), they can only work on a recent version of FreeBSD-CURRENT at the moment. I suppose the rtld(1) fix will be merged back to 8-STABLE and 9-STABLE in a week or so, but 8.4-RELEASE may not contain this fix, and we will have to wait for another couple of weeks for 9.2-RELEASE to "officially" have it. On the other hand, there should be some support for older release as well, i.e. for the releases (8.x, pre 9.2) who still has this bug in handling $ORIGIN.

I shall consult Ian on the possible scenarios, but definitely want to merge this patches as soon as possible.

comment:10 Changed 12 months ago by igloo

  • Status changed from new to patch

comment:11 Changed 12 months ago by ian@…

commit ce1094ea87e58f5172f623c61cbb250a3194e0b0

Author: Ian Lynagh <ian@well-typed.com>
Date:   Sat Apr 27 00:28:31 2013 +0100

    Use -rpath flags on all Elf OSes, not just Linux
    
    Patch from pgj; part of #7819.

 rules/distdir-way-opts.mk |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

comment:12 Changed 12 months ago by ian@…

commit 9843083601adb7db03844aecd9a855b0d06609a7

Author: Ian Lynagh <ian@well-typed.com>
Date:   Sat Apr 27 00:26:06 2013 +0100

    When linking with $ORIGIN rpaths, use the "-z origin" linker flag too
    
    On FreeBSD, one needs use "-z origin" in order to enable resolution
    of $ORIGIN in RPATH. Part of #7819.

 rts/ghc.mk                |    2 +-
 rules/distdir-way-opts.mk |    2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

comment:13 follow-up: Changed 12 months ago by igloo

  • Owner pgj deleted
  • Status changed from patch to new

Thanks, I've applied the patches.

Do the nightly builders have a patched ld? If so, I guess we should close the ticket now: I'm not sure what else we can do but wait for FreeBSD to fix the bug.

comment:14 in reply to: ↑ 13 Changed 12 months ago by pgj

Replying to igloo:

Thanks, I've applied the patches.

Thanks!

Do the nightly builders have a patched ld?

Currently they do not have, but I can patch the rtld(1) any time. My problem was rather that the users who may want to use produced snapshots will not have this patch installed. But perhaps I could add a note on this to somewhere in the wiki.

If so, I guess we should close the ticket now: I'm not sure what else we can do but wait for FreeBSD to fix the bug.

I can understand that you do not want to clutter the upstream sources with such corner cases -- I think I will be able to handle this in the corresponding port, I was just hoping that we could have a flag to keep it working without relying on $ORIGIN.

comment:15 follow-up: Changed 12 months ago by igloo

Ah, you need the patch when linking against the libraries, rather than when creating the libraries?

What other fix did you have in mind?

comment:16 in reply to: ↑ 15 Changed 12 months ago by pgj

  • Owner set to pgj

Replying to igloo:

What other fix did you have in mind?

Okay, I think I have solved it: Disable DYNAMIC_GHC_PROGRAMS for FreeBSD by default. Later I can override this in the Ports Collection for the FreeBSD releases that contain the aforementioned rtld(1) fix.

diff --git a/mk/config.mk.in b/mk/config.mk.in
index 55f5756..af70215 100644
--- a/mk/config.mk.in
+++ b/mk/config.mk.in
@@ -134,6 +134,9 @@ DYNAMIC_TOO = YES
 ifeq "$(TargetOS_CPP)" "mingw32"
 # This doesn't work on Windows yet
 DYNAMIC_GHC_PROGRAMS = NO
+# This does not work on FreeBSD yet as well
+else ifeq "$(TargetOS_CPP)" "freebsd"
+DYNAMIC_GHC_PROGRAMS = NO
 else
 DYNAMIC_GHC_PROGRAMS = YES
 endif

If there is no objection, I push this fix later today and close the ticket.

comment:17 follow-up: Changed 12 months ago by igloo

That'll be OK for 7.8. In 7.9 ghci won't work with DYNAMIC_GHC_PROGRAMS=NO, but that gives more than a year for the rtld patches to trickle down.

comment:18 Changed 12 months ago by pali.gabor@…

commit 8ab3cc1b878ac5915295e6f31ac9b592a4bde8c6

Author: Gabor Pali <pali.gabor@gmail.com>
Date:   Sat Apr 27 23:13:12 2013 +0200

    Set DYNAMIC_GHC_PROGRAMS=NO for FreeBSD because $ORIGIN is not resolved
    properly (see #7819)

 mk/config.mk.in |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

comment:19 in reply to: ↑ 17 Changed 12 months ago by pgj

  • Resolution set to fixed
  • Status changed from new to closed

Replying to igloo:

That'll be OK for 7.8. In 7.9 ghci won't work with DYNAMIC_GHC_PROGRAMS=NO, but that gives more than a year for the rtld patches to trickle down.

Okay, now closing the ticket.

comment:20 Changed 12 months ago by kazu-yamamoto

OK. I confirmed that ghc is now statically linked to GHC libraries.
Thank you!

Note: See TracTickets for help on using tickets.