Opened 3 years ago

Closed 20 months ago

#5743 closed feature request (fixed)

Configurably use system-provided libffi

Reported by: nomeata Owned by: trommler
Priority: normal Milestone: 7.6.2
Component: Build System Version: 7.7
Keywords: Cc: Jens, Petersen, <petersen@…>, slyfox@…, pho@…
Operating System: Linux Architecture: Unknown/Multiple
Type of failure: None/Unknown Difficulty: Unknown
Test Case: Blocked By:
Blocking: Related Tickets: #4496

Description

Hi,

both Debian and Fedora patch ghc to use the system-wide installed libffi, as required by distribution policies:
http://anonscm.debian.org/darcs/pkg-haskell/experimental/ghc/patches/system-libffi
http://pkgs.fedoraproject.org/gitweb/?p=ghc.git;a=blob;f=ghc-use-system-libffi.patch

It would be helpful for us if we would not have to patch this, but rather trigger this by a ./configure flag.

Thanks,
Joachim

Attachments (2)

0001-Add-configure-option-to-use-system-provided-libffi.patch (8.0 KB) - added by trommler 21 months ago.
0001-Link-in-tree-libffi-to-rts.-Fixes-trac-5743.patch (1.0 KB) - added by trommler 20 months ago.
Fix in-tree libffi linking

Download all attachments as: .zip

Change History (23)

comment:1 Changed 2 years ago by igloo

  • Difficulty set to Unknown
  • Milestone set to 7.6.1

It would be simpler if we can always do the same thing.

Perhaps we should look at whether OS X comes with libffi, and whether there's a bin tarball for mingw.

comment:2 Changed 2 years ago by slyfox

  • Cc Jens Petersen <petersen@…> slyfox@… added; Jens Petersen <petersen@…> removed

comment:3 follow-up: Changed 2 years ago by slyfox

Moreover, ghc does not embed RPATH to libffi for packages using it:

sf haskell-updater-1.2.0.5 # /usr/bin/ghc -package Cabal-1.14.0 --make /var/tmp/portage/app-admin/haskell-updater-1.2.0.5-r1/work/haskell-updater-1.2.0.5/Setup.lhs -dynamic -o setup
Linking setup ...
sf haskell-updater-1.2.0.5 # lddtree ./setup 
setup => ./setup (interpreter => /lib64/ld-linux-x86-64.so.2)
    libHSCabal-1.14.0-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/Cabal-1.14.0/libHSCabal-1.14.0-ghc7.4.1.so
    libHSprocess-1.1.0.1-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/process-1.1.0.1/libHSprocess-1.1.0.1-ghc7.4.1.so
    libHSpretty-1.1.1.0-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/pretty-1.1.1.0/libHSpretty-1.1.1.0-ghc7.4.1.so
    libHSdirectory-1.1.0.2-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/directory-1.1.0.2/libHSdirectory-1.1.0.2-ghc7.4.1.so
    libHSunix-2.5.1.0-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/unix-2.5.1.0/libHSunix-2.5.1.0-ghc7.4.1.so
    librt.so.1 => /lib64/librt.so.1
        libpthread.so.0 => /lib64/libpthread.so.0
    libutil.so.1 => /lib64/libutil.so.1
    libdl.so.2 => /lib64/libdl.so.2
    libHSbytestring-0.9.2.1-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/bytestring-0.9.2.1/libHSbytestring-0.9.2.1-ghc7.4.1.so
    libHSold-time-1.1.0.0-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/old-time-1.1.0.0/libHSold-time-1.1.0.0-ghc7.4.1.so
    libHSold-locale-1.0.0.4-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/old-locale-1.0.0.4/libHSold-locale-1.0.0.4-ghc7.4.1.so
    libHSfilepath-1.3.0.0-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/filepath-1.3.0.0/libHSfilepath-1.3.0.0-ghc7.4.1.so
    libHScontainers-0.4.2.1-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/containers-0.4.2.1/libHScontainers-0.4.2.1-ghc7.4.1.so
    libHSdeepseq-1.3.0.0-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/deepseq-1.3.0.0/libHSdeepseq-1.3.0.0-ghc7.4.1.so
    libHSarray-0.4.0.0-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/array-0.4.0.0/libHSarray-0.4.0.0-ghc7.4.1.so
    libHSbase-4.5.0.0-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/base-4.5.0.0/libHSbase-4.5.0.0-ghc7.4.1.so
    libHSinteger-gmp-0.4.0.0-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/integer-gmp-0.4.0.0/libHSinteger-gmp-0.4.0.0-ghc7.4.1.so
    libgmp.so.10 => /usr/lib64/libgmp.so.10
    libHSghc-prim-0.2.0.0-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/ghc-prim-0.2.0.0/libHSghc-prim-0.2.0.0-ghc7.4.1.so
    libHSrts-ghc7.4.1.so => /usr/lib64/ghc-7.4.1/libHSrts-ghc7.4.1.so
        libffi.so.5 => not found
    libm.so.6 => /lib64/libm.so.6
    libc.so.6 => /lib64/libc.so.6

Notice libffi.so.5 => not found.

sf haskell-updater-1.2.0.5 # ls -l /usr/lib/ghc-7.4.1/libffi.so*
-rwxr-xr-x 1 root root 30832 апр.  14 21:20 /usr/lib/ghc-7.4.1/libffi.so
-rw-r--r-- 1 root root 30832 апр.  14 21:20 /usr/lib/ghc-7.4.1/libffi.so.5
-rw-r--r-- 1 root root 30832 апр.  14 21:19 /usr/lib/ghc-7.4.1/libffi.so.5.0.10
sf haskell-updater-1.2.0.5 # ls -l /usr/lib/libffi.so*
lrwxrwxrwx 1 root root    15 апр.  14 11:53 /usr/lib/libffi.so -> libffi.so.6.0.0
lrwxrwxrwx 1 root root    15 апр.  14 11:53 /usr/lib/libffi.so.6 -> libffi.so.6.0.0
-rwxr-xr-x 1 root root 30816 апр.  14 11:53 /usr/lib/libffi.so.6.0.0

comment:4 Changed 2 years ago by trommler

  • Owner set to trommler

I would also like to see this fixed for the ghc package I maintain at openSUSE. So I'd like to work on this.

comment:5 follow-up: Changed 2 years ago by slyfox

A note:

what is missing in fedora/debian patch is:

  • the real request for libffi's library and include path via pkg-config.
  • ./configure time convigurability

Gentoo, for example, allws multiple libffi headers in the following locations:
/usr/lib64/libffi-3.0.11/include/ffi.h
/usr/lib64/libffi-3.0.11/include/ffitarget.h

comment:6 in reply to: ↑ 5 Changed 2 years ago by trommler

Replying to slyfox:

Gentoo, for example, allws multiple libffi headers in the following locations:
/usr/lib64/libffi-3.0.11/include/ffi.h
/usr/lib64/libffi-3.0.11/include/ffitarget.h

I will add a new option to ./configure called --with-system-libffi and have an optional
parameter for it so you can specify which library/module you want to use for ffi.

Is that what you had in mind?

comment:7 Changed 2 years ago by trommler

comment:8 Changed 22 months ago by igloo

  • Milestone changed from 7.6.1 to 7.6.2

comment:9 Changed 21 months ago by trommler

  • Status changed from new to patch

The patch validates on Linux with and without the new flag --with-system-libffi. Please validate on MacOS and Windows before pushing. I tried to validate on MacOS but failed to compile ghc-7.6 branch without the patch.

The patch adds three flags to configure:

--with-system-libffi use system provided libffi

--with-ffi-includes find includes for libffi in non-standard directory

--with-ffi-libraries find libraries for libffi in non-standard directory

comment:10 in reply to: ↑ 3 ; follow-up: Changed 21 months ago by trommler

Replying to slyfox:

Moreover, ghc does not embed RPATH to libffi for packages using it:

In fact ghc does not set any RPATH (or RUNPATH if you are using ELF's new dynamic tags) in the libraries that come with it. I relies Cabal setting the correct RPATH/RUNPATH in the binaries using a library.

As this is a more general issue with the way shared libraries are built (see for example trac #7062) and installed in ghc, I did not yet add a patch that just fixes the symptom.

comment:11 in reply to: ↑ 10 Changed 20 months ago by trommler

Replying to trommler:

As this is a more general issue with the way shared libraries are built (see for example trac #7062) and installed in ghc, I did not yet add a patch that just fixes the symptom.

In HEAD linking of shared libraries now has been changed to embed RPATH/RUNPATH (see #3072) so I will prepare another patch to embed RPATH/RUNPATH for libffi.

comment:12 Changed 20 months ago by igloo

See also #4496

comment:13 Changed 20 months ago by ian@…

commit 3005e90936c47c1f71672bf6c84fff20cb14014b

Author: Ian Lynagh <ian@well-typed.com>
Date:   Thu Nov 29 22:22:39 2012 +0000

    Add configure option to use system provided libffi; fixes #5743
    
    Based on patch from Peter Trommler:
    
        From 293495d40f62e691520331a41c6d85d82e120169 Mon Sep 17 00:00:00 2001
        From: Peter Trommler <ptrommler@acm.org>
        Date: Sun, 21 Oct 2012 18:47:01 +0200
        Subject: [PATCH] Add configure option to use system provided libffi This
         fixes track # 5743 and #4496.

 configure.ac        |   55 +++++++++++++++++++++++++++++++++++++++++++++++++++
 ghc.mk              |   11 ++++++++-
 mk/config.mk.in     |    7 ++++++
 rts/ghc.mk          |   42 ++++++++++++++++++++++++++++++++++----
 rts/package.conf.in |    3 ++
 5 files changed, 111 insertions(+), 7 deletions(-)

comment:14 Changed 20 months ago by igloo

  • Resolution set to fixed
  • Status changed from patch to closed

In #4496 I said:

On the Mac I have /usr/lib/libffi.dylib and /usr/include/ffi/*.h.

However, I can't see any mingw tarballs that seem likely to include it.

I can't see any sign of libffi in mingw64 either.

I really don't like having this be conditional, but I can't see a better alternative, so I've applied the patch. Thanks.

I'm not sure if it's worth listing the new flag explicitly on
http://hackage.haskell.org/trac/ghc/wiki/Building/Using#Runtheconfigurescript
as it'll only really be of interest to a handful of packagers, and that page only tries to list the most common flags, leaving the rest to ./configure --help.

comment:15 follow-up: Changed 20 months ago by PHO

  • Cc pho@… added

Your patch mentions a variable LIBFFI_LIBS, which isn't defined anywhere. I think you need something like:

diff --git a/rts/ghc.mk b/rts/ghc.mk
index e3c9fa6..d3638ac 100644
--- a/rts/ghc.mk
+++ b/rts/ghc.mk
@@ -386,12 +386,14 @@ rts/dist/build/AutoApply_HC_OPTS += -fno-PIC -static
 endif
 endif
 
-# add CFLAGS for libffi
+# add CFLAGS and LIBS for libffi
 # ffi.h triggers prototype warnings, so disable them here:
 ifeq "$(UseSystemLibFFI)" "YES"
 LIBFFI_CFLAGS = $(addprefix -I,$(FFIIncludeDir))
+LIBFFI_LIBS   = $(addprefix -L,$(FFILibDir)) -lffi
 else
 LIBFFI_CFLAGS =
+LIBFFI_LIBS   = -Lrts/dist/build -lffi
 endif
 rts/Interpreter_CC_OPTS += -Wno-strict-prototypes $(LIBFFI_CFLAGS)
 rts/Adjustor_CC_OPTS    += -Wno-strict-prototypes $(LIBFFI_CFLAGS)

comment:16 in reply to: ↑ 15 ; follow-up: Changed 20 months ago by trommler

  • Owner trommler deleted
  • Resolution fixed deleted
  • Status changed from closed to new

Replying to PHO:

Your patch mentions a variable LIBFFI_LIBS, which isn't defined anywhere. I think you need something like:

This was part of my original patch but did not make it into the above commit:

ifneq "$(UseSystemLibFFI)" "YES"
LIBFFI_LIBS = -Lrts/dist/build -lffi 
else
LIBFFI_LIBS =
endif

In the case of UseSystemLibFFI=="YES" the correct linker flags will be set as part of rts/libs.depend, hence the empty definition for LIBFFI_LIBS.

Tests driver dynHelloWorld, dynlibs T3807 and T5373 fail here with unresolved symbols from ffi when built with the in-tree libffi.

I will reopen, assign to me, and prepare a patch.

comment:17 Changed 20 months ago by trommler

  • Owner set to trommler
  • Version changed from 7.2.1 to 7.7

Changed 20 months ago by trommler

Fix in-tree libffi linking

comment:18 Changed 20 months ago by trommler

  • Status changed from new to patch

comment:19 in reply to: ↑ 16 Changed 20 months ago by PHO

Replying to trommler:

In the case of UseSystemLibFFI=="YES" the correct linker flags will be set as part of rts/libs.depend, hence the empty definition for LIBFFI_LIBS.

I get it. But then what about FFILibDir? If I understand this correctly, rts/libs.depend contains only -l flags so -L$(FFILibDir) won't be passed to the linker.

Apart from that, $(rts_LD_OPTS) seems to be ignored too. Or am I missing some point?

comment:20 Changed 20 months ago by ptrommler@…

commit 7ee5bedc5e5039c9bb3ba06b50a4395e579a4b0e

Author: Peter Trommler <ptrommler@acm.org>
Date:   Fri Nov 30 13:40:10 2012 +0100

    Link in-tree libffi to rts. Fixes trac #5743.

 rts/ghc.mk |    6 ++++++
 1 files changed, 6 insertions(+), 0 deletions(-)

comment:21 Changed 20 months ago by igloo

  • Resolution set to fixed
  • Status changed from patch to closed

I've applied the patch, thanks. I think we only use rts_LD_OPTS when calling ld directly.

If you have any problems building, it's probably best to open a new ticket giving details, as this one is starting to get a little large and most of the history probably won't be relevant.

Note: See TracTickets for help on using tickets.