Opened 7 years ago

Closed 3 years ago

Last modified 3 years ago

#3072 closed feature request (fixed)

considerations for management of shared libs

Reported by: duncan Owned by: igloo
Priority: low Milestone: 7.6.2
Component: Package system Version: 6.10.1
Keywords: Cc: pho@…
Operating System: Linux Architecture: Unknown/Multiple
Type of failure: None/Unknown Test Case:
Blocked By: Blocking:
Related Tickets: Differential Revisions:

Description

Some thought needs to be put into how to manage shared libs on the various platforms we support. Sadly the solution is likely to be different on Windows vs OSX vs ELF-format platforms. For ELF platforms our current idea is to put all shared libs in /usr{/local}/lib with names like libHSfoo-x.y.z-ghc-a.b.c.so. Having them all in the system dir like that has it's downsides and it does not work for per-user installs. If we want to have multiple instances of the same version of a lib then it's even more problematic.

All ELF linkers support the RPATH/RUNPATH mechanism for executables and shared libs to locate each other without them having to be on the standard linker path. This basically involves embedding the path in them at link time. It must be acknowledged that some linux distributions eschew the use of rpath for various reasons. We must take that into consideration. Also we must consider relocatable/redistributable binaries...

The GNU and Solaris linkers support an extension which allows the above mechanism to work, even for relocatable binaries (so long as the libs are kept relative to the executable). Basically one can set an rpath that uses $ORIGIN and at runtime the $ORIGIN is substituted for the file system location of the executable or shared lib in question.

Note that on Linux at least the $ORIGIN is always passed by the kernel so that the dynamic linker can resolve it. See the attached program which demonstrates this.

Attachments (2)

origin.c (1.3 KB) - added by duncan 7 years ago.
ghc-7.7.20121101-corelibs-rpath.patch (1.5 KB) - added by markwright 3 years ago.
set RPATH using $ORIGIN in ghc core libs for ghc-7.7.20121101

Download all attachments as: .zip

Change History (21)

Changed 7 years ago by duncan

comment:1 Changed 7 years ago by duncan

Should have linked to the docs for $ORIGIN:

Solaris: http://docs.sun.com/app/docs/doc/805-3050/6j2tng0mg?l=ko&a=view

Solaris and Linux: http://www.scons.org/wiki/UsingOrigin

GNU man ld.so

$ORIGIN and rpath
  ld.so  understands the string $ORIGIN (or equivalently ${ORIGIN}) in an
  rpath specification (DT_RPATH or DT_RUNPATH) to mean the directory con‐
  taining  the  application  executable.  Thus, an application located in
  somedir/app could be compiled with gcc  -Wl,-rpath,'$ORIGIN/../lib'  so
  that  it  finds  an  associated shared library in somedir/lib no matter
  where somedir is located in the directory hierarchy.  This  facilitates
  the  creation  of  "turn-key"  applications  that  do  not  need  to be
  installed into special directories, but can instead  be  unpacked  into
  any directory and still find their own shared libraries.

This may also be useful for binary distributions of ghc that use shared libs.

comment:2 Changed 6 years ago by igloo

  • difficulty set to Unknown
  • Milestone set to 6.12.1

comment:3 Changed 6 years ago by simonmar

  • Component changed from None to Package system

comment:4 Changed 6 years ago by ross

  • Type changed from proposal to feature request

comment:5 Changed 6 years ago by igloo

  • Milestone changed from 6.12.1 to 6.14.1
  • Type of failure set to None/Unknown

comment:6 Changed 6 years ago by igloo

See also #3268.

comment:7 Changed 5 years ago by igloo

  • Milestone changed from 7.0.1 to 7.0.2

comment:8 Changed 4 years ago by igloo

  • Milestone changed from 7.0.2 to 7.2.1

comment:10 Changed 4 years ago by igloo

  • Milestone changed from 7.2.1 to 7.4.1

comment:11 Changed 4 years ago by igloo

  • Milestone changed from 7.4.1 to 7.6.1
  • Priority changed from normal to low

comment:12 Changed 3 years ago by igloo

  • Milestone changed from 7.6.1 to 7.6.2

Changed 3 years ago by markwright

set RPATH using $ORIGIN in ghc core libs for ghc-7.7.20121101

comment:13 Changed 3 years ago by markwright

From: http://hackage.haskell.org/trac/ghc/ticket/7062#comment:21 "GHC libraries do not set a RUNPATH or RPATH tag for other GHC libraries they depend on" I attached ghc-7.7.20121101-corelibs-rpath.patch to set the rpath in the ghc core libs, so they look like:

$ readelf --dynamic /usr/lib64/ghc-7.7.20121101/containers-0.5.0.0/libHScontainers-0.5.0.0-ghc7.7.20121101.so 

Dynamic section at offset 0x1c37f0 contains 35 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libHSdeepseq-1.3.0.1-ghc7.7.20121101.so]
 0x0000000000000001 (NEEDED)             Shared library: [libHSarray-0.4.0.1-ghc7.7.20121101.so]
 0x0000000000000001 (NEEDED)             Shared library: [libHSbase-4.6.0.0-ghc7.7.20121101.so]
 0x0000000000000001 (NEEDED)             Shared library: [libHSinteger-gmp-0.5.0.0-ghc7.7.20121101.so]
 0x0000000000000001 (NEEDED)             Shared library: [libgmp.so.10]
 0x0000000000000001 (NEEDED)             Shared library: [libHSghc-prim-0.3.0.0-ghc7.7.20121101.so]
 0x0000000000000001 (NEEDED)             Shared library: [libc.so.6]
 0x000000000000000e (SONAME)             Library soname: [libHScontainers-0.5.0.0-ghc7.7.20121101.so]
 0x0000000000000010 (SYMBOLIC)           0x0
 0x000000000000000f (RPATH)              Library rpath: [$ORIGIN/../deepseq-1.3.0.1:$ORIGIN/../array-0.4.0.1:$ORIGIN/../base-4.6.0.0:$ORIGIN/../integer-gmp-0.5.0.0:$ORIGIN/../ghc-prim-0.3.0.0:$ORIGIN/../rts-1.0]
 0x000000000000001d (RUNPATH)            Library runpath: [$ORIGIN/../deepseq-1.3.0.1:$ORIGIN/../array-0.4.0.1:$ORIGIN/../base-4.6.0.0:$ORIGIN/../integer-gmp-0.5.0.0:$ORIGIN/../ghc-prim-0.3.0.0:$ORIGIN/../rts-1.0]
... snip ...
$ 

comment:14 follow-up: Changed 3 years ago by simonmar

@markwright: will this work when the libraries are in the build tree, or only when installed?

comment:15 in reply to: ↑ 14 Changed 3 years ago by markwright

Replying to simonmar:

@markwright: will this work when the libraries are in the build tree, or only when installed?

Only when they are installed (where it works great, I built 170 pkgs from hackage using in the Gentoo Haskell overlay with 7.7.20121102 using this patch with no LD_LIBRARY_PATH).

In the build tree it can not find the libraries (of course a workaround would be to set LD_LIBRARY_PATH):

argus (ghclive) ghc-9999 $ env | grep LD_LIBRARY_PATH
argus (ghclive) ghc-9999 $ find . -type f -name libHScontainers*.so -print
./libraries/containers/dist-install/build/libHScontainers-0.5.0.0-ghc7.7.20121102.so
argus (ghclive) ghc-9999 $ ldd libraries/containers/dist-install/build/libHScontainers-0.5.0.0-ghc7.7.20121102.so
	linux-vdso.so.1 (0x00007fffae782000)
	libHSdeepseq-1.3.0.1-ghc7.7.20121102.so => not found
	libHSarray-0.4.0.1-ghc7.7.20121102.so => not found
	libHSbase-4.6.0.0-ghc7.7.20121102.so => not found
	libHSinteger-gmp-0.5.0.0-ghc7.7.20121102.so => not found
	libgmp.so.10 => /usr/lib64/libgmp.so.10 (0x00007f36da50e000)
	libHSghc-prim-0.3.0.0-ghc7.7.20121102.so => not found
	libc.so.6 => /lib64/libc.so.6 (0x00007f36da168000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f36dab6e000)
argus (ghclive) ghc-9999 $ 

comment:16 Changed 3 years ago by igloo

  • Owner set to igloo

comment:17 Changed 3 years ago by ian@…

commit 7dd7008f98c76024288899b8cea607141c91ada5

Author: Ian Lynagh <[email protected]>
Date:   Thu Nov 8 21:46:46 2012 +0000

    Give dynamic libraries, as well as programs, RPATHs
    
    Based on a patch from markwright in #3072.

 rules/build-package-way.mk |    4 ++--
 rules/build-prog.mk        |   25 ++++++++++++-------------
 rules/distdir-way-opts.mk  |   12 ++++++++++++
 3 files changed, 26 insertions(+), 15 deletions(-)

comment:18 Changed 3 years ago by igloo

  • Resolution set to fixed
  • Status changed from new to closed

I think we now do as much as we can, so I'm closing this ticket.

comment:19 Changed 3 years ago by PHO

  • Cc pho@… added
Note: See TracTickets for help on using tickets.