Opened 2 years ago

Last modified 6 months ago

#9237 new bug

GHC not recognizing INPUT(-llibrary) in linker scripts

Reported by: mmikolajczyk Owned by:
Priority: normal Milestone:
Component: Runtime System (Linker) Version: 7.8.2
Keywords: Cc: jeff@…, hgolden, trommler
Operating System: Linux Architecture: x86_64 (amd64)
Type of failure: GHCi crash Test Case:
Blocked By: #9498, #11238 Blocking:
Related Tickets: #2615, #10046 Differential Rev(s):
Wiki Page:

Description

I have tried to build accelerate-llvm package and encountered an invalid behaviour of linker. I am using Arch Linux and GHC 7.8.2.

I have installed llvm-general-3.4.2.2 and llvm-general-pure-3.4.2.2 from this branch: https://github.com/tvh/llvm-general/tree/curatedTargetMachine, and accelerate from HEAD to common sandbox. Then I tried to build accelerate-llvm using the sandbox and got this output during building:

Building accelerate-llvm-0.15.0.0...
Preprocessing library accelerate-llvm-0.15.0.0...
[ 1 of 30] Compiling Data.Range.Range ( Data/Range/Range.hs, dist/build/Data/Range/Range.o )
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
Loading package primitive-0.5.3.0 ... linking ... done.
Loading package array-0.5.0.0 ... linking ... done.
Loading package deepseq-1.3.0.2 ... linking ... done.
Loading package old-locale-1.0.0.6 ... linking ... done.
Loading package time-1.4.2 ... linking ... done.
Loading package vector-0.10.11.0 ... linking ... done.
Loading package mwc-random-0.13.1.2 ... linking ... done.
Loading package bytestring-0.10.4.0 ... linking ... done.
Loading package containers-0.5.5.1 ... linking ... done.
Loading package transformers-0.4.1.0 ... linking ... done.
Loading package mtl-2.2.1 ... linking ... done.
Loading package text-1.1.1.3 ... linking ... done.
Loading package parsec-3.1.5 ... linking ... done.
Loading package unix-2.7.0.1 ... linking ... done.
Loading package setenv-0.1.1.1 ... linking ... done.
Loading package pretty-1.1.1.1 ... linking ... done.
Loading package template-haskell ... linking ... done.
Loading package llvm-general-pure-3.4.2.2 ... linking ... done.
Loading package utf8-string-0.3.8 ... linking ... done.
Loading package llvm-general-3.4.2.2 ... <command line>: can't load .so/.DLL for: /usr/lib/libcurses.so (-lncursesw: cannot open shared object file: No such file or directory)

After some searching I narrowed down the issue to /usr/lib/libcurses.so file. In Arch, this file contains INPUT(-lncursesw). If I change it to INPUT(libncursesw.so) or INPUT(/usr/lib/libncursesw.so) it works fine. Symlinking /usr/lib/libcurses.so to /usr/lib/libncursesw.so also works.

This bug seems to be connected to #2615. GHC still doesn't follow INPUT commands containing -llibrary form. Ld documentation allows this: If you use `INPUT (-lfile)', ld will transform the name to libfile.a, as with the command line argument `-l'. (https://sourceware.org/binutils/docs/ld/File-Commands.html)

Change History (15)

comment:1 Changed 2 years ago by rwbarton

I think GHC is trying to load /usr/lib/libcurses.so at runtime with dlopen, in order to run Template Haskell code in Data.Range.Range. It's not trying to link it with ld. So, this is a limitation of the system dlopen, and we are in "workaround" territory.

(Incidentally, my dlopen on Debian (libc 2.18-4) doesn't seem to handle either INPUT(-llibrary) or INPUT(filename).)

#2615 was from when GHC used its own custom loader rather than dlopen. I'm not eager to bring any of that back, but I guess we could try to detect a linker script if dlopen fails.

One thing (well, two things) I don't understand is where the dependency on /usr/lib/libcurses.so is coming from, and what kind of dependency it is exactly. Who is loading /usr/lib/libcurses.so: does dlopen load dependencies for us, or do we track them ourselves and load them first? Maybe we can arrange things when we build the llvm-general-3.4.2.2 library so that the dependency is on the real shared object /usr/lib/libncursesw.so.

comment:2 follow-up: Changed 2 years ago by rwbarton

Oh, hmm, I actually looked at the code and we do already inspect the error string from dlopen and try to parse a linker script. The problem is that we don't support INPUT(-llibrary). I guess we can transform the name to liblibrary.so in that case?

I still would like to avoid incurring dependencies on .sos that are really linker scripts, if possible...

comment:3 Changed 2 years ago by rwbarton

  • Milestone set to 7.8.4

comment:4 Changed 22 months ago by thoughtpolice

  • Milestone changed from 7.8.4 to 7.10.1

Moving (in bulk) to 7.10.4

comment:5 Changed 22 months ago by anjefu

  • Cc jeff@… added

comment:6 Changed 19 months ago by thoughtpolice

  • Milestone changed from 7.10.1 to 7.12.1

Moving to 7.12.1 milestone; if you feel this is an error and should be addressed sooner, please move it back to the 7.10.1 milestone.

comment:7 Changed 18 months ago by hgolden

  • Cc hgolden added

comment:8 Changed 18 months ago by hgolden

  • Summary changed from GHC not recognizing "-llibrary" form in implicit linker scripts to GHC not recognizing INPUT(-llibrary) in linker scripts

comment:9 Changed 11 months ago by ezyang

  • Component changed from Compiler to Compiler (Linking)

comment:10 in reply to: ↑ 2 ; follow-up: Changed 11 months ago by hgolden

  • Component changed from Compiler (Linking) to Runtime System (Linker)
  • Operating System changed from Unknown/Multiple to Linux
  • Type of failure changed from Compile-time crash to GHCi crash

Replying to rwbarton:

Oh, hmm, I actually looked at the code and we do already inspect the error string from dlopen and try to parse a linker script. The problem is that we don't support INPUT(-llibrary). I guess we can transform the name to liblibrary.so in that case?

I think that's what's needed. I don't run Arch Linux right now. Maybe I'll have some time to install it in a VM and try out this approach.

When I fixed #2615, the hardest part was generating a test to prove it worked. I envision the same difficulty here.

I still would like to avoid incurring dependencies on .sos that are really linker scripts, if possible...

I don't see any way to avoid this unless we find a way to call ld as a helper. Doing this would be way beyond my level of understanding.

comment:11 Changed 11 months ago by thoughtpolice

  • Milestone changed from 7.12.1 to 8.0.1

Milestone renamed

comment:12 Changed 7 months ago by trommler

  • Cc trommler added

comment:13 in reply to: ↑ 10 Changed 7 months ago by trommler

  • Blocked By 9498, 10458 added

Replying to hgolden:

I still would like to avoid incurring dependencies on .sos that are really linker scripts, if possible...

I don't see any way to avoid this unless we find a way to call ld as a helper. Doing this would be way beyond my level of understanding.

I think this could be solved by my proposal in #9498 in the static case and #10458 in the dynamic case.

comment:14 Changed 7 months ago by trommler

  • Blocked By 11238 added; 10458 removed

It was decided to not risk merging a first version of the redesign of dynamic linking for 8.0.1 and so the fix for #10458 is not going to fix the dynamic case in this ticket. #11238 is the ticket for a redesign of dynamic linking. Changing BlockedBy accordingly.

comment:15 Changed 6 months ago by thomie

  • Milestone 8.0.1 deleted
Note: See TracTickets for help on using tickets.