ghc-pkg misbehaves after possible miscompilation on m68k and sh4

changed milestone to %8.6.1

changed weight to 5

I think I can reproduce it in qemu-sh4 on ghc-HEAD. Will look at it.

$ file utils/ghc-pkg/dist-install/build/tmp/ghc-pkg
utils/ghc-pkg/dist-install/build/tmp/ghc-pkg: ELF 32-bit LSB executable, Renesas SH, version 1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 3.2.0, not stripped

$ LD_LIBRARY_PATH="$(pwd)/rts/dist/build:"$(for d in $(find -name '*.so' | fgrep dist-install); do dirname "$(pwd)/$d"; done | tr '$\n' ':') utils/ghc-pkg/dist-install/build/tmp/ghc-pkg --version
GHC package manager version 8.7.20180704

$ LD_LIBRARY_PATH="$(pwd)/rts/dist/build:"$(for d in $(find -name '*.so' | fgrep dist-install); do dirname "$(pwd)/$d"; done | tr '$\n' ':') utils/ghc-pkg/dist-install/build/tmp/ghc-pkg --version | cat

Here is the minimal reproducer:

-- a.hs
import System.IO

main = do
    hSetBuffering stdout (BlockBuffering Nothing)
    hPutStrLn stdout "hello"

$ ghc --make a.hs -O1 -dynamic
$ ./a
<no output>

Confirmed locally that:

broken: sh4 and m68k
seem to work: x86_64, mipsn32, powerpc, powerpc64, sparc

I have not got to the bottom of the bug yet but on the way there. Somehow flushStdHandles fails to flush stdout when (and only when) in full buffered mode.

A few observations:

the trigger needs both '''-O1' and -dynamic
both sh4 and m68k are arches with 2-byte instruction width

I'll keep debugging.

Replying to [ticket:15338#comment:156574 slyfox]:

I'll keep debugging.

I think I understand breakage mechanics now:

The example above breaks because stdout haskell closure is evaluated twice:

once in the test binary
once in shared library

and creates two objects. hFlush is called only for second (shared library) object because it is called from there.

Duplication happens because base_GHCziIOziHandleziFD_stdout_closure (closure behind System.IO.stdout object) is copied by the linker COPY relocation:

$ objdump -R -D bug/ghc-pkg

00415248 <base_GHCziIOziHandleziFD_stdout_closure>:
        ...
                        415248: R_SH_COPY       base_GHCziIOziHandleziFD_stdout_closure

It should be just an external reference like R_SH_GLOB_DAT instead.

Normally COPY relocations are used only for immutable (const in C land) data. But _closures are mutable. I'll double-check generated C code and file a toolchain bug upstream.

Replying to [ticket:15338#comment:156583 slyfox]:

Normally COPY relocations are used only for immutable (const in C land) data. But _closures are mutable. I'll double-check generated C code and file a toolchain bug upstream.

James (jrtc27) pointed out that COPY relocations are fine for mutable data as long as shared library allows interposition of the symbol. James also noted that GHC uses -Bsymbolic which forbids symbol interposition and binds global symbols to library's definition.

-Bsymbolic is set in GHCs driver: http://git.haskell.org/ghc.git/blob/HEAD:/compiler/main/SysTools.hs#l550

Thus smaller C-only reproducer that illustrates the problem is the following:

/* lib.c: */
int things[] = { 99,98,97,96 };
int shlib_f(int i) {
    return things[i];
}

/* bin.c */
#include <stdio.h>

/* declarations from lib.c */
extern int things[];
int shlib_f(int i);

int main() {
    printf("main  (before store): things[0] = %i\n", things[0]);
    printf("shlib (before store): things[0] = %i\n", shlib_f(0));
    things[0] = 45;
    /* check if library sees 'things' changed. It should */
    printf("main   (after store): things[0] = %i\n", things[0]);
    printf("shlib  (after store): things[0] = %i\n", shlib_f(0));
    return 0;
}

#/bin/bash

# a.sh

#cross=sh4-unknown-linux-gnu-
cc=${cross}gcc
cflags="-O1 -Wall"

$cc $cflags -shared -fPIC                     lib.c -o libbug.so  -Wl,-Bsymbolic
$cc $cflags         -fno-PIC -fno-PIE -no-pie bin.c -o bug.no-pie -L. -lbug -Wl,-rpath=.
$cc $cflags            -fPIC    -fPIE    -pie bin.c -o bug.pie    -L. -lbug -Wl,-rpath=.

echo "target: ${cross}; emulator=${emulator}; no-pie:"
${emulator} ./bug.no-pie
echo "target: ${cross}; emulator=${emulator}; pie:"
${emulator} ./bug.pie

Here we define a shared library with things global symbol and shlib_f() that refers to that global symbol. Here is how things break (even on x86_64):

$ cross=x86_64-pc-linux-gnu- emulator= ./a.sh

target: x86_64-pc-linux-gnu-; emulator=; no-pie:
main  (before store): things[0] = 99
shlib (before store): things[0] = 99
main   (after store): things[0] = 45
shlib  (after store): things[0] = 99
target: x86_64-pc-linux-gnu-; emulator=; pie:
main  (before store): things[0] = 99
shlib (before store): things[0] = 99
main   (after store): things[0] = 45
shlib  (after store): things[0] = 99

Note: the value of things[0] disagrees between binary and library copy. That's how we get stdout closure evaluated twice. It matter because stdout is devined via unsafePerformIO:

-- libraries/base/GHC/IO/Handle/FD.hs
stdout :: Handle
{-# NOINLINE stdout #-}
stdout = unsafePerformIO $ do
   -- ToDo: acquire lock
   setBinaryMode FD.stdout
   enc <- getLocaleEncoding
   mkHandle FD.stdout "<stdout>" WriteHandle True (Just enc)
                nativeNewlineMode{-translate newlines-}
                (Just stdHandleFinalizer) Nothing

Here we effectively allocate the buffer cache as many times as stdout is evaluated. This breaks the invariant of unsafePerformIO being evaluated only once.

Replying to [ticket:15338#comment:156619 slyfox]:

Replying to [ticket:15338#comment:156583 slyfox]:

Normally COPY relocations are used only for immutable (const in C land) data. But _closures are mutable. I'll double-check generated C code and file a toolchain bug upstream.

James (jrtc27) pointed out that COPY relocations are fine for mutable data as long as shared library allows interposition of the symbol. James also noted that GHC uses -Bsymbolic which forbids symbol interposition and binds global symbols to library's definition.

Even for immutable data it can pose problematic if you perform address comparisons, but you're right that mutable data can lead to more problems.

$ cross=x86_64-pc-linux-gnu- emulator= ./a.sh

target: x86_64-pc-linux-gnu-; emulator=; no-pie:
main  (before store): things[0] = 99
shlib (before store): things[0] = 99
main   (after store): things[0] = 45
shlib  (after store): things[0] = 99
target: x86_64-pc-linux-gnu-; emulator=; pie:
main  (before store): things[0] = 99
shlib (before store): things[0] = 99
main   (after store): things[0] = 45
shlib  (after store): things[0] = 99

Note: the value of things[0] disagrees between binary and library copy.

Actually x86_64 is more unusual here in that PIE is *also* broken by this. Most architectures will fall back on the normal GOT mechanisms for PIE, but on x86_64 RIP-relative addressing is available, giving a cheaper position-independent way to access globals (and of course it's still fine for PIE because we can still use COPY relocations). If you were to run that for i386 or a traditional RISC architecture, you would see that PIE does not exhibit the bug.

Surely the way forward is to ditch -Bsymbolic, and ensure that whatever non-PIC patterns exist (as mentioned by https://ghc.haskell.org/trac/ghc/wiki/Commentary/PositionIndependentCode#LinkingonELF) in the DSOs produced are fixed? Presumably the only non-PIC bits are whatever would work for PIE (otherwise how on earth can you have a variable image base) and must be PC-relative? For code that should be fine (you could even use -Bsymbolic-functions and I believe it would work), and for data there can't be many instances. In fact, I think -Bsymbolic might well work just fine for NCG as it already tries to avoid COPY relocations (does it always successfully avoid them?), in which case we could just drop -Bsymbolic when compiling via C (GCC should just do the right thing if you give it valid C). Thoughts?

Trac metadata

Trac field	Value
CC	slyfox → jrtc27, slyfox

Indeed we had similar issues on ARM resulting in #4210 (closed).

Sent https://phabricator.haskell.org/D4959 for review (remove -Bsymbolic from UNREG targets first).

Trac metadata

Trac field	Value
Differential revisions	- → D4959

mentioned in commit 8ec48990

Some packages like haskell-ansi-wl-pprint still seem to have issues though:

Running debian/hlibrary.setup configure --ghc -v2 --package-db=/var/lib/ghc/package.conf.d --prefix=/usr --libdir=/usr/lib/haskell-packages/ghc/lib --libexecdir=/usr/lib --builddir=dist-ghc --ghc-option=-optl-Wl\,-z\,relro --haddockdir=/usr/lib/ghc-doc/haddock/ansi-wl-pprint-0.6.8.2/ --datasubdir=ansi-wl-pprint --htmldir=/usr/share/doc/libghc-ansi-wl-pprint-doc/html/ --enable-library-profiling
Configuring ansi-wl-pprint-0.6.8.2...
hlibrary.setup: failed to parse output of 'ghc-pkg dump'
make: *** [/usr/share/cdbs/1/class/hlibrary.mk:142: configure-ghc-stamp] Error 1
dpkg-buildpackage: error: debian/rules build-arch subprocess returned exit status 2

This is with the patch tested on GHC 8.2.2 with the patch and with optimization set to -O0.

Ok, looks like just adding the patch but not changing the optimization level addresses the build issue with haskell-ansi-wl-pprint.

changed milestone to %8.8.1

These won't be fixed for in GHC 8.6.

changed milestone to %8.10.1

Bumping milestones of low-priority tickets.

added incorrect runtime result label

added Pnormal label

removed milestone

Is this issue closed? This is the only reference I find towards ghc_6.rsp: commitBuffer: invalid argument (invalid character).

I get this error while compiling a small program on Windows with non-ASCII user:

D:\tools
λ ghc -o tools.exe tools.hs
Linking tools.exe ...
C:\Users\Евгений\AppData\Local\Temp\ghc8488_0\ghc_3.rsp: commitBuffer: invalid argument (invalid character)

I installed Haskell with choco, as suggested here. With choco package manager it is hard to avoid the default path, which has a non-latin username.

mentioned in issue #18332 (closed)

I think it's fixed, yes. You vry likely have an unrelated problem to -Bsymbolic UNREG failures. Please file a new bug.

Released long ago. Closing.

closed

Trac field	Value
Version	8.4.3
Type	Bug
TypeOfFailure	OtherFailure
Priority	normal
Resolution	Unresolved
Component	Compiler
Test case
Differential revisions
BlockedBy
Related
Blocking
CC
Operating system
Architecture

ghc-pkg misbehaves after possible miscompilation on m68k and sh4

Child items 0

Activity