LLVM Backend generates assembly unparsable by gnu toolchains

module Main where

import System.Random
import qualified Data.Vector as V

g1 = mkStdGen 0 :: StdGen
v2 = take 10000 $ randoms g1 :: [Int]

main :: IO ()
main = do
    undefined

$ /e/ghc_regSpill/inplace/bin/ghc-stage2.exe Repro.hs -O2 -fllvm -O2
Loaded package environment from C:\ghc\msys64\home\Andi\tmp\minmax\unpred\.ghc.environment.x86_64-mingw32-8.7.20190215
[1 of 1] Compiling Main             ( Repro.hs, Repro.o )
C:\\ghc\\msys64\\tmp\\ghc385768_0\\ghc_6.s: Assembler messages:
C:\\ghc\\msys64\\tmp\\ghc385768_0\\ghc_6.s:354: Error: junk at end of line, first unrecognized character is `,'
`gcc.exe' failed in phase `Assembler'. (Exit code: 1)

I've reproduced this with ghc 8.4/8.6 and HEAD using llvm 7.0.1 on windows.

It only occurs with -O2.

The issues arise from these directives:

	.section	.rdata,"dr",discard,__xmm@000000000000004e0000000000000000

Trac metadata

Trac field	Value
Version	8.9
Type	Bug
TypeOfFailure	OtherFailure
Priority	high
Resolution	Unresolved
Component	Compiler (LLVM)
Test case
Differential revisions
BlockedBy
Related
Blocking
CC
Operating system	Unknown/Multiple
Architecture

changed weight to 7

added LLVM backend Tbug Trac import labels

As far as I can tell the issue is that the format used is only supported for ELF.

See https://sourceware.org/binutils/docs/as/Section.html

Assuming that is the reason hopefully we just need to pass an appropriate flag to llvm.

Trac metadata

Trac field	Value
Operating system	Unknown/Multiple → Windows

Attached file Repro.ll (download).

Steps to reproduce.

Andi@Horzube MINGW64 ~/tmp/minmax/llvm
$ opt -O3 Repro.ll > opt.bc

Andi@Horzube MINGW64 ~/tmp/minmax/llvm
$ llc opt.bc -o llc.s -O3

Andi@Horzube MINGW64 ~/tmp/minmax/llvm
$ as llc.s
llc.s: Assembler messages:
llc.s:367: Error: junk at end of line, first unrecognized character is `,'

The issue here is that the target "x86_64-unknown-windows" generates .section directives which are not supported for pe output by the assembler.

As far as I can tell however "x86_64-unknown-windows" (or one of the synonyms) IS the proper target. So this seems to be an issue with llvm.

$ llc --version
LLVM (http://llvm.org/):
  LLVM version 7.0.1
  Optimized build.
  Default target: x86_64-w64-mingw32
  Host CPU: skylake

  Registered Targets:
    aarch64    - AArch64 (little endian)
    aarch64_be - AArch64 (big endian)
    amdgcn     - AMD GCN GPUs
    arm        - ARM
    arm64      - ARM64 (little endian)
    armeb      - ARM (big endian)
    bpf        - BPF (host endian)
    bpfeb      - BPF (big endian)
    bpfel      - BPF (little endian)
    hexagon    - Hexagon
    lanai      - Lanai
    mips       - Mips
    mips64     - Mips64 [experimental]
    mips64el   - Mips64el [experimental]
    mipsel     - Mipsel
    msp430     - MSP430 [experimental]
    nvptx      - NVIDIA PTX 32-bit
    nvptx64    - NVIDIA PTX 64-bit
    ppc32      - PowerPC 32
    ppc64      - PowerPC 64
    ppc64le    - PowerPC 64 LE
    r600       - AMD GPUs HD2XXX-HD6XXX
    sparc      - Sparc
    sparcel    - Sparc LE
    sparcv9    - Sparc V9
    systemz    - SystemZ
    thumb      - Thumb
    thumbeb    - Thumb (big endian)
    x86        - 32-bit X86: Pentium-Pro and above
    x86-64     - 64-bit X86: EM64T and AMD64
    xcore      - XCore

Reported upstream as #40824.

added upstream label

$ as --version
GNU assembler (GNU Binutils) 2.30
Copyright (C) 2018 Free Software Foundation, Inc.
This program is free software; you may redistribute it under the terms of
the GNU General Public License version 3 or later.
This program has absolutely no warranty.
This assembler was configured for a target of `x86_64-w64-mingw32'.

Windows 10 Pro - Build 17763 (64 Bit)

Going by the answers upstream our options are:

• Switch to the llvm assembler llvm-mc for the llvm backend.
• Hope binutils implements the llvm extensions eventually.
• Go back to LLVM 6

I don't think it would be wise to let this issue hold us back on an old llvm release. However, the other options aren't much better. Ideally we would just use llvm-mc but this is easier said than done since we use invoke the assembler through gcc. Using as directly is complicated by the fact that we would then be responsible for running cpp ourselves.

The problem with mc is that we can get assembly with CPP directives in it:

8:52 PM AndreasK> bgamari: our generated assembly contains CPP directives?
8:55 PM <bgamari> no, but a TH splice may have added an assembler source file that does

We could (depending on the backend used) invoke gcc on the generated assembly just for CPP and the pass the result on to as/llvm-mc depending on the backend.

AndreasK> bgamari: We could pipe assembly through the gcc CPP, and then pipe the result through as/llvm-mc depending on backend though right?
9:00 PM <bgamari> AndreasK, yes
9:00 PM we already have support for invoking the preprocessor
9:00 PM so really this is likely just a matter of plumbing

Given that LLVM will likely keep things specific to their assembler when it suits them I don't think we can count on this being resolved through upstream changes.

So I'm reopening this. Just needs someone to dive into the plumbing.

removed upstream label

added code generation label

added Windows label

added Phigh label

changed milestone to %8.8.2

added Phighest label and removed Phigh label

Bumping priority as this effectively breaks -fllvm on Windows.

It turns out that we already have a special case to handle this exact problem under Darwin. See https://gitlab.haskell.org/ghc/ghc/blob/8ea316da1dec77f20c742215874f8884b6e20558/compiler/main/DriverPipeline.hs#L1371. We should just use this exact same hack on Windows (and perhaps Linux too?)

Can confirm, still broken on llvm 9. I look into it.

assigned to @AndreasK

Using llvm-mc works, but we need to explicitly use the tripple x86_64-w64-windows-mingw32 for it to accept the assembly it emits.

Doing so we can either add llvm-mc support, or require clang. Both work when passed the right target tripple.

This is a regression in %8.8.1. However, I don't think this is affecting enough people to prioritize for %8.8.2. Let's try to fix for %8.10.1, however.

changed milestone to %8.10.1

changed milestone to %9.0.1

marked this issue as related to #17777 (closed)

mentioned in issue #17777 (closed)

I have been and will be occupied with other things for the near future. If anyone want's to pick this up feel free.

unassigned @AndreasK

changed milestone to %9.2.1

changed milestone to %9.4.1

changed title from LLVM Backend generates invalid assembly. to LLVM Backend generates assembly unparsable by gnu toolchains

mentioned in merge request !7448 (closed)

Happily, this will be resolved with the move to clang for our native toolchain in !7448 (closed).

mentioned in issue #21019 (closed)

marked this issue as related to #21019 (closed)

assigned to @bgamari

Relabel as higher priority.

added Phigh label and removed LLVM backend Phighest Tbug Trac import Windows code generation labels

For what it's worth, we now use a clang-based toolchain on Windows (see #21019 (closed)), which avoids this particular issue on that platform. However, fixing this in general will require using an LLVM-based toolchain for assembling code produced by the LLVM backend on all platforms until we can eliminate the LLVM mangler (#11138 ).

So should we remove the milestone now @bgamari that the main issue has been fixed?

removed milestone %9.4.1

mentioned in issue #22427 (closed)

mentioned in issue #12798

This has come up again in the context of FMA instructions on x86-64. It appears that LLVM generates FMA syntax that GNU as does not understand. I suspect the solution here needs to be to move to using llvm-as when generating code via the LLVM backend.

I don't think it's to do with FMA instructions. The assembler sees a .section directive with ".data.ZCMain_main_closure$def","awR",@progbits but gcc assembler doesn't understand R.

mentioned in commit d25857e1

mentioned in merge request !12005 (closed)

mentioned in commit 1fd1012c

mentioned in commit 90737ca9

mentioned in commit 40499fc4

mentioned in commit 836264fb

closed with commit ab533e71

mentioned in commit ab533e71

mentioned in issue ghcup-ci#3