Improve link times on Windows by using LLD for linking

Currently the Windows CI builds are typically lasting over 3 hours. The primary cause of this appears to be poor performance in ld.bfd, especially when linking testsuite tests.

One option to fix this is to try using LLD for linking. Unfortunately the msys2 gcc does not support -fuse-ld=lld.

Trac metadata

Trac field	Value
Version	8.6.3
Type	Bug
TypeOfFailure	OtherFailure
Priority	high
Resolution	Unresolved
Component	Compiler
Test case
Differential revisions
BlockedBy
Related
Blocking
CC
Operating system
Architecture

mentioned in issue #12913 (closed)

changed weight to 7

added Tbug Trac import labels

I have opened an msys2 ticket requesting that gcc ship with support for -fuse-ld=lld.

added upstream label

Tamar has done a bit of working optimising ld.bfd: https://github.com/Mistuke/binutils-gdb

Replying to bgamari:

One option to fix this is to try using LLD for linking. Unfortunately the msys2 gcc does not support -fuse-ld=lld.

LLD won't magically help here. Latest improvements by Martin Storsjö (mstorsjo) have made LLD able to link some mingw **gcc**-generated code, but, alas, when assembling **GHC**-generated assembly, mingw **gas** produces (I believe this is a sort of "optimisation") non-standard relocations, which LLD is unable to deal with (honestly, I haven't checked if this is still the case, but it was definitely so a year or two ago).

For quite a bit of time I have a GHC port which works flawlessly against native Windows SDK and uses clang in MSVC mode and MS or LLD linker. After recent LLD improvements by mstorsjo I've decided to try a less intrusive approach — use clang in mingw mode and LLD linker. That required quite a bit of work and I finally managed to produce stage2 GHC executable, which appeared to be severely broken — it is unable to do anything.

TL;DR the current state of affairs is that lld **can't** serve as a drop-in replacement for ld not only when GNU binutils toolchain is used, but even when clang is used as an assembler.

Perhaps, the latter might work against mingw SDK built using LLVM tools (e.g. https://github.com/mstorsjo/llvm-mingw), but I never tried this.

LLD won't magically help here. Latest improvements by Martin Storsjö (mstorsjo) have made LLD able to link some mingw gcc-generated code, but, alas, when assembling GHC-generated assembly, mingw gas produces (I believe this is a sort of "optimisation") non-standard relocations.

Frankly I wonder how difficult it would be to add support for these relocations. Given that LLVM code is generally pretty approachable I suspect that would be the easiest path forward.

In the meantime I have been pursuing testing Tamar's binutils patch.

I think it's easy, but (as I've already mentioned) the other problems exist. Mingw binutils introduce a lot of non-standard things, which LLD didn't support at all until recently, now some support have appeared, but AFAIUI, things go best when mingw sdk is built by LLVM toolsuite, not by mingw gcc/binutils.

And a general problem is that wrong linkage bugs are *very* hard to debug, this is why I haven't ever tried to continue my mingw clang/lld experiment — it might consume an unpredictable amount of time to understand what is wrong with the generated ghc executable — all the symptoms are that the image file is invalid, since the OS can't even load it properly.

Replying to [ticket:16084#comment:165709 bgamari]:

Frankly I wonder how difficult it would be to add support for these relocations. Given that LLVM code is generally pretty approachable I suspect that would be the easiest path forward.

In the meantime I have been pursuing testing Tamar's binutils patch.

mentioned in commit bb06c6b1

mentioned in commit 942b5019

I have pushed a new branch testing a new binutils build provided by Tamar to wip/fix-windows-ci (see !495 (merged)).

@Phyx wrote (in !487 (comment 187112)),

1. Speed linking As you are well aware, linking speed is an issue. In principle BFD shouldn't be so much slower on linux than on Windows, because most of the code is generic mid-end code! so it's shared. The platform ABI differences don't account for this slowdown. My working hypothesis is that the slowdown is at the two ends of the linker. The file read and file write. My suspicion is that the Windows version has to do much more work than the linux one. A fundamental differce between linux and Windows file I/O is that linux I/O is optimized for path based APIs while Windows is Handle based. CreateFile is a relatively expensive API to perform, so everytime these POSIX functions are called on windows the file is opened and closed and you pay this expensive overhead. One thing I will try is changing the fread/fopen to mmap calls. bypassing the buffer managers etc. I'm hoping this makes up the differce.

Case in point that it's not an inherent platform issue is that LLD and link.exe are both much faster than ld.

mentioned in merge request !487 (merged)

added compiler perf label

added Windows label

mentioned in commit afe2837c

mentioned in commit fa6a0a05