Currently the Windows CI builds are typically lasting over 3 hours. The primary cause of this appears to be poor performance in ld.bfd, especially when linking testsuite tests.
One option to fix this is to try using LLD for linking. Unfortunately the msys2 gcc does not support -fuse-ld=lld.
Trac metadata
Trac field
Value
Version
8.6.3
Type
Bug
TypeOfFailure
OtherFailure
Priority
high
Resolution
Unresolved
Component
Compiler
Test case
Differential revisions
BlockedBy
Related
Blocking
CC
Operating system
Architecture
Edited
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Child items
0
Show closed items
No child items are currently assigned. Use child items to break down this issue into smaller parts.
One option to fix this is to try using LLD for linking. Unfortunately the msys2 gcc does not support -fuse-ld=lld.
LLD won't magically help here. Latest improvements by Martin Storsjö (mstorsjo) have made LLD able to link some mingw **gcc**-generated code, but, alas, when assembling **GHC**-generated assembly, mingw **gas** produces (I believe this is a sort of "optimisation") non-standard relocations, which LLD is unable to deal with (honestly, I haven't checked if this is still the case, but it was definitely so a year or two ago).
For quite a bit of time I have a GHC port which works flawlessly against native Windows SDK and uses clang in MSVC mode and MS or LLD linker. After recent LLD improvements by mstorsjo I've decided to try a less intrusive approach — use clang in mingw mode and LLD linker. That required quite a bit of work and I finally managed to produce stage2 GHC executable, which appeared to be severely broken — it is unable to do anything.
TL;DR the current state of affairs is that lld **can't** serve as a drop-in replacement for ld not only when GNU binutils toolchain is used, but even when clang is used as an assembler.
LLD won't magically help here. Latest improvements by Martin Storsjö (mstorsjo) have made LLD able to link some mingw gcc-generated code, but, alas, when assembling GHC-generated assembly, mingw gas produces (I believe this is a sort of "optimisation") non-standard relocations.
Frankly I wonder how difficult it would be to add support for these relocations. Given that LLVM code is generally pretty approachable I suspect that would be the easiest path forward.
In the meantime I have been pursuing testing Tamar's binutils patch.
I think it's easy, but (as I've already mentioned) the other problems exist. Mingw binutils introduce a lot of non-standard things, which LLD didn't support at all until recently, now some support have appeared, but AFAIUI, things go best when mingw sdk is built by LLVM toolsuite, not by mingw gcc/binutils.
And a general problem is that wrong linkage bugs are *very* hard to debug, this is why I haven't ever tried to continue my mingw clang/lld experiment — it might consume an unpredictable amount of time to understand what is wrong with the generated ghc executable — all the symptoms are that the image file is invalid, since the OS can't even load it properly.
Frankly I wonder how difficult it would be to add support for these relocations. Given that LLVM code is generally pretty approachable I suspect that would be the easiest path forward.
In the meantime I have been pursuing testing Tamar's binutils patch.
Speed linking
As you are well aware, linking speed is an issue. In principle BFD shouldn't be so much slower on linux than on Windows, because most of the code is generic mid-end code! so it's shared.
The platform ABI differences don't account for this slowdown. My working hypothesis is that the slowdown is at the two ends of the linker. The file read and file write. My suspicion is
that the Windows version has to do much more work than the linux one. A fundamental differce between linux and Windows file I/O is that linux I/O is optimized for path based APIs while
Windows is Handle based. CreateFile is a relatively expensive API to perform, so everytime these POSIX functions are called on windows the file is opened and closed and you pay this
expensive overhead. One thing I will try is changing the fread/fopen to mmap calls. bypassing the buffer managers etc. I'm hoping this makes up the differce.
Case in point that it's not an inherent platform issue is that LLD and link.exe are both much faster than ld.