Turn on SplitSections by default

[ezyang]

This is a general tracking bug for what is blocking us from turning on SplitSections by default, implemented in #8405. Fixing this, and mandating use of the gold linker, will help solve #11285 (split objects make linking slow), and will make ghc-split unnecessary, mooting #9832.

I encountered some misinformation while talking to people about this feature, regarding whether or not this feature only supported by Apple-style linkers (i.e. on Mac OS X)? On IRC I heard a claim made that --split-sections was only supported by Mac OS X. However, olsner pointed out that -split-sections on GHC was OUR reimplementation of the feature, and likely does not work on Mac OS X. ​https://github.com/snowleopard/shaking-up-ghc/issues/153#issuecomment-171078800 A later comment mentioned that

So, can we turn on this feature by default? Does it subsume split-objs? Does it work on all the tier-1 platforms? It would be nice if bgamari, olsner and co could weigh in here. Thanks!

Also, there are two different ways we can do this. One is we can turn it on by default in the GHC build system, so all the base libraries and GHC are built with it (but when you invoke ghc, it does NOT have split sections); the other is to have GHC always run split sections, so user code also compiles with it.

[refold]

The gold linker only supports ELF, so it can't be used on OS X and Windows.

GNU ld didn't support --gc-sections for PE/COFF ​until recently, and this feature ​still appears to be buggy according to olsner. So we'll need to fix those bugs in ld to make -split-sections work on Windows. We'll also need to ship with an unreleased version of ld.

According to the links you provided, OS X ld doesn't support --gc-sections at all, but maybe the -dead_strip option could be used.

So on ELF platforms -split-sections should be ready to replace -split-objs; further testing and investigation is necessary on Windows and OS X.

[olsner]

So on ELF platforms -split-sections should be ready to replace -split-objs; further testing and investigation is necessary on Windows and OS X.

OS X should just use the built-in subsection support.

For non-Linux ELF platforms, an issue is that --gc-sections (under that name) is only available from GNU ld, other linkers might not have it or might call it something else. That means dropping -split-objs probably would leave some platforms without dead-code stripping at all.

It should be good to go with GNU ld on non-Windows though. And for unixes without GNU ld we could get configure to check if the linker has something like --gc-sections and how to activate it. (Looks like e.g. Solaris has a -z discard-unused=sections flag.)

[awson]

I have fully working -split-sections on Windows.

To make it possible quite recently I've fixed ​a blocker bug in GHC (thanks to Phyx for putting it in).

But also we need the very latest binutils because of ​the bug in binutils I've helped to fix just recently.

And the final thing to make it all work are correct linker scripts (generated from the single pep.sc template), which solve ​section merging problem. I've incorporated it in my own patched version of binutils but I still didn't propose the corresponding patch to upstream binutils. I have no much spare time at the moment, and this patch also contains an extra bugfix for current binutils, thus some extra work is required here, but if the need is urgent I'll try to wrap it in coming days.

[refold]

Replying to awson:

I have fully working -split-sections on Windows.

That's awesome! Thank you for working on this.

[thomie]

Removing -split-objs also solves the following tickets:

• #11315
• #8964
• #8685
• #8629
• #8300
• #7637

[dobenour]

Is there anything left before we can do this?

If not I will go ahead and put a patch up on Phabricator that makes -split-objs a deprecated synonym for -split-sections.

[Phyx-]

@awson never put his patches up. So at the moment, -split-sections does not work on PE targets. So no, you can't flip it on yet.

I'm working on getting things up and running.

[dobenour]

Phyx-: Ah okay. For some reason I thought that he had.

[awson]

Well, things are absolutely OK with -split-sections for 64-bit PECOFF obj files, but there exist a critical problem for the 32-bit case.

The problem is that to realistically work with -split-sections (even to build GHC itself, particularly template-haskell package, particularly Language.Haskell.TH.Syntax module compiled in profiling mode) we need big-obj support (to have more than 32K sections per object file), but binutils has no big-obj support for 32-bit PECOFF object files.

Perhaps, the simplest solution would be to have another splitter, which would split the whole assembly to chunks with no more than 32K sections in each.

[awson]

If somebody wants to enjoy the benefits of -split-sections vs -split-objs (they are tremendous) immediately, she has the following options:

1. I could try to put linker script patches to upstream binutils. Honestly, I don't want to bother with this ATM;
2. I could publish these patches for those interested to build their own binutils;
3. I could publish linker scripts which could be used with existing binutils.

Since all options require (at least) to modify any existing build scripts, I believe that, perhaps, the option 3 might be the most welcomed one. OTOH, options 2 and 3 require more involvement than the first one.

[dobenour]

awson: could GHC have the new linker script embedded into it, so that GHC can use it automatically? GHC invokes the linker itself, so it can pass any custom linker script necessary.

[refold]

@awson Please do whatever is easiest for you. If you don't want to bother with integrating your patches into upstream binutils, maybe someone else is, so please at least open source them.

[awson]

Well, I've updated ​relevant binutils ticket with patches solving the problem.

[awson]

Linker scripts for section merging on Windows

[awson]

Scripts attached do section merging on Windows.

Caveat:

As far as I remember, when I worked on this (almost year ago) cabal (or was it GHC?) didn't respect linker flags when building library's prelinked object file (for GHCi). I may be wrong or cabal may be changed since that time, but if you will try to use attached linker scripts with existing binutils, please make sure ld receives the option feeding the linker script to it. If this problem is still there, please ask cabal (or GHC?) maintainers to fix it. Also it may be a problem that we need to use different linker scripts for different scenarios: "x" for building executables/dlls and "xr" for building prelinked object files.

[awson]

The summary (for Windows):

1. I always used patched binutils and never used separate linker scripts, thus can't comment on potential problems with the latter.
2. On 64-bit GHC I don't use -split-objs at all and use -split-sections exclusively for almost a year and I never had any problems with this. The only important thing one should remember is that we sometimes need to supply -opta-Wa,-mbig-obj to GHC to make it build correct object files (I know 2 package which require this: template-haskell when building GHC itself and haskell-src-exts). One possible option is to always use this flag if -split-sections is used.
3. -split-sections doesn't introduce any noticeable overhead when compiling and thus is dramatically faster than -split-objs when building packages, it is slightly slower when linking final executables though.
4. 64-bit GHC itself can be built in release configuration with -split-objs replaced by -split-sections absolutely with no problems (only -opta-Wa,-mbig-obj should be added to ghc-options in template-haskell cabal file). 32-bit GHC building process is less smooth and requires a special handling of template-haskell package (one needs to recompile Language.Haskell.TH.Syntax module in profiling configuration with -split-objs). Overall, building release GHC itself with -split-sections is much faster than with -split-objs.

[olsner]

I assumed the linker problem and the required fix would be much more complicated, but after reading the patches it looks like it could simply be an issue with how we name sections on Windows builds? (See also ​https://sourceware.org/bugzilla/show_bug.cgi?id=19254#c3)

That means maybe we "only" need:

• unpatched binutils (but a new enough version)
• -Wa,-mbig-objs for the assembler
• fixed section names
• a PE-compatible linker script for ld -r section merging

[awson]

It looks current binutils' linker scripts doesn't handle rodata sections at all, thus I believe we should patch binutils anyway.

Regarding section namespace separator, indeed gcc on Windows generates $ instead of ., and this, perhaps, would be correct to make GHC do the same thing, but even unpatched Windows bintuils have separate handling of .text.* sections for example. I suspect that this is because there exist another tools from GNU/Unix land, ported to Windows which also use . for section hierarchy handling.

Also, sections with $ should be sorted according to their suffix names, which, I believe is not what we want in our (GHC) case (and this is not the case for ELF files).

[Ben Gamari <ben@…>]

In 8f71d958/ghc:

Enable split sections by default where possible

On non-windows platforms with GNU ld, enable SplitSections in the GHC
build by default.

Reviewers: austin, bgamari

Reviewed By: bgamari

Subscribers: DemiMarie, thomie

Differential Revision: https://phabricator.haskell.org/D1800

GHC Trac Issues: #11445

[bgamari]

To get any further on this we would need to fix the Windows situation which won't be happening for 8.2. Bumping.

[bgamari]

This ticket won't be resolved in 8.4; remilestoning for 8.6. Do holler if you are affected by this or would otherwise like to work on it.

[bgamari]

These won't be fixed for 8.6, bumping to 8.8.

[thomie]

The build system now uses -split-sections to build GHC and the core libraries on supported platforms, see comment:21.

See #12913 and #13939 for remaining work to be done for Windows.

Closing.

[bgamari]

The new Hadrian build system has been merged. Relabeling the tickets concerning the legacy make build system to prevent confusion.