First noticed here. I haven't managed to boil this down to a test case with no dependencies yet, so for the time being, this requires vector. To reproduce, follow these steps:
$ git clone https://github.com/erikd/vector$ cd vector/$ cabal install --only-dependencies --enable-tests -w /opt/ghc/8.2.1/bin/ghc$ cabal configure --enable-tests -w /opt/ghc/8.2.1/bin/ghc$ cabal test
When building vector-tests-O2, GHC will stall when compiling the Tests.Vector module. On machines with modest memory allowances (e.g., the machines used on Travis CI), GHC will be killed with an out-of-memory error after trying to compile Tests.Vector for a while.
Trac metadata
Trac field
Value
Version
8.1
Type
Bug
TypeOfFailure
OtherFailure
Priority
highest
Resolution
Unresolved
Component
Compiler
Test case
Differential revisions
BlockedBy
Related
Blocking
CC
Operating system
Architecture
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Child items
0
Show closed items
No child items are currently assigned. Use child items to break down this issue into smaller parts.
In particular, limiting GHC's memory to 4 GB is sufficient:
$ bash -c 'ulimit -v 4000000; cabal test'...Preprocessing test suite 'vector-tests-O2' for vector-0.12.0.1...[6 of 7] Compiling Tests.Vector ( tests/Tests/Vector.hs, dist/build/vector-tests-O2/vector-tests-O2-tmp/Tests/Vector.o )tests/Tests/Vector.hs:9:1: warning: [-Wunused-imports] The import of ‘Data.Foldable’ is redundant except perhaps to import instances from ‘Data.Foldable’ To import instances alone, use: import Data.Foldable() |9 | import Data.Foldable (Foldable(foldMap)) | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^tests/Tests/Vector.hs:25:1: warning: [-Wunused-imports] The import of ‘Data.Monoid’ is redundant except perhaps to import instances from ‘Data.Monoid’ To import instances alone, use: import Data.Monoid() |25 | import Data.Monoid | ^^^^^^^^^^^^^^^^^^tests/Tests/Vector.hs:29:1: warning: [-Wunused-imports] The import of ‘Data.Functor.Identity’ is redundant except perhaps to import instances from ‘Data.Functor.Identity’ To import instances alone, use: import Data.Functor.Identity() |29 | import Data.Functor.Identity | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^tests/Tests/Vector.hs:438:5: warning: [-Wname-shadowing] This binding for ‘limitUnfolds’ shadows the existing binding imported from ‘Utilities’ at tests/Tests/Vector.hs:5:1-24 |438 | limitUnfolds f (theirs, ours) | ^^^^^^^^^^^^ghc: out of memory
Looks like a regular "Core program gets very large" issue. The Core program got very large in 8.0.1 too (I didn't try earlier versions) but now it gets even more very large.
I tried with the versions of GHC before and after the Join points commit since I have them lying around. Already before the Join points commit the Core program grows to be very large quickly (actually, even more so than with HEAD).
Note that #10800 (closed) is another ticket reporting a compile-time regression in vector's testsuite between GHC 7.8 and 7.10. While it was apparently resolved in GHC 8.0, we never were able to come to a conclusion on what fixed it.
I setup a test environment to build just tests/Utilities.hs, tests/Boilerplater.hs, and tests/Tests/Vector.hs with both GHC 8.3 (13131ce9 and 8.0.2. In addition to building Tests.Vector as-is, I also built it enabling each of the testcases in the tests list individually (that is, commenting out all but one). The result is interesting: While compilation on master is significantly faster than 8.0.2 in the individual cases, it is significantly slower when all of the testcases are enabled. Clearly there is something non-linear going on here. I'm on the case.
Indeed in most phases of simplification the program with all testcases enabled is roughly three to five times larger in master than it was in 8.0.2. After CorePrep, however, the program is nearly a factor of 8 larger,
It may also be interesting to note that if you sum the allocations of all of the testcases individually, you get something a bit higher than the allocations required to compile the "everything" case. I wonder if
So, there are two (possibly related) questions here:
Why did the Core get so much larger from 8.0 to 8.2?
How was GHC 8.0 able to compile the "everything" case with less than a factor of two of the allocations of the individual cases, while 8.2 requires a factor of five?
Comparing the compilation of just the Data.Vector.Vector (Bool) with GHC 8.0.2 and 8.2 is also quite interesting: starting early in simplification we see that 8.2's core is typically less than half(!) the size of 8.0's (perhaps some goodness due to early inlining?). For instance,
This remains the case throughout most of simplification; however, it seems that either Core Tidy or Core Prep cleans things up in 8.0, resulting in rather similar program sizes,
As I suspected, there appears to be a large number of duplicate top-level bindings in the code that gets passed Core Tidy in 8.2. In particular, I count at least 25 copies of the same binding,
Then grep for ^\$s\$wfoldlM in the dump-simpl output. You should see a number of top-level bindings of the type that I mention in ticket:13535#comment:135332. In my case they are all exactly of size,
It seems that the bindings in question are being excluded from CSE by CSE.noCSE. I suspect they may be join points, but I'm still trying to work out the particulars.
Indeed I've confirmed that the bindings in question are join points. I don't see any reason why top-level join points can't be CSE'd. I'll try fixing this.
Please excuse the nonsense in ticket:13535#comment:135368. My trace mislead me into believing that the bindings in question were join points when they were not (and now that I look again at this, I don't believe that such top-level join points could even arise).
The problem is instead that the not (isAlwaysActive (idInlineActivation id)) condition in noCSE fires for the bindings in question. This appears to be due to the INLINE[0] pragma applied to these bindings by worker/wrapper (namely in WorkWrap.splitFun.wrap_id).
I'm still a bit boggled by how all these identical functions get created in the first place. I have not yet had a chance to gaze at the repro case. Do you understand why?