#12227 closed bug (fixed)

regression: out of memory with -O2 -ddump-hi on a complex INLINE function

Reported by: j6carey Owned by:
Priority: normal Milestone: 8.0.2
Component: Compiler (CodeGen) Version: 8.0.1
Keywords: Cc:
Operating System: Linux Architecture: x86_64 (amd64)
Type of failure: Compile-time performance bug Test Case: perf/compiler/T12227
Blocked By: Blocking:
Related Tickets: Differential Rev(s): Phab:D2397
Wiki Page:

Description

Summary:

Unlike GHC 7.10.2, when GHC 8.0.1 tries to compile a certain complex INLINE function with -O2 -ddump-hi, it runs out of memory. Dropping either -O2 or -ddump-hi enables successful compilation.

Oddly enough, triggering the issue requires using the function argument indirectly through a variable bound by a "where" clause, even though it is obvious to a human reader that the indirection is trivial.

(Even after simplifying the original code to work around the out of memory issue, that indirection greatly increases the size of the ".dump-hi" file. It might be that the "out of memory" would not happen if one had sufficient RAM.)

Using INLINABLE instead of INLINE seems to result in a much smaller RHS being recorded for the function in question, and does not trigger the problem. (Perhaps INLINABLE records a more optimized RHS than does INLINE, though that is not obvious to me from the GHC User's Guide.)

The attached files contain a reduced version of the original code that still triggers the problem. (Of course, the reduced version is rather artificial and pointless, other than to demonstrate the problem.) Probably someone with sufficient understanding could shrink the example further.

Please note that I have worked around the issue, and so I am not blocked by this problem, at least not currently.

Attachments:

  • crash.bash
  • Crash.hs

Observed behavior:

On 64-bit x86 Ubuntu 16.04 LTS, with the attached files in the current directory, GHC 7.10.2 succeeds:

$ ./crash.bash
The Glorious Glasgow Haskell Compilation System, version 7.10.2
[1 of 1] Compiling Crash            ( Crash.hs, Crash.o )

whereas GHC 8.0.1 runs out of memory (or wedges my system if I comment out both "ulimit" commands in the script):

$ ./crash.bash
The Glorious Glasgow Haskell Compilation System, version 8.0.1
[1 of 1] Compiling Crash            ( Crash.hs, Crash.o )
ghc: out of memory

Attachments (2)

crash.bash (151 bytes) - added by j6carey 12 months ago.
Script to compile Crash.hs with arguments that trigger the problem.
Crash.hs (6.4 KB) - added by j6carey 12 months ago.
Source code to be compiled in order to trigger the problem.

Download all attachments as: .zip

Change History (13)

Changed 12 months ago by j6carey

Attachment: crash.bash added

Script to compile Crash.hs with arguments that trigger the problem.

Changed 12 months ago by j6carey

Attachment: Crash.hs added

Source code to be compiled in order to trigger the problem.

comment:1 Changed 12 months ago by j6carey

Let me add that the build tool "stack" seems to pass "-ddump-hi" to GHC, which is why I noticed this issue.

comment:2 Changed 12 months ago by simonpj

I've learned a bit about this example.

  1. It builds big, deeply-nested coercions.
  1. The actual memory exhaustion comes from the pretty printer. Here's a dump from -ticky of GHC itself, with the -ddump-hi on:
        Entries      Alloc    Alloc'd  Non-void Arguments      STG Name
    --------------------------------------------------------------------------------
       24320676 1066014032          0   4 MEiM                 $waboveNest{v rb81} (ghc-8.1:Pretty) (fun)
       19400250  621778256          0   3 MEM                  beside{v r16q} (ghc-8.1:Pretty) (fun)
        2773145  143762232          0   4 EMiL                 ghc-8.1:Pretty.$wsep1{v rb7d} (fun)
        3466978  117540632          0   2 >L                   base-4.9.0.0:GHC.Base.map{v 01X} (fun)
        4143110  114461032          0   1 M                    oneLiner{v r16K} (ghc-8.1:Pretty) (fun)
        1939310   97813720          0   1 M                    ghc-8.1:Coercion.coercionKind_go{v rrbv} (fun)
        1475028   66514184          0   3 i.M                  containers-0.5.7.1@containers-0.5.7.1:Data.IntMap.Base.$winsert{v rnBj} (fun)
         902398   65925152          0   2 LL                   ghc-8.1:Util.$wsplitAtList{v rhOj} (fun)
    
    So clearly the pretty printer is behaving very badly when given deeply nested things to print.
  1. Aside from that, a tremendous amount of time is spent in coercionKind.

comment:3 Changed 12 months ago by thomie

The pretty pinter issue might be the same issue as https://github.com/haskell/pretty/issues/32. I can have a look later this week.

comment:4 Changed 12 months ago by simonpj

Sounds very similar. Thanks for taking a look.

comment:5 Changed 12 months ago by thomie

Differential Rev(s): Phab:D2397
Status: newpatch
Test Case: perf/compiler/T12227

The fix for that pretty issue also fixes this bug. I don't understand why exactly, but I'm happy to apply it.

comment:6 Changed 12 months ago by bgamari

Well caught, thomie!

comment:7 Changed 12 months ago by Ben Gamari <ben@…>

In 89a8be71/ghc:

Pretty: remove a harmful $! (#12227)

This is backport of [1] for GHC's copy of Pretty. See Note [Differences
between libraries/pretty and compiler/utils/Pretty.hs].

[1] http://git.haskell.org/packages/pretty.git/commit/bbe9270c5f849a5bb74c9166a5f4202cfb0dba22
    https://github.com/haskell/pretty/issues/32
    https://github.com/haskell/pretty/pull/35

Reviewers: bgamari, austin

Reviewed By: austin

Differential Revision: https://phabricator.haskell.org/D2397

GHC Trac Issues: #12227

comment:8 Changed 11 months ago by thomie

Milestone: 8.2.1
Status: patchmerge
Type of failure: None/UnknownCompile-time performance bug

To merge this to the 8.0 branch, you'll need 372dbc4e78abfb6b5d72c0fea27a1c858c5cd797 first.

comment:9 Changed 11 months ago by thomie

Milestone: 8.2.18.0.2

comment:10 Changed 10 months ago by Ben Gamari <ben@…>

In 2756af8/ghc:

Pretty: remove a harmful $! (#12227)

This is backport of [1] for GHC's copy of Pretty. See Note [Differences
between libraries/pretty and compiler/utils/Pretty.hs].

[1] http://git.haskell.org/packages/pretty.git/commit/bbe9270c5f849a5bb74c9166a5f4202cfb0dba22
    https://github.com/haskell/pretty/issues/32
    https://github.com/haskell/pretty/pull/35

Reviewers: bgamari, austin

Reviewed By: austin

Differential Revision: https://phabricator.haskell.org/D2397

GHC Trac Issues: #12227

(cherry picked from commit 89a8be71a3715c948cebcb19ac81f84da0e6270e)

comment:11 Changed 10 months ago by bgamari

Resolution: fixed
Status: mergeclosed
Note: See TracTickets for help on using tickets.