Opened 9 years ago

Last modified 16 months ago

#2346 new bug

Compilation of large source files requires a lot of RAM

Reported by: choener Owned by:
Priority: lowest Milestone:
Component: Compiler Version: 6.8.2
Keywords: Cc:
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: Compile-time performance bug Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s):
Wiki Page:

Description (last modified by simonpj)

We have automatically generated source files with up to ~20000 lines of code.

One type definition, two functions called "grammar" (cmGrammer, ghGrammar) and one or more functions called "algebra" (prettyprint, scoremax, count, *) are generated. Each function has a where-clause with 100+ locally visible function defined.

Attached, you find two files (ADPTriCombinators.lhs, RnaI.lhs) with helper-functions and three of our source files. One is the smallest example we have and the other the largest. RF00549RED.hs is reduced to only the type, one algebra and one grammar.

Copy all files into a directory and then execute:

ghc --make RF00390.hs (should work)
ghc --make RF00549.hs (WARNING: EATS ALL MEMORY)
ghc --make RF00549RED.hs (WARNING: EATS ALL MEMORY)

System Info:

Linux workstation 2.6.25-ARCH #1 SMP PREEMPT 
Fri May 16 14:52:43 CEST 2008 
i686 AMD Athlon(tm) 64 Processor 3500+ AuthenticAMD 
GNU/Linux (2 GByte RAM, 2 GByte Swap)

gcc version 4.3.0 (GCC)

Thanks, Christian Höner zu Siederdissen

Attachments (2)

ghcmem.tar.bz2 (192.4 KB) - added by choener 9 years ago.
the 5 source files
ghcmem.tar.2.bz2 (187.1 KB) - added by morabbin 4 years ago.
Make the test files compile with modern GHC

Download all attachments as: .zip

Change History (24)

Changed 9 years ago by choener

Attachment: ghcmem.tar.bz2 added

the 5 source files

comment:1 Changed 9 years ago by ajd

Type: bugcompile-time performance bug

comment:2 Changed 9 years ago by simonpj

Description: modified (diff)
difficulty: Unknown

comment:3 Changed 9 years ago by igloo

Milestone: 6.10.1

comment:4 Changed 9 years ago by igloo

I'm not sure if this is the only problem, but one issue is that in a module like this:

module Test where

f alg1 alg2 = (a1, a2, a3, a4, a5)
    where
      (a1_1, a2_1, a3_1, a4_1, a5_1) = alg1
      (a1_2, a2_2, a3_2, a4_2, a5_2) = alg2

      a1 x (y,z) = (a1_1 x y, a1_2 x z)
      a2 x (y,z) = (a2_1 x y, a2_2 x z)
      a3 x (y,z) = (a3_1 x y, a3_2 x z)
      a4 x (y,z) = (a4_1 x y, a4_2 x z)
      a5 x (y,z) = (a5_1 x y, a5_2 x z)

we end up with a quadratic amount of code (and also, I think, a quadratic amount of type)

e.g. in the above module we get 5 things like this:

       \ (x_ag3 :: t5_alb) (ds_dn0 :: (t6_ale, t7_alf)) ->
         case ds_dn0 of wild_B1 { (y_ag5, z_ag7) ->
         (case w_soy
          of wild1_XO { (a1_1_akE, a2_1_akG, a3_1_akI, a4_1_akK, a5_1_akM) ->
          a2_1_akG x_ag3 y_ag5
          },
          case w1_soz
          of wild1_XO { (a1_2_ako, a2_2_akq, a3_2_aks, a4_2_aku, a5_2_akw) ->
          a2_2_akq x_ag3 z_ag7
          })
         },

comment:5 Changed 9 years ago by igloo

Milestone: 6.10.16.10.2

comment:6 Changed 8 years ago by igloo

Milestone: 6.10.26.12.1

comment:7 Changed 8 years ago by simonmar

Type of failure: Compile-time performance bug

comment:8 Changed 8 years ago by igloo

Milestone: 6.12.16.12 branch

comment:9 Changed 7 years ago by igloo

Milestone: 6.12 branch6.12.3

comment:10 Changed 7 years ago by igloo

Milestone: 6.12.36.14.1
Priority: normallow

comment:11 Changed 6 years ago by igloo

Milestone: 7.0.17.0.2

comment:12 Changed 6 years ago by igloo

Milestone: 7.0.27.2.1

comment:13 Changed 6 years ago by igloo

Milestone: 7.2.17.4.1

comment:14 Changed 5 years ago by igloo

Milestone: 7.4.17.6.1
Priority: lowlowest

comment:15 Changed 5 years ago by igloo

Milestone: 7.6.17.6.2

comment:16 Changed 4 years ago by morabbin

These all compile on an 8Gb RAM Macbook Pro without running out of memory. RF00549.hs produces a 52Mb .o! Sure, they take a few hours to compile, but they don't fail, nor do they appear to eat all the memory (i.e., my machine was not rendered useless). It seemed quite acceptable to me, given the sizes of the files.

Changed 4 years ago by morabbin

Attachment: ghcmem.tar.2.bz2 added

Make the test files compile with modern GHC

comment:17 Changed 3 years ago by thoughtpolice

Milestone: 7.6.27.10.1

Moving to 7.10.1.

comment:18 Changed 2 years ago by thomie

Architecture: x86Unknown/Multiple
Operating System: LinuxUnknown/Multiple

RF00549.hs still fails to compile with a 2GB RAM limit on x86_64 Linux (ulimit -v 2000000).

This ticket is listed as desugaring let-bindings on Status/SLPJ-Tickets.

comment:19 Changed 2 years ago by thoughtpolice

Milestone: 7.10.17.12.1

Moving to 7.12.1 milestone; if you feel this is an error and should be addressed sooner, please move it back to the 7.10.1 milestone.

comment:20 Changed 2 years ago by thoughtpolice

Moving to 7.12.1 milestone; if you feel this is an error and should be addressed sooner, please move it back to the 7.10.1 milestone.

comment:21 Changed 21 months ago by thoughtpolice

Milestone: 7.12.18.0.1

Milestone renamed

comment:22 Changed 16 months ago by thomie

Milestone: 8.0.1
Note: See TracTickets for help on using tickets.