Opened 6 years ago

Last modified 15 months ago

#2346 new bug

Compilation of large source files requires a lot of RAM

Reported by: choener Owned by:
Priority: lowest Milestone: 7.6.2
Component: Compiler Version: 6.8.2
Keywords: Cc:
Operating System: Linux Architecture: x86
Type of failure: Compile-time performance bug Difficulty: Unknown
Test Case: Blocked By:
Blocking: Related Tickets:

Description (last modified by simonpj)

We have automatically generated source files with up to ~20000 lines of code.

One type definition, two functions called "grammar" (cmGrammer, ghGrammar) and one or more functions called "algebra" (prettyprint, scoremax, count, *) are generated. Each function has a where-clause with 100+ locally visible function defined.

Attached, you find two files (ADPTriCombinators.lhs, RnaI.lhs) with helper-functions and three of our source files. One is the smallest example we have and the other the largest. RF00549RED.hs is reduced to only the type, one algebra and one grammar.

Copy all files into a directory and then execute:

ghc --make RF00390.hs (should work)
ghc --make RF00549.hs (WARNING: EATS ALL MEMORY)
ghc --make RF00549RED.hs (WARNING: EATS ALL MEMORY)

System Info:

Linux workstation 2.6.25-ARCH #1 SMP PREEMPT 
Fri May 16 14:52:43 CEST 2008 
i686 AMD Athlon(tm) 64 Processor 3500+ AuthenticAMD 
GNU/Linux (2 GByte RAM, 2 GByte Swap)

gcc version 4.3.0 (GCC)

Thanks,
Christian Höner zu Siederdissen

Attachments (2)

ghcmem.tar.bz2 (192.4 KB) - added by choener 6 years ago.
the 5 source files
ghcmem.tar.2.bz2 (187.1 KB) - added by morabbin 15 months ago.
Make the test files compile with modern GHC

Download all attachments as: .zip

Change History (18)

Changed 6 years ago by choener

the 5 source files

comment:1 Changed 6 years ago by ajd

  • Type changed from bug to compile-time performance bug

comment:2 Changed 6 years ago by simonpj

  • Description modified (diff)
  • Difficulty set to Unknown

comment:3 Changed 6 years ago by igloo

  • Milestone set to 6.10.1

comment:4 Changed 6 years ago by igloo

I'm not sure if this is the only problem, but one issue is that in a module like this:

module Test where

f alg1 alg2 = (a1, a2, a3, a4, a5)
    where
      (a1_1, a2_1, a3_1, a4_1, a5_1) = alg1
      (a1_2, a2_2, a3_2, a4_2, a5_2) = alg2

      a1 x (y,z) = (a1_1 x y, a1_2 x z)
      a2 x (y,z) = (a2_1 x y, a2_2 x z)
      a3 x (y,z) = (a3_1 x y, a3_2 x z)
      a4 x (y,z) = (a4_1 x y, a4_2 x z)
      a5 x (y,z) = (a5_1 x y, a5_2 x z)

we end up with a quadratic amount of code (and also, I think, a quadratic amount of type)

e.g. in the above module we get 5 things like this:

       \ (x_ag3 :: t5_alb) (ds_dn0 :: (t6_ale, t7_alf)) ->
         case ds_dn0 of wild_B1 { (y_ag5, z_ag7) ->
         (case w_soy
          of wild1_XO { (a1_1_akE, a2_1_akG, a3_1_akI, a4_1_akK, a5_1_akM) ->
          a2_1_akG x_ag3 y_ag5
          },
          case w1_soz
          of wild1_XO { (a1_2_ako, a2_2_akq, a3_2_aks, a4_2_aku, a5_2_akw) ->
          a2_2_akq x_ag3 z_ag7
          })
         },

comment:5 Changed 6 years ago by igloo

  • Milestone changed from 6.10.1 to 6.10.2

comment:6 Changed 5 years ago by igloo

  • Milestone changed from 6.10.2 to 6.12.1

comment:7 Changed 4 years ago by simonmar

  • Type of failure set to Compile-time performance bug

comment:8 Changed 4 years ago by igloo

  • Milestone changed from 6.12.1 to 6.12 branch

comment:9 Changed 4 years ago by igloo

  • Milestone changed from 6.12 branch to 6.12.3

comment:10 Changed 4 years ago by igloo

  • Milestone changed from 6.12.3 to 6.14.1
  • Priority changed from normal to low

comment:11 Changed 3 years ago by igloo

  • Milestone changed from 7.0.1 to 7.0.2

comment:12 Changed 3 years ago by igloo

  • Milestone changed from 7.0.2 to 7.2.1

comment:13 Changed 3 years ago by igloo

  • Milestone changed from 7.2.1 to 7.4.1

comment:14 Changed 2 years ago by igloo

  • Milestone changed from 7.4.1 to 7.6.1
  • Priority changed from low to lowest

comment:15 Changed 20 months ago by igloo

  • Milestone changed from 7.6.1 to 7.6.2

comment:16 Changed 15 months ago by morabbin

These all compile on an 8Gb RAM Macbook Pro without running out of memory. RF00549.hs produces a 52Mb .o! Sure, they take a few hours to compile, but they don't fail, nor do they appear to eat all the memory (i.e., my machine was not rendered useless). It seemed quite acceptable to me, given the sizes of the files.

Changed 15 months ago by morabbin

Make the test files compile with modern GHC

Note: See TracTickets for help on using tickets.