Skip to content
Snippets Groups Projects

GHCi fails to garbage collect declaration `l = length [1..10^8]` entered at prompt

    • Open Issue created by KevinBuhr @kabuhr

    This was recently reported on Stack Overflow (https://stackoverflow.com/questions/48723119/binding-len-length-xs-and-then-calculating-len-causes-ghc-to-consume-lots).

    With GHC 8.2.2, assigning the length of a large list as a top-level declaration at the GHCi prompt and evaluating it:

    Prelude> l = length [1..10^8]
Prelude> l
10000000
Prelude>

    allocates about 7 gigs of memory before displaying a result, and then memory continues to grow over the next few seconds (at the idle GHCi prompt) until 10 gigs of memory has been allocated. None of it is freed until the interpreter session ends.

    In contrast, directly evaluating the expression or assigning it with a let-statement does **not** exhibit this behavior:

    Prelude> length [1..10^8]
10000000
Prelude> let l = length [1..10^8]
Prelude> l
10000000
Prelude>

    Also, loading the top-level declaration from a file and then evaluating it at the prompt does **not** exhibit the problem.

    I took a look at Ticket 14336 and applied the -fno-it patch referenced there. This slightly changed the pattern of memory usage: it grew to about 4 gigs before displaying the result, but then continued to grow for several seconds at the GHCi prompt before landing at about 10 gigs.

    I verified that the bug still exists on the current "master" branch (though I didn't check the "ghc-8.4" branch).

    Trac metadata
    Trac field Value
    Version 8.2.2
    Type Bug
    TypeOfFailure OtherFailure
    Priority normal
    Resolution Unresolved
    Component GHCi
    Test case
    Differential revisions
    BlockedBy
    Related
    Blocking
    CC
    Operating system
    Architecture
    To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information

    Child items 0

    • No child items are currently assigned. Use child items to break down this issue into smaller parts.

    Activity

    • KevinBuhr changed weight to 5

      changed weight to 5

    • KevinBuhr added GHCi Tbug Trac import labels

      added GHCi Tbug Trac import labels

    • Ben Gamari
      Ben Gamari @bgamari ·
      Maintainer

      Just noticed this; quite an interesting issue. I was indeed able to reproduce this in GHC 8.4.1 and confirmed that performMajorGC does not cause reclaim.

    • Ben Gamari
      Ben Gamari @bgamari ·
      Maintainer

      Here is the code produced (from -ddump-rn -ddump-simpl -ddump-bcos) by the let l form,

      λ> let l = length [1..10^8]

==================== Renamer ====================
let l_a2bl = length [1 .. 10 ^ 8]


==================== Simplified expression ====================
GHC.Base.returnIO
  @ [()]
  (GHC.Types.:
     @ ()
     ((Data.Foldable.length
         @ []
         Data.Foldable.$fFoldable[]
         @ GHC.Integer.Type.Integer
         (GHC.Enum.enumFromTo
            @ GHC.Integer.Type.Integer
            GHC.Enum.$fEnumInteger
            1
            (GHC.Real.^
               @ GHC.Integer.Type.Integer
               @ GHC.Integer.Type.Integer
               GHC.Num.$fNumInteger
               GHC.Real.$fIntegralInteger
               10
               8)))
      `cast` (UnsafeCo representational GHC.Types.Int ()
              :: (GHC.Types.Int :: *) ~R# (() :: *)))
     (GHC.Types.[] @ ()))



==================== Proto-BCOs ====================
ProtoBCO ExprTopLevel_E0#0 []:
   let sat_s3Wi = ... in ...
   bitmap:  0 []
   ALLOC_AP    0
   PUSH_BCO
     ProtoBCO sat_s3Wi#0 []:
        let sat_s3Wh = ... in ...
        bitmap:  0 []
        ALLOC_AP    0
        PUSH_BCO
          ProtoBCO sat_s3Wh#0 []:
             let sat_s3Wg = ... in ...
             bitmap:  0 []
             ALLOC_AP    0
             PUSH_BCO
               ProtoBCO sat_s3Wg#0 []:
                  let sat_s3Wf = ... in ...
                  bitmap:  0 []
                  PUSH_UBX (1) 8#
                  PACK     GHC.Integer.Type.S# 1
                  PUSH_UBX (1) 10#
                  PACK     GHC.Integer.Type.S# 1
                  PUSH_LL  1 0
                  PUSH_G   GHC.Real.$fIntegralInteger
                  PUSH_G   GHC.Num.$fNumInteger
                  PUSH_APPLY_PPPP
                  PUSH_G   GHC.Real.^
                  SLIDE    6 2
                  ENTER
             MKAP     0 words, 1 stkoff
             PUSH_UBX (1) 1#
             PACK     GHC.Integer.Type.S# 1
             PUSH_LL  1 0
             PUSH_G   GHC.Enum.$fEnumInteger
             PUSH_APPLY_PPP
             PUSH_G   GHC.Enum.enumFromTo
             SLIDE    5 2
             ENTER
        MKAP     0 words, 1 stkoff
        PUSH_L   0
        PUSH_G   Data.Foldable.$fFoldable[]
        PUSH_APPLY_PP
        PUSH_G   Data.Foldable.length
        SLIDE    4 1
        ENTER
   MKAP     0 words, 1 stkoff
   PUSH_G   GHC.Types.[]
   PUSH_L   1
   PACK     : 2
   PUSH_L   0
   PUSH_APPLY_P
   PUSH_G   GHC.Base.returnIO
   SLIDE    3 2
   ENTER

      Here is the same output for the let-less form,

      λ> l = length [1..10^8]

==================== Renamer ====================
interactive:Ghci2.l = length [1 .. 10 ^ 8]


==================== Tidy Core ====================
Result size of Tidy Core
  = {terms: 25, types: 11, coercions: 0, joins: 0/0}

-- RHS size: {terms: 10, types: 5, coercions: 0, joins: 0/0}
l :: Int
[GblId]
l = length
      @ []
      Data.Foldable.$fFoldable[]
      @ Integer
      (enumFromTo
         @ Integer
         GHC.Enum.$fEnumInteger
         1
         (^ @ Integer
            @ Integer
            GHC.Num.$fNumInteger
            GHC.Real.$fIntegralInteger
            10
            8))

-- RHS size: {terms: 1, types: 0, coercions: 0, joins: 0/0}
$trModule1_r3XB :: GHC.Prim.Addr#
[GblId, Caf=NoCafRefs]
$trModule1_r3XB = "interactive"#

-- RHS size: {terms: 2, types: 0, coercions: 0, joins: 0/0}
$trModule2_r3XM :: GHC.Types.TrName
[GblId, Caf=NoCafRefs]
$trModule2_r3XM = GHC.Types.TrNameS $trModule1_r3XB

-- RHS size: {terms: 1, types: 0, coercions: 0, joins: 0/0}
$trModule3_r3XN :: GHC.Prim.Addr#
[GblId, Caf=NoCafRefs]
$trModule3_r3XN = "Ghci2"#

-- RHS size: {terms: 2, types: 0, coercions: 0, joins: 0/0}
$trModule4_r3XO :: GHC.Types.TrName
[GblId, Caf=NoCafRefs]
$trModule4_r3XO = GHC.Types.TrNameS $trModule3_r3XN

-- RHS size: {terms: 3, types: 0, coercions: 0, joins: 0/0}
interactive:Ghci2.$trModule :: GHC.Types.Module
[GblId, Caf=NoCafRefs]
interactive:Ghci2.$trModule
  = GHC.Types.Module $trModule2_r3XM $trModule4_r3XO




==================== Proto-BCOs ====================
ProtoBCO sat_s3XU#0 []:
   let sat_s3XT = ... in ...
   bitmap:  0 []
   ALLOC_AP    0
   PUSH_BCO
     ProtoBCO sat_s3XT#0 []:
        let sat_s3XS = ... in ...
        bitmap:  0 []
        PUSH_UBX (1) 8#
        PACK     GHC.Integer.Type.S# 1
        PUSH_UBX (1) 10#
        PACK     GHC.Integer.Type.S# 1
        PUSH_LL  1 0
        PUSH_G   GHC.Real.$fIntegralInteger
        PUSH_G   GHC.Num.$fNumInteger
        PUSH_APPLY_PPPP
        PUSH_G   GHC.Real.^
        SLIDE    6 2
        ENTER
   MKAP     0 words, 1 stkoff
   PUSH_UBX (1) 1#
   PACK     GHC.Integer.Type.S# 1
   PUSH_LL  1 0
   PUSH_G   GHC.Enum.$fEnumInteger
   PUSH_APPLY_PPP
   PUSH_G   GHC.Enum.enumFromTo
   SLIDE    5 2
   ENTER
 
ProtoBCO Ghci2.l#0 []:
   Data.Foldable.length
     @ [] Data.Foldable.$fFoldable[] @ GHC.Integer.Type.Integer sat_s3XU
   bitmap:  0 []
   PUSH_G   sat_s3XU
   PUSH_G   Data.Foldable.$fFoldable[]
   PUSH_APPLY_PP
   PUSH_G   Data.Foldable.length
   ENTER
 
ProtoBCO $trModule2_r3XM#0 []:
   GHC.Types.TrNameS $trModule1_r3XB
   bitmap:  0 []
   PUSH_UBX (1) 26770624##
   PACK     GHC.Types.TrNameS 1
   ENTER
 
ProtoBCO $trModule4_r3XO#0 []:
   GHC.Types.TrNameS $trModule3_r3XN
   bitmap:  0 []
   PUSH_UBX (1) 26770592##
   PACK     GHC.Types.TrNameS 1
   ENTER
 
ProtoBCO Ghci2.$trModule#0 []:
   GHC.Types.Module $trModule2_r3XM $trModule4_r3XO
   bitmap:  0 []
   PUSH_G   $trModule4_r3XO
   PUSH_G   $trModule2_r3XM
   PACK     GHC.Types.Module 2
   ENTER

      It looks to me like CorePrep is floating out the enumFromTo application, which then gets retained.

    • trac-import added runtime perf label

      added runtime perf label

    • Ben Gamari added Pnormal label

      added Pnormal label

    • Ömer Sinan Ağacan
      Ömer Sinan Ağacan @osa1 ·
      Maintainer

      The difference between x = ... vs. let x = ... is fixed with !97 (merged), and the Core looks reasonable in both versions now, but as far as I can see from +RTS -h GHCi still retains the memory for l after :r, so the bug remains. -fno-it doesn't make any difference in +RTS -h output.

      Edited by Ben Gamari
    • Ömer Sinan Ağacan mentioned in issue #15503

      mentioned in issue #15503

    • Ben Gamari changed milestone to %9.0.1

      changed milestone to %9.0.1

    • Ben Gamari changed milestone to %9.2.1

      changed milestone to %9.2.1

    • Zubin
      Zubin @wz1000 ·
      Owner

      This seems to be fixed in %8.10.7 at least:

      ❯ ghci +RTS -s                                               ✔  3.1s   21:13:58  ─╯
GHCi, version 8.10.7: https://www.haskell.org/ghc/  :? for help
Loaded package environment from /home/zubin/.ghc/x86_64-linux-8.10.7/environments/default
Loaded GHCi configuration from /home/zubin/dotfiles/.ghci
λ> l = length [1..10^8]
l :: Int
λ> l
100000000
it :: Int
λ> 
Leaving GHCi.
   7,363,799,672 bytes allocated in the heap
      91,307,344 bytes copied during GC
      21,110,248 bytes maximum residency (7 sample(s))
    • Zubin closed

      closed

    • Simon Peyton Jones
      Simon Peyton Jones @simonpj ·
      Developer

      Do we have a regression test? Reopening to get one. @wz1000 perhaps?

    • Simon Peyton Jones reopened

      reopened

    • Zubin assigned to @wz1000

      assigned to @wz1000

    Please register or sign in to reply