Opened 10 years ago

Closed 9 months ago

#1791 closed feature request (fixed)

heap overflow should generate an exception

Reported by: guest Owned by:
Priority: normal Milestone: 8.2.1
Component: Runtime System Version: 6.8
Keywords: Cc: iampure@…, simonmar
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: None/Unknown Test Case: outofmem2
Blocked By: Blocking:
Related Tickets: Differential Rev(s): Phab:D2790
Wiki Page:

Description (last modified by rwbarton)

Heap overflow should produce a HeapOverflow exception that can be caught, rather than shutting down the entire RTS immediately.

[Original ticket description follows. The submitter happened to expose another bug, which was that heap overflow was not detected at all when a single allocation exceeded the maximum heap size. The program below now exits with a "Heap exhausted" message.]


I want to use the -M option for the goals that are stated in the manual.

./TestProgram +RTS -M5m -RTS 

Expected output:

Something like "out of heap space"

Actual result:

Machine going into a state where it swaps memory 

This is the code for TestProgram:

import Control.Monad.ST 
import Data.Array.ST
import Data.Array.MArray
import Data.Array.Base(unsafeNewArray_)
main = print (runST (do make_empty_table >> return ()))

make_empty_table::  ST s (STArray s (Int, Int) (Maybe ep))
make_empty_table = 
       unsafeNewArray_ ((1, 1), (16384, 16384))

This was tested with 6.9.20071018 on an athlon-xp, and confirmed by dcoutts also on x86-64 with ghc-6.8.0.20071015.

Change History (18)

comment:1 Changed 10 years ago by simonmar

Component: CompilerRuntime System
Milestone: 6.8 branch
Owner: set to igloo
Type: bugmerge

Fixed, thanks!

Wed Oct 24 10:54:20 BST 2007  Simon Marlow <simonmar@microsoft.com>
  * FIX #1791: fail with out-of-heap when allocating more than the max heap size in one go

comment:2 Changed 10 years ago by simonmar

Test Case: outofmem2

comment:3 Changed 10 years ago by igloo

Resolution: fixed
Status: newclosed

Merged

comment:4 Changed 10 years ago by guest

Resolution: fixed
Status: closedreopened
Type: mergebug

I actually expected to get an exception stating "out of heap", like it also works for stack overflows. I have a catch all exception catcher, but this one isn't triggered. Also the actual output doesn't state it's an exception, so I guess it really is not. So, I am reopening it again. Again, to be clear: I want that

Control.Exception.catch 
         action 
         (\exception -> do 
           appendFile "/home/me/exception" (show exception)
           return Nothing)

where action contains the code that overflows the heap does not result in a complete program failure, it should just be a _local_ problem (i.e. an exception). This also means that the test case is likely to be wrong.

comment:5 Changed 10 years ago by simonmar

Owner: igloo deleted
Status: reopenednew
Summary: -M option does not workheap overflow should generate an exception
Type: bugfeature request

Correct - heap overflow doesn't currently generate an exception. It's something we should really do, but it isn't completely clear how to do it. For example, in order to run the exception handler, you need some heap space. So perhaps you want a "soft" and a "hard" limit on heap space, and to get an exception when the "soft" limit is reached.

Another question is: which thread(s) should get the exception?

comment:6 Changed 10 years ago by igloo

Milestone: 6.8 branch6.10 branch

comment:7 in reply to:  5 Changed 10 years ago by demetrix

Replying to simonmar:

Another question is: which thread(s) should get the exception?

Why not the thread currently executing IO action which failed to allocate memory? Such behaviour would be the same as in almost any other programming language. Perhaps the only potential problem here is when additional threads are spawned not explicitely with forkIO or forkOS but rather implicitely with par - maybe in that case the thread which forces the evaluation of the "parallel" value should get the exception?

comment:8 Changed 9 years ago by simonmar

Architecture: UnknownUnknown/Multiple

comment:9 Changed 9 years ago by simonmar

Operating System: UnknownUnknown/Multiple

comment:10 Changed 8 years ago by igloo

Milestone: 6.10 branch_|_

comment:11 Changed 5 years ago by morabbin

Type of failure: None/Unknown

HeapOverflow is defined in Control.Exception.Base, and noted as not being thrown by GHC currently. Seems like a stable point to leave this issue.

comment:12 Changed 10 months ago by dobenour

Cc: simonmar added

How difficult would this be to actually implement? I can think of several valid use cases.

comment:13 Changed 10 months ago by rwbarton

Description: modified (diff)

comment:14 Changed 10 months ago by simonmar

There are a few questions to nail down, like under what precise conditions the exception is thrown, and to which threads. The actual implementation probably isn't all that difficult. Start by making a proposal?

comment:15 Changed 10 months ago by dobenour

I created a patch for this, but it is just a best-effort implementation.

comment:16 Changed 10 months ago by bgamari

Differential Rev(s): Phab:D2790
Status: newpatch

comment:17 Changed 9 months ago by Ben Gamari <ben@…>

In 12ad4d4/ghc:

Throw an exception on heap overflow

This changes heap overflow to throw a HeapOverflow exception instead of
killing the process.

Test Plan: GHC CI

Reviewers: simonmar, austin, hvr, erikd, bgamari

Reviewed By: simonmar, bgamari

Subscribers: thomie

Differential Revision: https://phabricator.haskell.org/D2790

GHC Trac Issues: #1791

comment:18 Changed 9 months ago by bgamari

Milestone: 8.2.1
Resolution: fixed
Status: patchclosed
Note: See TracTickets for help on using tickets.