Opened 8 years ago

Last modified 4 years ago

#5071 new bug

GHCi crashes on large alloca/allocaBytes requests

Reported by: guest Owned by:
Priority: normal Milestone:
Component: Compiler Version: 7.0.3
Keywords: report-impact Cc: dagitj@…, daniel@…
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: None/Unknown Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s):
Wiki Page:

Description

GHCi, and ghc, seems to exit (crash?) on certain types of memory allocation exceptions.

The FFI addendum says that the allocation functions should be able to return null: http://www.cse.unsw.edu.au/~chak/haskell/ffi/ffi/ffise5.html#x8-230005

Section 5.8 says this:

If any of the allocation functions fails, a value of Ptr.nullPtr is produced. If free or reallocBytes is applied to a memory area that has been allocated with alloca or allocaBytes, the behaviour is undefined. Any further access to memory areas allocated with alloca or allocaBytes, after the computation that was passed to the allocation function has terminated, leads to undefined behaviour. Any further access to the memory area referenced by a pointer passed to realloc, reallocBytes, or free entails undefined behaviour.

In practice, code examples and documentation appear to rely on alloca NOT returning a nullPtr:

I reported this on the libraries list, and offered a documentation tweak, and I was asked to create a ticket: http://www.haskell.org/pipermail/libraries/2011-March/016117.html

That email has details about the testing I did at the time to see the crashing behavior in ghci. I was using ghc 7.0.2 on 64bit windows, but this also appears to affect 7.0.3 on linux. It's likely that other versions of ghc are affected.

My recommendation would be to make the exception thrown by alloca catchable. Possibly offering an alternative to alloca, say alloca', that can produce a nullPtr instead of using exceptions. I would advice against changing the existing alloca function to produce nullPtr as that could make a lot of existing code unsafe.

For example, it would be nice if the following printed "exception", instead of exiting:

$ ulimit -d 100000 -m 1000000 -v 100000
$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 7.0.3
$ ./Alloca 
Alloca: out of memory (requested 2148532224 bytes)
$ cat Alloca.hs
import Foreign
import Foreign.Marshal
import Foreign.Marshal.Alloc
import GHC.Exception
import System.IO.Error

main :: IO ()
main = do
  (allocaBytes maxBound compare)
    `catch` (\_ -> print "exception")
  where
  compare p = print ((nullPtr :: Ptr Int) == p)

Change History (12)

comment:1 Changed 8 years ago by dmwit

Cc: daniel@… added

comment:2 Changed 8 years ago by guest

The Haskell 2010 FFI section agrees that nullPtr should be produced: http://www.haskell.org/onlinereport/haskell2010/haskellch31.html#x39-28600031

comment:3 Changed 8 years ago by chak

I would suggest to make the libraries comply with the Haskell standard. Raising an exception instead of returning a nullPtr will break all code that is written against the standard.

comment:4 in reply to:  3 ; Changed 8 years ago by simonmar

Replying to chak:

I would suggest to make the libraries comply with the Haskell standard. Raising an exception instead of returning a nullPtr will break all code that is written against the standard.

Have you seen any code that actually checks for nullPtr? I haven't. I think we made a mistake in the standard here. In fact, by changing alloca to raise an exception I think we'll be fixing a lot of broken code!

comment:5 Changed 8 years ago by guest

As an additional data point, I modified the sample program to call mallocBytes. This is what I discovered:

  • mallocBytes throws a "resource exhausted (out of memory)" exception
  • the exception is catchable

So it would be possible to fix the alloca family of functions by implementing them in terms of mallocBytes. I think for now in the programs I write I'll use malloc and avoid alloca, as a workaround for this bug.

comment:6 Changed 8 years ago by guest

I don't know if Haskell's alloca was intended to behave like C's alloca, but this is from the man page for alloca on my linux machine:

 RETURN VALUE
       The alloca() function returns a pointer to the beginning of the allocated space.
 If the allocation causes stack overflow, program behavior is undefined.

The following program segfaults for me:

$ cat alloc.c 
#include <stdio.h>
#include <stdlib.h>
#include <limits.h>
#include <alloca.h>

int main() {


  int *p = alloca(INT_MAX);
  printf("p = %p\n", p);
  return 0;
}

What initially attracted me to using alloca is that I thought it was equivalent to the pseudo-code: bracket malloc free. Perhaps I had the wrong expectation and the FFI should be amended to say that alloca requests for too many bytes are undefined like the C variant?

comment:7 in reply to:  5 Changed 8 years ago by simonmar

Replying to guest:

I think for now in the programs I write I'll use malloc and avoid alloca, as a workaround for this bug.

Note that malloc is much slower than alloca.

comment:8 in reply to:  6 Changed 8 years ago by simonmar

Replying to guest:

I don't know if Haskell's alloca was intended to behave like C's alloca

Only in the sense that it deallocates automatically when exiting the scope.

What initially attracted me to using alloca is that I thought it was equivalent to the pseudo-code: bracket malloc free.

Yes, that's the idea - indeed, that's the reference implementation.

Perhaps I had the wrong expectation and the FFI should be amended to say that alloca requests for too many bytes are undefined like the C variant?

I can't think of any good reason to make the behaviour undefined, that would be worse than both of the other designs (nullPtr and exception).

comment:9 Changed 8 years ago by igloo

Milestone: 7.4.1

comment:10 Changed 7 years ago by marlowsd@…

commit d95a7f1311578cf37c0889098d32cbcb8964d906

Author: Simon Marlow <marlowsd@gmail.com>
Date:   Mon Dec 12 14:18:30 2011 +0000

    Avoid integer overflow when calling allocGroup() (#5071)

 rts/hooks/OutOfHeap.c |    8 ++++++--
 rts/sm/Storage.c      |    7 +++++--
 2 files changed, 11 insertions(+), 4 deletions(-)

comment:11 Changed 7 years ago by simonmar

difficulty: Unknown
Milestone: 7.4.1_|_

Handling out-of-memory conditions in general is quite tricky, and GHC doesn't make any attempt to reify memory exhaustion as an exception that user code can act upon (see #1791).

The API used by allocaBytes and co is allocate() in the RTS. It is specified to never fail, because it is used in parts of the system where failure cannot be easily handled (e.g. GMP). It either allocates more memory or exits the program.

So in summary we can't easily fix this bug, except perhaps in "obvious" cases such as allocating maxBound bytes.

comment:12 in reply to:  4 Changed 4 years ago by thomie

Keywords: report-impact added

Quoting comment:4:

I think we made a mistake in the standard here. In fact, by changing alloca to raise an exception I think we'll be fixing a lot of broken code!

Note: See TracTickets for help on using tickets.