Tiered memory allocation restricts available memory
This was first noticed when running a binary produced with GHC 8.0.2 through a queueing system (Sun Grid Engine) that limits available memory by setting ulimit -v
.
When running with **unlimited** address space, the binary reserves 1TB of RAM and everything works as expected. When running with **limited** address space, reservation happens in powers of 2 leading to significant unused memory at higher memory values.
The tests are summarized in the following table:
Limit VIRT Unused (all in GB)
8 4 8
10 8 2
16 8 8
32 16 16
34 32 2
64 32 32
66 64 2
128 64 64
130 128 2
256 128 128
258 256 2
512 256 256
514 512 2
1024 512 512
In the last line, setting ulimit -v
to 1024GB (1073741824) causes the binary to only be able to reserve 512GB.
If at runtime the software needs more than 512GB it will abort with "out of memory" instead of expanding to as close to the specified ulimit as possible.
PS: Is there any way to make the "out of memory" error a little bit more informative? Something that informs the user of how much memory was already being used and how much more was going to be reserved when the failure was seen?
Trac metadata
Trac field | Value |
---|---|
Version | 8.0.2 |
Type | Bug |
TypeOfFailure | OtherFailure |
Priority | normal |
Resolution | Unresolved |
Component | Runtime System |
Test case | |
Differential revisions | |
BlockedBy | |
Related | |
Blocking | |
CC | |
Operating system | |
Architecture |