unexpected out of memory error on FreeBSD
I wrote a daemon that periodically scrapes information from a webpage that crashes after about 20 hours with the error: "out of memory (requested 1048576 bytes)". The program runs fine on my Ubuntu machine, but always crashes on FreeBSD even though there is plenty of memory to spare/memory stays constant/etc.
I boiled it down to the following code, which consistently crashes after about 4000-6000 iterations when compiled with -O2, and about 35000 without.
import Data.Conduit
import Data.Conduit.List
import qualified Data.ByteString as B
import Network.HTTP.Conduit
main :: IO ()
main = do
manager <- newManager def
loop manager 1
loop :: Manager -> Int -> IO ()
loop manager i = do
putStrLn $ show i
request <- parseUrl "http://localhost:8000/courselist.html"
html <- runResourceT $ do
response <- http request manager
fmap B.concat $ responseBody response $$+- consume
loop manager $ i + 1
For debugging, I served a copy of of a typical html sample from localhost using "python -m SimpleHTTPServer". The html file is attached.
Sample Output:[[BR]]
...
35361
35362
35363
35364
35365
main: out of memory (requested 1048576 bytes)
Other information:
$ uname -a
FreeBSD oriskova 9.1-RELEASE FreeBSD 9.1-RELEASE #0 r243826: Tue Dec 4 06:55:39 UTC 2012 root@obrian.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC i386
$ gcc -v
Using built-in specs.
Target: i386-undermydesk-freebsd
Configured with: FreeBSD/i386 system compiler
Thread model: posix[[BR]]
gcc version 4.2.1 20070831 patched [FreeBSD]
I wasn't sure if I should submit this or not since it seemed fairly specific to http-conduit, but the wiki said to go ahead and submit runtime errors, so that's what I did. This is my first time ever submitting a bug report and I tried my best to follow all the instructions at http://hackage.haskell.org/trac/ghc/wiki/ReportABug, but please tell me if there is something I'm doing wrong.
Trac metadata
Trac field | Value |
---|---|
Version | 7.4.2 |
Type | Bug |
TypeOfFailure | OtherFailure |
Priority | normal |
Resolution | Unresolved |
Component | Runtime System |
Test case | |
Differential revisions | |
BlockedBy | |
Related | |
Blocking | |
CC | |
Operating system | |
Architecture |