File read/write operations >= 2GiB fail on macOS with an "Invalid argument" exception
Summary
File reads/writes bigger than 2GB result in an "Invalid argument" exception on macOS. Files bigger than 2GB still work, but individual read/write operations bigger than 2GB fail.
This was already reported for the bytestring
library, but the actual problem is in base
. I didn't see an earlier report.
bytestring
report: https://github.com/haskell/bytestring/issues/153
Python has a workaround for this issue and more discussion: https://bugs.python.org/issue24658
Steps to reproduce
{-# LANGUAGE ScopedTypeVariables #-}
import Foreign.Ptr (Ptr)
import Foreign.Marshal.Alloc (mallocBytes, free)
import System.IO (hPutBuf, withBinaryFile, IOMode (WriteMode))
-- more than 2GiB
numBytes :: Int
numBytes = 2264375889
main :: IO ()
main = do
(ptr :: Ptr ()) <- mallocBytes numBytes
-- the next line produces the exception on macOS
withBinaryFile "test.out" WriteMode (\h -> hPutBuf h ptr numBytes)
free ptr
Expected behavior
The base
library provides operating system independent buffering and encoding functionality. I think it should work around operating system peculiarities and write all the data to the test.out
file without throwing an excepion.
Environment
Tested with GHC 8.6.5 on macOS 10.14.6