Compiling with profiling on Windows can cause linker errors

changed weight to 5

Attached file wai.zip (download).

bug demo

closed

I've opened a stack bug since I was unable to reproduce using cabal.

Trac metadata

Trac field	Value
Resolution	Unresolved → ResolvedInvalid

reopened

No need to file a stack bug, as your initial hunch was correct: this is a GHC bug. You can reproduce this issue using nothing but GHC and your repro case:

{-# LANGUAGE BangPatterns #-}
-- NetworkRequestHeader.hs
module NetworkRequestHeader (parseHeaderLines, parseRequestLine) where

import Control.Exception
import Control.Monad
import Data.ByteString.Internal (ByteString(..), memchr)
import Data.Word
import Foreign.ForeignPtr (withForeignPtr)
import Foreign.Ptr (Ptr, plusPtr, minusPtr, nullPtr)
import Foreign.Storable (peek)

-- | Error types for bad 'Request'.
data InvalidRequest = NonHttp

instance Show InvalidRequest where show _ = ""
instance Exception InvalidRequest

parseHeaderLines :: [ByteString]
                 -> IO (ByteString
                       ,ByteString  --  Path
                       ,ByteString  --  Path, parsed
                       )
parseHeaderLines [] = throwIO $ NonHttp
parseHeaderLines (firstLine:_) = do
    (method, path') <- parseRequestLine firstLine
    let path = path'
    return (method, path', path)

parseRequestLine :: ByteString
                 -> IO (ByteString
                       ,ByteString)
parseRequestLine (PS fptr off len) = withForeignPtr fptr $ \ptr -> do
    when (len < 14) $ throwIO NonHttp
    let methodptr = ptr `plusPtr` off
        limptr = methodptr `plusPtr` len
        lim0 = fromIntegral len

    pathptr0 <- memchr methodptr 32 lim0 -- ' '
    when (pathptr0 == nullPtr || (limptr `minusPtr` pathptr0) < 11) $
        throwIO NonHttp
    let pathptr = pathptr0 `plusPtr` 1
        lim1 = fromIntegral (limptr `minusPtr` pathptr0)

    httpptr0 <- memchr pathptr 32 lim1 -- ' '
    when (httpptr0 == nullPtr || (limptr `minusPtr` httpptr0) < 9) $
        throwIO NonHttp
    let httpptr = httpptr0 `plusPtr` 1
        lim2 = fromIntegral (httpptr0 `minusPtr` pathptr)

    checkHTTP httpptr
    queryptr <- memchr pathptr 63 lim2 -- '?'

    let !method = bs ptr methodptr pathptr0
        !path
          | queryptr == nullPtr = bs ptr pathptr httpptr0
          | otherwise           = bs ptr pathptr queryptr

    return (method,path)
  where
    check :: Ptr Word8 -> Int -> Word8 -> IO ()
    check p n w = do
        w0 <- peek $ p `plusPtr` n
        when (w0 /= w) $ throwIO NonHttp
    checkHTTP httpptr = do
        check httpptr 0 72 -- 'H'
        check httpptr 1 84 -- 'T'
        check httpptr 2 84 -- 'T'
        check httpptr 3 80 -- 'P'
        check httpptr 4 47 -- '/'
        check httpptr 6 46 -- '.'
    bs ptr p0 p1 = PS fptr o l
      where
        o = p0 `minusPtr` ptr
        l = p1 `minusPtr` p0

{-# LANGUAGE BangPatterns #-}
-- Main.hs
module Main (main) where

import NetworkRequestHeader

import Control.Monad

main :: IO ()
main = void $ parseHeaderLines []

$ ghc -O -fforce-recomp -prof -fprof-auto Main.hs
[1 of 2] Compiling NetworkRequestHeader ( NetworkRequestHeader.hs, NetworkRequestHeader.o )
[2 of 2] Compiling Main             ( Main.hs, Main.o )
Linking Main.exe ...
.\NetworkRequestHeader.o:fake:(.text+0x920): undefined reference to `__chkstk_ms'
.\NetworkRequestHeader.o:fake:(.text+0xc10): undefined reference to `__chkstk_ms'
.\NetworkRequestHeader.o:fake:(.text+0xc70): undefined reference to `__chkstk_ms'
.\NetworkRequestHeader.o:fake:(.text+0xdd8): undefined reference to `__chkstk_ms'
.\NetworkRequestHeader.o:fake:(.text+0xe90): undefined reference to `__chkstk_ms'
.\NetworkRequestHeader.o:fake:(.text+0xee0): more undefined references to `__chkstk_ms' follow
collect2.exe: error: ld returned 1 exit status
`gcc.exe' failed in phase `Linker'. (Exit code: 1)

Note that both the -O and -fprof-auto flags are required to trigger the linker errors. (This explains why you seemingly couldn't trigger the error with cabal-install, as cabal-install's --enable-profiling option doesn't imply the -fprof-auto flag, whereas stack's --profile option does.)

Phyx-, any ideas as to what might be causing this?

Trac metadata

Trac field	Value
Resolution	ResolvedInvalid → Unresolved

For whatever reason libgcc is not passed to the linker by gcc. Would need to look at what the profiling options pass to the linker.

In any case you can work around it by passing -lgcc to ghc or add extra-libraries: gcc to your cabal file. We should probably add gcc explicitly to the libraries list of the rts.

Hmm guess not, seems name mangling is wrong.. patch coming.

changed milestone to %8.6.4

Thanks for the report crockeea and the reproducer RyanGlScott.

If you require, you can work around this by making a file

PROVIDE(__chkstk_ms = ___chkstk_ms);

called workaround.ldscript and tell GHC -optl"workaround.ldscript" or add that as ghc-options in your cabal file.

added 1 deleted label

Merged to master in fb031b9b.

closed

Trac metadata

Trac field	Value
Resolution	Unresolved → ResolvedFixed

removed 1 deleted label

added compiler crash label

added Pnormal label

It seems that there is a regression of this in GHC 8.8.*. When I compile the above example with GHC 8.8.1 or 8.8.3 I get:

[1 of 2] Compiling NetworkRequestHeader ( NetworkRequestHeader.hs, NetworkRequestHeader.o )
[2 of 2] Compiling Main             ( Main.hs, Main.o )
Linking Main.exe ...
.\NetworkRequestHeader.o:fake:(.text+0x968): undefined reference to `__chkstk_ms'
.\NetworkRequestHeader.o:fake:(.text+0xc00): undefined reference to `__chkstk_ms'
.\NetworkRequestHeader.o:fake:(.text+0xd28): undefined reference to `__chkstk_ms'
.\NetworkRequestHeader.o:fake:(.text+0xdf8): undefined reference to `__chkstk_ms'
.\NetworkRequestHeader.o:fake:(.text+0xf80): undefined reference to `__chkstk_ms'
.\NetworkRequestHeader.o:fake:(.text+0x11f8): more undefined references to `__chkstk_ms' follow
collect2.exe: error: ld returned 1 exit status
`gcc.exe' failed in phase `Linker'. (Exit code: 1)

Hm. The error in #16166 (comment 266880) does not occur with GHC 8.10.1 or GHC 8.6.5. Indeed, commit fb031b9b was backported to the ghc-8.6 branch with 0f253b89, but this commit does not appear to have ever made it to ghc-8.8.

@bgamari, are there plans for any subsequent GHC 8.8 releases? If so, would this be a candidate for backporting?

changed milestone to %8.8.4

Yes, it's very likely that there will be another 8.8 release (due to the Windows issues reported in #17926 (closed)). However, there isn't a firm timeline yet.

I'll backport.

Curious though how did this happen? How can it be in 8.10 and 8.6 but not 8.8?

I would have assumed that at the time I fixed this it was during the 8.8 period and backported it to 8.6?

Just wondering if I mislabeled something?

I believe that commit fb031b9b actually occurred during the GHC 8.10 development period, and that it was backported to ghc-8.6, but not ghc-8.8, due to an oversight.

@RyanGlScott ah thanks! The time lines are all a bit of a blur to me.

Trac field	Value
Version	8.6.3
Type	Bug
TypeOfFailure	OtherFailure
Priority	normal
Resolution	Unresolved
Component	Compiler
Test case
Differential revisions
BlockedBy
Related
Blocking
CC
Operating system
Architecture

Compiling with profiling on Windows can cause linker errors

Child items 0

Activity