Opened 4 years ago

Closed 4 years ago

Last modified 4 years ago

#7866 closed bug (invalid)

floor (0/0) :: Int is different with -O0 and -O1

Reported by: alang9 Owned by:
Priority: normal Milestone:
Component: Compiler Version: 7.6.2
Keywords: Cc: Artyom.Kazak@…, bgamari@…
Operating System: Linux Architecture: x86_64 (amd64)
Type of failure: Incorrect result at runtime Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s):
Wiki Page:


This program:

main = print (floor (0/0) :: Int)

prints a different result with -O0:


and -O1:


Change History (6)

comment:1 Changed 4 years ago by Artyom.Kazak

Cc: Artyom.Kazak@… added

Apparently, it’s a x64-specific bug; I can’t reproduce it with GHC 7.6.2 on x86 Linux.

comment:2 Changed 4 years ago by bgamari

difficulty: Unknown

This appears to be due to the (lack of) handling of NaN in compiler/nativeGen/X86/CodeGen.hs:coerceFP2Int which in the SSE case lowers to a plain CVTTSD2TIQ instruction. In the event that the source is NaN, this places the indeterminate integer value (which appears to be 0x80000000) in the destination register. Oddly enough, your test produces instead a value of -0x8000000000000000.

Regardless, assuming the produce is NaN handling, it's not entirely clear how this should be handled without performance loss. If the goal were perfect "correctness" (whatever this means in the fact of NaN values being cast to integers), we'd want to ensure we check for an indeterminate result and return the "appropriate" value (which judging from the -O0 results is apparently 0). This seems like an excessive measure, however, punishing all users for what is likely a rare occurrence and one that should be handled explicitly anyways.

comment:3 Changed 4 years ago by bgamari

Resolution: invalid
Status: newclosed

Looking at the report it's not entirely clear how operations on NaN are defined. That being said, the documentation of RealFloat is quite clear that many operations on NaN are undefined. In light of this, it makes sense to assume similar behavior in RealFrac operations. For this reason, I'm going to close this as not a bug. If someone would like to disagree, feel free to reopen it with an explanation.

comment:4 Changed 4 years ago by bgamari

Cc: bgamari@… added

comment:5 Changed 4 years ago by bgamari

According the IEEE754-1985, Section 7.1 (as I don't have access to the latest version of the standard) attempting to convert an unrepresentable floating point value like should result in an invalid operation exception. The specification states that when there is no trap, the result should be NaN when floating point. The specification makes no statement on what should happen when there is no trap and the destination format is unable to represent NaN.

comment:6 Changed 4 years ago by bgamari

Interestingly enough building with -O1 -fllvm also results in 0. However, it seems this is only accidental as the conversion is lowered to %ln1JE = fptosi double %ln1JD to i64 in LLVM IR which is undefined when the source can't fit in the destination type.

Note: See TracTickets for help on using tickets.