showFloat for higher precision types produces strange results for some values
|Reported by:||axman6||Owned by:|
|Type of failure:||Incorrect result at runtime||Test Case:|
|Related Tickets:||Differential Rev(s):|
I've written a library which is a quda-double type a la the QD C/C++ package, and showFloat does not behave correctly for numbers with such high precision.
My type has ~212 bits of precision, and when using showFloat from Numeric, I get strange results for integer values:
show (1 :: QDouble) = "0.00000000000000000000000000000000000000000000001e47" show (1.1 :: QDouble) = "1.1" show (1000 :: QDouble) = "0.00000000000000000000000000000000000000000000001e50" -- These seems to suggest it happens for any number with only a -- few high bits set to 1 in the result of decodeFloat show (1.125 :: QDouble) = "0.00000000000000000000000000000000000000000000001125e47" show (1.625 :: QDouble) = "0.00000000000000000000000000000000000000000000001625e47"
The problem seems to be related to the result of floatDigits, which starts causing problem when it's larger than 56 (floatDigits x, show x):
(56,"1.0") (57,"01.0") (60,"001.0") (212,"0.00000000000000000000000000000000000000000000001e47")
My fix has been to use a modified version of showFloat from Numeric by changing the floatToDigits function to include a fix for times when large numbers of zeros are produced:
fixup2 (xs,k) = let (zs,ys) = span (==0) xs in (ys,k-length zs) in fixup2 (map fromIntegral (reverse rds), k)
This fixes the symptom but not the issue itself (though it seems like a reasonable thing to have any result returned by floatToDigits.
I have attached as minimal test case as I could come up with. Using floatToDigits from Numeric causes the strange behaviour, while floatToDigits' included in the test case does not.