Opened 18 months ago

Closed 18 months ago

Last modified 18 months ago

#8823 closed bug (invalid)

showFloat for higher precision types produces strange results for some values

Reported by: axman6 Owned by:
Priority: low Milestone:
Component: Prelude Version: 7.8.1-rc1
Keywords: Cc:
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: Incorrect result at runtime Test Case:
Blocked By: Blocking:
Related Tickets: Differential Revisions:

Description

I've written a library which is a quda-double type a la the QD C/C++ package, and showFloat does not behave correctly for numbers with such high precision.

My type has ~212 bits of precision, and when using showFloat from Numeric, I get strange results for integer values:

show (1 :: QDouble) = "0.00000000000000000000000000000000000000000000001e47"
show (1.1 :: QDouble) = "1.1"
show (1000 :: QDouble) = "0.00000000000000000000000000000000000000000000001e50"
-- These seems to suggest it happens for any number with only a
-- few high bits set to 1 in the result of decodeFloat
show (1.125 :: QDouble) = "0.00000000000000000000000000000000000000000000001125e47"
show (1.625 :: QDouble) = "0.00000000000000000000000000000000000000000000001625e47"

The problem seems to be related to the result of floatDigits, which starts causing problem when it's larger than 56 (floatDigits x, show x):

(56,"1.0")
(57,"01.0")
(60,"001.0")
(212,"0.00000000000000000000000000000000000000000000001e47")

My fix has been to use a modified version of showFloat from Numeric by changing the floatToDigits function to include a fix for times when large numbers of zeros are produced:

  fixup2 (xs,k) = let (zs,ys) = span (==0) xs in (ys,k-length zs)
 in
 fixup2 (map fromIntegral (reverse rds), k)

This fixes the symptom but not the issue itself (though it seems like a reasonable thing to have any result returned by floatToDigits.

I have attached as minimal test case as I could come up with. Using floatToDigits from Numeric causes the strange behaviour, while floatToDigits' included in the test case does not.

Attachments (1)

TestCase.hs (5.2 KB) - added by axman6 18 months ago.
Minimal test showing strange behaviour

Download all attachments as: .zip

Change History (4)

Changed 18 months ago by axman6

Minimal test showing strange behaviour

comment:1 Changed 18 months ago by axman6

To reproduce the bug, try changing the value of floatDigits to ~56 and running floatToDigits 10 {FOne,FOneThousand,FTwo}. As can be seen with FPi the correct results are produced.

comment:2 Changed 18 months ago by rwbarton

  • Resolution set to invalid
  • Status changed from new to closed

Your RealFloat instance is invalid. See the documentation for decodeFloat:

If decodeFloat x yields (m,n), then x is equal in value to m*bn, where b is the floating-point radix, and furthermore, either m and n are both zero or else b(d-1) <= abs m < bd, where d is the value of floatDigits x.

comment:3 Changed 18 months ago by axman6

Hmmm, it seems you're right, but nonetheless my implementation of decodeFloat is the only one that makes sense for my type since it can represent more than 212 bits of data (for example 1.0000000...005 is represented with the doubles 1.0 and 0.5eN, and 0 for the two other doubles]). It seems that perhaps in my case I may need floatDigits to be very large indeed or use another method to show the number.

Note: See TracTickets for help on using tickets.