Opened 3 years ago

Closed 3 years ago

#5525 closed bug (fixed)

Mismatch between documentation and behaviour of encodeFloat

Reported by: daniel.is.fischer Owned by:
Priority: normal Milestone:
Component: libraries/base Version: 7.3
Keywords: Cc:
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: Documentation bug Difficulty:
Test Case: Blocked By:
Blocking: Related Tickets:

Description

In the process of elaborating the docs to treat #3898, I wrote

    -- | 'encodeFloat' performs the inverse of 'decodeFloat' in the
    -- sense that for finite @x@ with the exception of @-0.0@,
    -- @'uncurry' 'encodeFloat' ('decodeFloat' x) = x@.
    -- @'encodeFloat' m n@ is the closest representable floating-point
    -- number to @m*b^^n@ (or @±Infinity@ if overflow occurs).

Unfortunately, the last sentence is not quite true. If the end result is a subnormal number, encodeFloat performs two roundings, which may lead to a wrong result (can be 1 ulp off). Changing encodeFloat so that it always finds the closest representable number would cost a lot in performance, it would bring it close to fromRational. So the thing to do is change the docs, I just don't know how yet.

Change History (1)

comment:1 Changed 3 years ago by daniel.is.fischer

  • Resolution set to fixed
  • Status changed from new to closed

Fixed. Not overly elegant, but meh.

Note: See TracTickets for help on using tickets.