Opened 3 years ago

Closed 3 years ago

#5524 closed bug (fixed)

Unnecessary overflow in encodeFloat

Reported by: daniel.is.fischer Owned by: daniel.is.fischer
Priority: normal Milestone:
Component: libraries/base Version: 7.2.1
Keywords: overflow, encodeFloat Cc:
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: None/Unknown Difficulty:
Test Case: Blocked By:
Blocking: Related Tickets:

Description

Prelude> encodeFloat (2^1024) (-1024)
Infinity

The real result is 1.0 of course.

The docs for encodeFloat now say that the result is infinite if overflow occurs, so technically we're within the spec, but I think in this case it shouldn't overflow.

Change History (2)

comment:1 Changed 3 years ago by daniel.is.fischer

  • Status changed from new to merge

Fixed by [3b9a78d45eab41ecd616e805a3adc25d5d3c8332]
There's a small performance hit when encoding a J# s# ba# with few limbs (1-2%).

comment:2 Changed 3 years ago by daniel.is.fischer

  • Resolution set to fixed
  • Status changed from merge to closed
Note: See TracTickets for help on using tickets.