Opened 3 years ago

Closed 3 years ago

Last modified 3 years ago

#10364 closed feature request (wontfix)

Feature request: Add support for FMA

Reported by: lerkok Owned by: ekmett
Priority: normal Milestone:
Component: Core Libraries Version: 7.11
Keywords: report-impact Cc: core-libraries-committee@…, erkokl@…
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: None/Unknown Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s):
Wiki Page:

Description (last modified by lerkok)

FMA (fused-multiply-add) has been around for quite some-time, and is natively supported by many of the newer processors. It's an operation that is heavily used in HPC applications. I think it's time for Haskell to have native support for it as well.

Ideally, it should be added to the Num class as a method. Assuming that is a possibility, we would have the following signature and default definition:

class Num a where
   ...
   fma :: a -> a -> a -> a
   fma x y z = x * y + z

Except of course the Float and Double instances would ensure that the rounding is done only once.

If adding it to the Num class is not an option, then RealFloat class would be the next best place; and perhaps that's arguably also a better place because those types are the ones one usually has in mind when using FMA.

I think either Num or RealFloat would be fine choices.

Implementation:

  • If the underlying architecture supports it (which is very common), directly emit FMA instruction
  • Otherwise, FFI out to C and use fma and fmaf from the math library

A direct software implementation might also be possible for platforms where neither choice above is an option, but that bridge can be crossed when we get there.

As a final note; while supporting these functions might seem going-out-of-our-way; it is indeed a big selling point in HPC applications. Furthermore, hardware manufacturers are putting big resources to make these supported natively in the instruction sets. Supporting FMA right out of the box would be a very good step in wider adaptation of Haskell in the HPC community.

Change History (15)

comment:1 Changed 3 years ago by lerkok

Cc: erkokl@… added

comment:2 Changed 3 years ago by lerkok

Description: modified (diff)

comment:3 Changed 3 years ago by lerkok

Description: modified (diff)

comment:4 Changed 3 years ago by ekmett

This probably should go out through a standard libraries@ proposal process first, to get a sense of how willing the community would be to accept a change that far up the chain. You may get some horribly long fusedMultiplyAdd name though. =) I could definitely see putting it in RealFloat. We had a proposal last year for adding things like expm1 and log1p, etc. which passed, which will involve adding members nearby.

Warning though: when we lobbied for expm1 the comunity pushed back against having sensible defaults like the one you mention here and preferred leaving it undefined, to prompt people to fill them in proactively. I'm not sure that is/was a good idea, but some folks fought pretty hard against defaults that might lose precision favoring bottoms instead.

comment:5 Changed 3 years ago by lerkok

Ah, excellent. Perhaps fma can be tacked on to that proposal.

I was reading through https://wiki.haskell.org/Library_submissions and it actually suggests proposals should now be filed as tickets; unless I'm misreading it. There used to be an e-mail to the libraries@ mailing list I think; is that still the way to go?

comment:6 Changed 3 years ago by ekmett

The libraries@ list is still here. We typically escalate anything to it that seems remotely controversial to get a sense of community feedback, buy-in, counter-proposals, etc. That is especially true for things that affect the types and classes in the Haskell Report, like this.

https://wiki.haskell.org/Library_submissions mentions it in section 7.

If the CLC decides that the discussion must be discussed with the libraries@ mailing list, the original proposer may be asked to moderate the libraries@ mailing list discussion. The general format for this is:

  • Send your proposal by email to the libraries@… mailing list (which you need to subscribe to before posting).
  • Set a deadline for discussion (no less than two weeks), and act as chair/moderator for the discussion.
  • Following discussion, it's the responsibility of the core libraries committee to determine if the proposal should be accepted. Send an email to the core libraries committee requesting a decision, optionally including your own summary of the mailing list discussion.
  • From that point forward, the CLC and maintainer will again be responsible for making a decision on how to proceed, and will inform the proposer and the mailing list. Possible decisions include acceptance, rejection, and requesting a patch be provided.

comment:7 Changed 3 years ago by ekmett

Keywords: report-impact added

comment:8 Changed 3 years ago by ekmett

That said, creating a ticket like this is likely very much the right place to start. It ensures it can't fall off our radar forever. =)

comment:9 Changed 3 years ago by lerkok

Just sent a proposal to the mailing list: https://mail.haskell.org/pipermail/libraries/2015-April/025574.html

Hopefully fma can make it in together with expm1 and friends..

comment:10 Changed 3 years ago by Lemming

RealFloat would exclude an instance Complex.

comment:11 Changed 3 years ago by ekmett

Good point. That probably pushes it up to Floating or Num then. In Num you could use it for accelerated modular arithmetic as well, which has nothing to do with floating point.

comment:12 Changed 3 years ago by carter

sounds like it makes sense to 1) put fma in Num and 2) add ghc primitive support to applicable supported microarchitectures

comment:13 Changed 3 years ago by lerkok

There was quite a bit of discussion and feedback on the mailing list, that I very much appreciated. See here:

https://mail.haskell.org/pipermail/libraries/2015-May/025597.html

Based on the analysis, I think this proposal should be shelved for the time being. And thus, I'm closing this feature-request.

For the record, I'm quoting the final message from the above discussion.

Thank you for all the feedback on this proposal. Based on the feedback, I
came to conclude that the original idea did not really capture what I
really was after, and hence I think this proposal needs to be shelved for
the time being.

I want to summarize the points made so far:

    * Almost everyone agrees that we should have this functionality
available. (But see below for the direction I want to take it in.)
    * There's some disagreement on the name chosen, but I think this is
less important for the time being.
    * The biggest gripe is where does "fma" really belong. Original
suggestion was 'RealFloat', but people pointed 'Num' is just a good place
as well.
    * Most folks want a default definition, and see "fma" as an
optimization.

It is these last two points actually that convinced me this proposal is not
really what I want to have. I do not see "fma" as an optimization. In
particular, I'd be very concerned if the compiler substituted "fma x y z"
for "x*y+z". The entire reason why IEEE754 has an fma operation is because
those two expressions have different values in general. By the same token,
I'm also against providing a default implementation. I see this not as an
increased-precision issue, but rather a semantic one; where "x*y+z" and
"fma x y z" *should* produce two different values, per the IEEE754 spec.
It's not really an optimization, but how floating-point values work. In
that sense "fma" is a separate operation that's related to multiplication
and addition, but is not definable in those terms alone.

Having said that, it was also pointed out that for non-float values this
can act as an optimization. (Modular arithmetic was given as an example.)
I'd think that functionality is quite different than the original proposal,
and perhaps should be tackled separately. My original proposal was not
aiming for that particular use case.

My original motivation was to give Haskell access to the floating-point
circuitry that hardware-manufacturers are putting a lot of effort and
energy into. It's a shame that modern processors provide a ton of
instructions around floating-point operations, but such operations are
simply very hard to use from many high-level languages, including Haskell.

Two other points were raised, that also convinced me to seek an alternative
solution:

   * Tikhon Jelvis suggested these functions should be put in a different
class, which suggests that we're following IEEE754, and not some idealized
model of numbers. I think this suggestion is spot on, and is very much in
line with what I wanted to have.
   * Takebonu Tani kindly pointed that a discussion of floats in the
absence of rounding-modes is a moot one, as the entire semantics is based
on rounding. Haskell simply picks "RoundNearestTiesToEven," but there are 4
other rounding modes defined by IEEE754, and I think we need a way to
access those from Haskell in a convenient way.

Based on this analysis, I'm withdrawing the original proposal. I think fma
and other floating-point arithmetic operations are very important to
support properly, but it should not be done by tacking them on to Num or
RealFloat; but rather in a new class that also considers rounding-mode
properly.

The advantage of the "separate" class approach is, of course, I (or someone
else) can create such a class and push it on to hackage, using FFI to
delegate the task of implementation to the land-of-C, by supporting
rounding modes and other floating-point weirdness appropriately. Once that
class stabilizes and its details are ironed out, then we can imagine
cooperating with GHC folks to actually bypass the FFI and directly generate
native code whenever possible.

This is the direction I intend to move on. Please drop me a line if you'd
like to help out and/or have any feedback.

comment:14 Changed 3 years ago by lerkok

Resolution: wontfix
Status: newclosed

comment:15 Changed 3 years ago by rwbarton

Be aware (if you aren't already) that GHC does not do any management of floating-point control registers, so functions called through FFI should take care to clean up their floating-point state, otherwise the rounding mode can change unpredictably at the level of Haskell code.

Note: See TracTickets for help on using tickets.