wiki:ForeignFunctionInterface

Version 12 (modified by chak@…, 5 years ago) (diff)

--

Foreign Function Interface

Ticket: #35

Brief Explanation

The Foreign Function Interface (FFI) adds support for invoking code and accessing data structures implemented in other programming languages and vice versa. The current proposal encompasses a general mechanism for inter-language operations as well as specific support for interoperating with the C programming language.

References

Pros

  • Widely accepted and used addendum.
  • Provides an essential facility.

Cons

  • Inappropriate use can subvert all semantic guarantees provided by Haskell and can cause memory corruption and program crashes.

Open Questions

The following topics still require a bit of discussion or a decision between multiple alternatives.

Libraries

A large part of the FFI addendum are the libraries living under Foreign in the library hierarchy. Given the general discussion surrounding the inclusion of libraries into Haskell', we need a concrete approach for the Foreign libraries. A tentative proposal was the following:

  • The FFI addendum has been written before the proposal for HierarchicalModules, it uses a flat names space. In Haskell', we will use the hierarchical names from the standard library.
  • Haskell' keeps a subset of the libraries defined by Haskell 98 (but with their hierarchical names) and adds the FFI libraries.

TODO: Decide whether we adopt the above tentative proposal.

Reinitialisation after hs_exit()

The FFI addendum currently requires that hs_exit() can be followed by another hs_init(). GHC doesn't support that and I am not convinced that we should require it. I propose to remove that requirement.

TODO: Decide whether we remove the requirement for re-initialisation.


Integration into the report

The FFI addendum is already written and formatted in the style of the report — it should be straight forward to integrate. In addition the following changes relative to the original addendum will be applied.

Hierarchical names for the libraries

Changes to the library portion as discussed above under "Open Questions".

Transparent marshalling of newtypes

The FFI addendum defines in Section 3.2 that The argument types ati produced by fatype must be marshallable foreign types; that is, each ati is either (1) a basic foreign type or (2) a type synonym or renamed datatype of a marshallable foreign type. We will improve on the second part of this statement as follows:

  • As the transparent marshalling of newtypes (aka renamed datatypes) is a fairly significant features, we will dedicate a separate (sub)subsection to it.
  • In contrast to the FFI addendum (and it's implementation in GHC), we require that a newtype in a foreign signature is not abstract. Only if its constructor is visible, can the newtype be transparently marshalled. (After all, marshalling makes only sense if we know the type of the value in foreign land.) This implies that we will export the newtypes in the modules Foreign.C.Types and
  • Clarify the connection between marshallable foreign types and the various flavours of foreign signatures discussed in Section 4.1.3. (E.g., in case of a foreign import "dynamic" the whole signature —grammar nonterminal ftype— doesn't need to be marshallable, only portions of it.)
  • We allow GHC's newtype wrapping of the IO monad.
  • We add one or two examples.

Topics that need discussion for the integration into Haskell'

unsafePerformIO

Some have expressed the opinion that the inclusion of unsafePerformIO sends the wrong signal. They aregue that although it is true that by giving inaccurate foreign imports one can define unsafePerformIO, but that doesn't mean it should be standard. Moreover, as the stated purpose of unsafePerformIO in the FFI (hiding marshalling and unmarshalling for otherwise pure functions) can be achieved with a runST-like device if the language has Rank2Types or RankNTypes.

Personally, I don't see the issue. We are including arbitrary C code. For this to be safe, there are additional proof obligations. The function unsafePerformIO is the same - for it to be safe, additional proof obligations have to be met. Concerning a runST-like device, if the idea is to limit the scope of effects that are permitted by putting it into another monad and using a different interface, then this will at least complicate the API quite considerably. Currently, marshalling is performed by functions that read and write arbitrary memory locations. The FFI needs these functions anyway; hence, any other marshalling would need to add functions in addition to the existing ones. Secondly, marshalling is somewhat unpredictable in that different C libraries require different marshalling. Providing an interface with limited effects that captures all possible situations seems tricky, but if anybody has a concrete proposal, let's see it.

-- ManuelChakravarty?

Include files

Foreign imports may specify a single include file. This is sufficient in theory, but most GHC users prefer to use -#include options instead, and the standard form is poorly supported by GHC. The hierarchical libraries use an INCLUDE pragma as a portable replacement for OPTIONS_GHC -#include. jhc lazily collects the imports needed and only includes the ones needed by the FFI functions used after optimizations. this means you can have FFI calls refering to libraries and includes that are not on a system and still compile the program if none of the routines are used.

The rationale behind the design in the addendum is that for cases where a single include files does not suffice, you can always write your own include file that does something more clever (and include that from Haskell). Include files can be used in so many different ways that any other solution, while covering some more complex uses, will never cover them all, which will force people into the fallback of providing extra include files anyway.

-- ManuelChakravarty?

ccall vs. stdcall

Cross-platform libraries (e.g. HOpenGL) often want to import the same foreign functions using the ccall convention on Unix and the stdcall convention on Windows. The usual method is to use CPP hackery.

Earlier versions of the FFI addendum had another conventions that would automatically adapt. IIRC we remove this eventually, because it wasn't always sufficient. Before considering this again, we should consult the mailing list archives for the exact reason of the earlier removal.

-- ManuelChakravarty?

CString

The specified CString conversions are not yet supported in the hierarchical libraries.

Is this a Haskell' issue?

-- ManuelChakravarty?

Specifying libraries

jhc allows libraries to be specified as well as include files. this is handy for its lazy linking, the syntax is "-lfoo include.h foo_func"

This is another feature that we had for a while and then removed as it didn't always work. I think this is something the package manager should handle.

-- ManuelChakravarty?

Changes to the FFI libraries

Additions:

  • We should also have castCharToCUChar, castCUCharToChar, castCharToCSChar, and castCSCharToChar (i.e., not only for CChar of which it is platform-dependent whether it is signed or not).
  • Types from ISO C99 (with conversion routines):
    WordPtr uintptr_t
    WordMax uintmax_t
    IntPtr  intptr_t
    IntMax  intmax_t
    
    ptrToWordPtr :: Ptr a -> WordPtr
    wordPtrToPtr :: WordPtr -> Ptr a
    
    ptrToIntPtr :: Ptr a -> IntPtr
    intPtrToPtr :: IntPtr -> Ptr a