Process and thread init/deinit hooks for C libraries

Haskell's FFI for C seems geared toward C as an optimization. But in many cases, we use the C FFI merely hook into a 3rd-party library. In this case, the FFI seems a bit lacking. In my specific case, I want to link against libmysqlclient_r to perform MySQL queries. However, the library requires global init and deinit. Worse, it requires init/deinit for *each* thread that wishes to interact with its API. This type of thing is not entirely uncommon among C libraries. Of course, we can bend over backward to inter-operate with such libraries (e.g. wrap the C API with our own versions that ensure proper initialization), but the process is painstaking and often requires decent performance overhead.

I would like to request some sort of hook registration where the FFI can be used to accomplish these common tasks. Perhaps it's too difficult, but it doesn't hurt to ask!

Trac metadata

Trac field	Value
Version	7.8.3
Type	FeatureRequest
TypeOfFailure	OtherFailure
Priority	normal
Resolution	Unresolved
Component	Compiler
Test case
Differential revisions
BlockedBy
Related
Blocking
CC
Operating system
Architecture

changed weight to 5

added Tfeature request Trac import labels

closed

have you looked at https://hackage.haskell.org/package/mysql and its higher level cousin mysql-simple ?

that sort of thread local storage machinery is totally manageable in userland, use the forkOS primitive to start a thread in a dedicated distinct OS thread.

point being, this sort of thing is manageable in userland (see also how folks hand ffi bindings to libs that need thread local storage). So unless i'm misunderstanding this ticket, im going to close it for now. Haskell-cafe or haskell reddit are good places to ask for pointers on C ffi engineering, though theres also numerous resources on the wiki and many many c ffi bindings on hackage.

Trac metadata

Trac field	Value
Resolution	Unresolved → ResolvedInvalid

The mysql package punts on this issue because it uses the single-threaded lib, so it can get away without calling most of the init/deinit functions manually. One issue is that the main library init function is not thread safe so must be called before any threads are created (or mutex protected). In a C program, this is easily done by adding a few calls in main. The trouble when connecting Haskell apps to these libraries is that we don't have that kind of access to "main" and thus have to come up with awkward APIs and monads, etc. to make sure everything happens properly. Of course, as I said, it's possible to do this in user-land, its just painful and makes Haskell less friendly in real-world scenarios where you want to interop with stuff like this.

I"ll repeat, you can get the behavior you want with forkOS and finalizers http://www.haskell.org/ghc/docs/latest/html/libraries/base-4.7.0.1/Foreign-ForeignPtr.html and http://www.haskell.org/ghc/docs/latest/html/libraries/base-4.7.0.1/Control-Concurrent.html

if you wish to have some help getting the design you want, please ask on #haskell on irc freenode, haskell-cafe mailing list, or the haskell reddit.

Excellent. If it really is cleanly possible without changes to the compiler, then that is most definitely better! Thanks.

Note that because ghc threads (aside from forkOS threads) aren't bound to specific OS threads, you'll have to do some intermediating the calls via the bound thread. But these are thing which folks on haskell-cafe and those other fora will be more than able to help you with

(Let me know if this discussion has moved elsewhere.)

I've wondered about this question, too. I have a multithreaded Haskell program that makes some calls into a JVM. To do this, each OS thread calling into the JVM needs to have some data initialized. While I could do the forkOS approach, I want to have any Haskell thread call into Java, rather than having to send a message to my forkOS'd Haskell thread, wait for it to be scheduled, have it do the call, send a message back, etc. One idea is to have a capability-local data structure, but that doesn't work because multiple OS threads might execute a capability (at different times). So I have to manage the initialization in the C code: whenever the C code is invoked in a given OS thread, the thread checks whether its data structures are initialized.

No conclusions here, just wanted to chime with my experience on this topic.

added FFI label

added Pnormal label