Opened 6 years ago

Closed 5 years ago

Last modified 5 years ago

#7867 closed feature request (duplicate)

Allow template-haskell to communicate with itself between compilation units through the interface file

Reported by: errge Owned by:
Priority: normal Milestone: 7.8.1
Component: Template Haskell Version: 7.6.3
Keywords: Cc: batterseapower
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: None/Unknown Test Case:
Blocked By: #3725, #8337, #8340, #8397, #8398 Blocking:
Related Tickets: Differential Rev(s):
Wiki Page:

Description (last modified by simonpj)

I'd like to be able to have something like this in the Q monad:

  type MessageName = String -- or something better, I don't know
  writeIface :: (Quasi q, Serializable a) => MessageName -> a -> q ()
  readIfaces :: (Quasi q, Serializable a) => MessageName -> q [a]

Or instead of returning [a], I'm of course fine with a Monoid, I just need the gathered results for every occurrance of the MessageName.

This has to be in the interface file, because I'd like this to work through compilation units, like class instances, I'd like to get every result in the readIfaces call that is ever imported from the current module (transitive closure).

Please note, that in a very hacky way this is already possible to do by abusing classes. Namely, I create an empty class for my message type. When I want to send a message, I create a new empty datatype named for the message and I make this an instance of the class. Then on the receiving side I just have to reify the class and I will receive all my instance names/messages.

This came up while implementing the HFlags library:

Maybe I shouldn't be proud of this, but we currently do the hacky way explained above. I would love to have a clean approach.

Change History (15)

comment:1 Changed 6 years ago by simonpj

Cc: batterseapower added
difficulty: Unknown

Annotations might do the job. I'm not sure wether TH gives you enough hooks to interact with annotations (Max would know), but they are properly persisted across interface files.


comment:2 Changed 6 years ago by simonpj

Description: modified (diff)

comment:3 in reply to:  1 ; Changed 6 years ago by errge

Hi Simon,

Thanks for your reply.

I actually found annotations before reporting this issue and my plan was to look into them and if they are persisted in interface files then add some access for them from Template Haskell reify (currently that's not there), but this is what I found:

errge@curry:/tmp $ cat Foo.hs
module Foo (foo) where

{-# ANN foo "my message" #-}
foo x = x
errge@curry:/tmp $ ghc -c -o Foo.o Foo.hs 
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
errge@curry:/tmp $ cp Foo.hi Foo.hi.old
// Edit file to have "my message newer"
errge@curry:/tmp $ cat Foo.hs
module Foo (foo) where

{-# ANN foo "my message newer" #-}
foo x = x
errge@curry:/tmp $ ghc -c -o Foo.o Foo.hs 
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
errge@curry:/tmp $ cmp Foo.hi.old Foo.hi
errge@curry:/tmp $ echo $?
errge@curry:/tmp $ ghc --version
The Glorious Glasgow Haskell Compilation System, version 7.6.2

So my annotation is definitely not persisted in the Foo.hi file with 7.6.2. Am I doing something wrong? Is this different with GHC HEAD?

comment:4 in reply to:  3 Changed 6 years ago by errge

Blocked By: 3725 added
Status: newinfoneeded

Nevermind my previous comment, I found the relevant bug:

OK, I will continue by investigating if it's possible to get this stuff from template haskell and if not, how hard would be to add it...

I'll get back to this ticket if I have or need further info.

comment:5 Changed 5 years ago by igloo

Status: infoneedednew

comment:6 Changed 5 years ago by errge

Ohh, sorry, I forgot to continue this ticket.

So, I looked a bit around, and Annotations sounds good and the way to go.

I checked, there is no Template Haskell functionality either for getting back annotations when reifying functions or modules (there is no module reification at all). Also, there is no support for generating annotations from TH.

If I have time to take care of either one, I'll try, but I'm a bit underqualified :)

comment:7 Changed 5 years ago by errge

Blocked By: 8337 added

comment:8 Changed 5 years ago by errge

Blocked By: 8340 added

comment:9 Changed 5 years ago by errge

Blocked By: 8397 added

comment:10 Changed 5 years ago by errge

Blocked By: 8398 added

comment:11 Changed 5 years ago by errge

difficulty: UnknownEasy (less than 1 hour)
Milestone: 7.8.1
Status: newpatch

2 of 5 of the patches is already merged and the functionality in the rest of the 3 covers this ticket fully. Changing milestone to 7.8.1 and difficulty to easy, action to patch.

comment:12 Changed 5 years ago by simonpj

errge, I've been talking to Austin

It's very late in the day for 7.8, and we are still discussing the design. So I don't think we can put this in 7.8, otherwise we'll mess it up and/or commit to a design that we subsequently regret.

Can you start a GHC Trac wiki page to describe clearly

  • The use-case you have in mind. Along the lines you have given recently in #8337, but can you give a small concrete example? I still don't get it; for example, is tcp_connect_timeout exported, and if not what good is it?
  • The programmer's-eye view of the design (eg the TH API changes). For example, I'm very dubious about qReifyModules because the notion of the "list of modules that are known to the compiler" is such an ill-defined one. Imagine if you were writing a new Haskell compiler: how would you implement that API? What is its actual semantics?
  • A sketch of the implementation changes needed (and links to tickets)

That would help more people participate in the conversation. At the moment you are implementing solutions to problems that we don't understand!



comment:13 Changed 5 years ago by errge

Simon, first of all thanks for your reply.

I'll do the wikipage, but in the meantime I'd like to clear up some confusion by answering your specific questions and giving further pointers.

  • You're right about qReifyModules, that's maybe the most shaky part of this. I actually don't like the current way how this is handled inside GHC, but I'm quite sure that I don't see enough to judge this. Currently GHC slurps up interface files and then uses them all while compiling different source files with --make. This results in the ill-definedness you are talking about, but this is unfortunately already visible via the TH API, in #8408 I show that with a concrete example. Let's discuss this in #8398, maybe we can come up with a better definition for module listing, I'll give it a try over the weekend to implement a better patch.

I'm all in for implementing this feature slowly and cleanly for a future release if not for 7.8, right about that. My end goal is to have the right and general compiler features, so the maintenance burden is minimal on me too regarding hflags and new GHC versions :)

On the other hand, I'd really like to have a working hflags as soon as I can: by using dirty tricks internally inside hflags (like communicating via instances instead of annotations). The only missing puzzle piece that I desperatly need support for from the compiler is to make a module orphan from TH.

Currently I can do this by generating a fake orphan rule, but that would generate a warning for the user when compiling with -Wall, I'd like to have a warningless way.

Can we maybe come up with a design/solution just for #8337 that is small and easy enough to be surely not messed up or regretted later? I still prefer option 1, stating that module anntations make a module orphan seems pretty harmless, but useful from my point of view. The patch would be also some clean and concise.

I'll report back here when the more general summarizing wikipage that you asked for is ready and then we can continue to work on the more difficult changes for annotation reification and module listing in a more principled way.

Can the already merged patches for #3725 and #8340 stay, please?

Thanks, Gergely Risko

Last edited 5 years ago by errge (previous) (diff)

comment:14 Changed 5 years ago by errge

The wiki page is ready.

All comments are welcome.

comment:15 Changed 5 years ago by errge

Resolution: duplicate
Status: patchclosed

Closing this bug report as we now have a wiki article that provides an overview and the related tickets that still need to get fixed for the feature to get implemented.

This bug will not get any patches and having it open just creates more confusion than benefit.

Last edited 5 years ago by simonpj (previous) (diff)
Note: See TracTickets for help on using tickets.