clean up dependency and usages handling in interface files

changed milestone to %7.10.1

changed weight to 5

added Ttask TemplateHaskell Trac import labels

Trac metadata

Trac field	Value
BlockedBy	#1480 (closed) →

I don't know what the status is here. Is it cleaned up? I did find this commit: 50d4cd7a

Author: Simon Peyton Jones <>
Date:   Tue Nov 5 15:17:22 2013 +0000

    More comments on Usage and Dependencies

Thomie, unfortunately this is not done yet and that is totally on me :(

The main task here is to "fix TH module reification to reify direct import list, not usages"

So just having more comments is not enought, actual coding (and probably some refactoring) has to happen. On the other hand, I am pretty sure that the better comments will help me to get back up to speed more quickly when I sit down to handle this.

I still plan to work on this, so feel free to leave me as owner unless someone else wants to tackle this before me in which case I am happy to discuss more if questions come up.

A good first step would be to write a precise specification for the change you propose. What will change? Is is just refactoring or will (TH) programmers see a difference? What will the difference be? I have no idea at the moment.

Once the specification is clear, and agreed, the implementation can follow. But the latter should not precede the former.

Simon

Simon, yes of course, it is important to first agree on what we want.

On the other hand, this is a much smaller change than my changes from previous year, so I do not think that we need a whole wiki page again.

Let me put it here what I think the issue is and then you can point it out where I was not precise enough and we can complete this to a "specification".

Take the following program:

{-# LANGUAGE TemplateHaskell #-}

import Language.Haskell.TH
import Language.Haskell.TH.Syntax

$(do let mod = Module (PkgName "binary-0.7.1.0") (ModName "Data.Binary")
     ModuleInfo info <- reifyModule mod
     runIO $ mapM_ print info
     return [])

main = return ()

On my machine, the output is:

Module (PkgName "base") (ModName "Data.Either")
Module (PkgName "base") (ModName "Data.Maybe")
Module (PkgName "base") (ModName "Data.Word")
Module (PkgName "base") (ModName "GHC.Base")
Module (PkgName "base") (ModName "GHC.IO")
Module (PkgName "base") (ModName "GHC.IO.IOMode")
Module (PkgName "base") (ModName "GHC.Word")
Module (PkgName "base") (ModName "Prelude")
Module (PkgName "base") (ModName "System.IO")
Module (PkgName "binary-0.7.1.0") (ModName "Data.Binary.Class")
Module (PkgName "binary-0.7.1.0") (ModName "Data.Binary.Generic")
Module (PkgName "binary-0.7.1.0") (ModName "Data.Binary.Get")
Module (PkgName "binary-0.7.1.0") (ModName "Data.Binary.Get.Internal")
Module (PkgName "binary-0.7.1.0") (ModName "Data.Binary.Put")
Module (PkgName "bytestring-0.10.4.0") (ModName "Data.ByteString")
Module (PkgName "bytestring-0.10.4.0") (ModName "Data.ByteString.Lazy")
Module (PkgName "bytestring-0.10.4.0") (ModName "Data.ByteString.Lazy.Internal")
Module (PkgName "ghc-prim") (ModName "GHC.Types")

And looking at http://hackage.haskell.org/package/binary-0.7.1.0/docs/src/Data-Binary.html it can be seen that we do not import GHC.Types in binary.

So from the programmer point of view, this change should make the ModuleInfo list returned by module reification only contain the actual direct imports.

So this is my goal, before discussing implementation details inside GHC, may I ask if this description is clear enough and if others agree with the goal?

Aha I see. So this is really a bug, thus:

Template Haskell offers

reifyModule :: Module -> Q ModuleInfo
data ModuleInfo = ModuleInfo [Module]
  -- Contains the import list of the module

The "import list of the module" M presumably means the list of modules that M imports directly. (Clarifying the comment would be good.)
But the program above shows that reifyModule reports a much longer list. (NB: A smaller test case would make easier.)

So the specification is simply to make reifyModudule behave as advertised.

Now I understand, it's easy enough to implement, at least for modules in the same package as M. Just pick modules whose Usage has Just _ in its usg_exports field. Sadly, for modules from other pacakges we simply don't record the necessary information in interface files at all. It would not be hard to fix this.

If I remember correctly the difference is not the package boundary, but the compilation unit boundary. So if you compile your modules with different GHC invocations by using -c instead of --make, then you will not have the usg_exports either, am I right?

If that is the case I prefer us to do the right fix and include this information in the interfaces files and do not introduce different behavior depending on compilation technique in the meantime.

I believe its the package boundary.

removed milestone

Moving to 7.12.1 milestone; if you feel this is an error and should be addressed sooner, please move it back to the 7.10.1 milestone.

changed milestone to %8.0.1

Milestone renamed

I don't understand how the explicit imports list of a module would be helpful to another module. I've never grokked the usecase for this.

I suggest just changing the documentation to indicate that it is a list of usages rather than imports, to resolve this. How's that sound? Assigning this ticket to myself, to implement via documentation.

Alternatively, this usages information could just get removed. Worth checking if there are uses of it on hackage. I am considering implementing reification of exports (https://ghc.haskell.org/trac/ghc/ticket/10391), which is what I think most people would want in module reification as that would be quite useful.

assigned to @trac-mgsloan and unassigned @trac-errge

added Pnormal label

Related to #22188

Using mi_usages for this field is wrong, (as pointed out in #1480 (closed)), mi_usages is ONLY for recompilation checking.
I find the interface for reifyModule distasteful as it requires doing a deep traversal of transitive dependencies through all package boundaries. This can be thousands of modules which don't bring anything into scope in the current module.
With #22188 then the mi_usages comes back to bite us more thoroughly, I have made it emit a warning if you try and traverse through a package which lacks the correct mi_usages information.

mentioned in commit 4e579aca

mentioned in merge request !9604 (closed)

mentioned in commit d73b6b36

mentioned in commit 40c9da19

mentioned in commit 5a3ac212

mentioned in commit 2b89c263

assigned to @mpickering and unassigned @mgsloan

mentioned in commit 3f3648db

mentioned in commit a5de39de

mentioned in commit 590ecbd8

mentioned in commit d9c8ea1a

mentioned in commit 20dc4049

mentioned in commit ce7358e7

mentioned in commit e5baf396

mentioned in commit 6083a96d

closed with commit 73e02068

mentioned in commit 73e02068

Trac field	Value
Version	7.7
Type	Task
TypeOfFailure	OtherFailure
Priority	normal
Resolution	Unresolved
Component	Template Haskell
Test case
Differential revisions
BlockedBy	#1480 (closed)
Related
Blocking
CC
Operating system
Architecture

clean up dependency and usages handling in interface files

Child items 0

Activity