Module re-exports at the package level

For various package reorganization purposes, especially for possibly turning base into a shim package that re-exports some modules from more specialized packages, good support for module re-exports would be nice.

I wrote up a design spec at ModuleReexports; this bug is to track the progress.

Trac metadata

Trac field	Value
Version	7.6.3
Type	FeatureRequest
TypeOfFailure	OtherFailure
Priority	normal
Resolution	Unresolved
Component	Package system
Test case
Differential revisions
BlockedBy
Related
Blocking
CC
Operating system
Architecture

changed weight to 5

added Tfeature request Trac import package system labels

This is primarily a Cabal question, isn't it? If Cabal wants to move this way, I'd be happy for GHC to move to accommodate.

Simon

I’ll open appropriate tickets against Cabal and haddock if there is consensus that the proposal is useful and the syntax and semantics are agreed on – currently, this is just a draft.

Hello nomeata, are you still working on this? This is on the hotpath for Backpack implementation work so I will probably go ahead and implement this if you are not working on it.

added backpack label

Not working on it here, so please do go ahead.

assigned to @ezyang

Here is an annoying problem for implementing this functionality.

Module reexports should roughly be analogous to how we deal with type/declaration reexports at the Haskell source level. To briefly overview how that mechanism works, we have a phase of compilation called "renaming", which takes all identifiers and figures out what true names they refer to. This is done by consulting the hi files associated with imported modules, which record the original name of any identifiers they export.

By analogy, the installed package information file is like an "hi" file, and we would like it to store a pointer to the original name of the module (in this case, the InstalledPackageId and original module name, if renaming is allowed.)

However, there is trouble in paradise. Entries in the InstalledPackageDb are created by the Cabal library; however, the Cabal library has no knowledge about module resolution (it only looks at the dependencies to do dependency resolution): all of this logic is in GHC proper (Packages and Finder). So at the time Cabal is writing the installed package info, it would need to consult GHC in order to figure out where modules actually came from! This seems like a design smell.

The alternative is to do module resolution in GHC. Here, we augment the creation of our module map (when processing -package flags) so that when we see a reexport, we consult the existing map and insert a pointer to precisely the original import. If the packages are passed to us in topologically sorted order (which Cabal seems to do today), then we only pay the cost of a single map lookup for every reexported module we encounter. The downside is that this mechanism is now no longer symmetric with original names.

It's not really clear which one is better.

Also, so minor UI things to watch out for:

1. Suppose a module Foo is defined by package a, and reexported by package b. I misspell my import and attempt to import module Fooo (sic). Clearly, when I give the list of corrections, I should only report Foo once. Now, suppose I have another module Foo defined by package c, so in our message we want to report that the module was found in multiple packages. In this case, we want to say "it was found in multiple packages: a (reexported by b), c" (and not just "a, c")
2. When we validate an entry in the installed package database, should we check if the reexported modules truly exist in the original package? Depending on what representation we have in the package database, this may not even be well defined. In any case, dependency validation should pick up most shenanigans.
3. In some functions, including packageDbModules from the GHC API, we asked to provide a Module for a re-exported module. Should the module returned be resolved to the original exporter, or still be the original name of the module?

SPJ thought of a good way around the implementation problem: we should implement a "mini" module finder in ghc-pkg, which simply takes all of the dependencies listed in the package, and reads their entries in order to resolve a module reexport. We hold the invariant that reexports in the installed package database point to the original name, so once we find the relevant entry, we know the original name.

Initial CR: https://phabricator.haskell.org/D41

changed weight to 7

Trac metadata

Trac field	Value
Priority	normal → high

changed milestone to %7.10.1

mentioned in commit 7f5c1086

closed

Trac metadata

Trac field	Value
Resolution	Unresolved → ResolvedFixed

This fix seems to break the current master build on OS X:

compiler/main/Packages.lhs:867:16:
     error: unterminated function-like macro invocation
          where -- ASSERT(m == m' && pkg == pkg' && e == e'
                   ^

compiler/HsVersions.h:39:9:  note: macro 'ASSERT' defined here
#define ASSERT(e)      if debugIsOn && not (e) then (assertPanic __FILE__ __LINE__) else
        ^
1 error generated.
<<ghc: 606003696 bytes, 82 GCs, 3823064/7170800 avg/max bytes residency (5 samples), 65M in use, 0.00 INIT (0.00 elapsed), 0.24 MUT (2.05 elapsed), 0.07 GC (0.08 elapsed) :ghc>>
make[1]: *** [compiler/stage1/build/.depend-v.haskell] Error 1
make: *** [all] Error 2

We really ought to switch to something like cpphs (which we can tweak to be better aware of Haskell syntax) for milestone:7.10.1 than having to keep in mind that there's now a 2nd cpp out there with slightly different semantics which keeps breaking builds... :-/

I pushed a bandaid fix [94873053/ghc]

commit 9487305393307d5eb34069c5821c11bb98b5ec90
Author: Edward Z. Yang <ezyang@cs.stanford.edu>
Date:   Sat Jul 26 10:41:28 2014 +0100

    Fix build on OS X due to macro-like string in comment
    
    Signed-off-by: Edward Z. Yang <ezyang@cs.stanford.edu>

mentioned in issue #9375 (closed)

added Phigh label