wiki:Design/BuildSystem

Version 10 (modified by igloo, 6 years ago) (diff)

--

Re-structuring the GHC build system

This page describes our current plan (Aug 08) for making GHC's build system more malleable. The basic plan is this:

  • Use our own build system for the work of actually building packages from source to .a/.so, including preprocessing (hsc2hs et. al.).
  • Use Cabal to preprocess the .cabal file and generate metadata in the form of Makefile bindings for our build system to use.
  • Use Cabal to generate the InstalledPackageInfo and do registration.
  • Use Cabal for Haddocking, installing, and anything else we need to do.

The advantages of this are:

  • The critical parts of the build system are under our control, and are easily modifiable.

  • Modifying the build system does not require modifying Cabal. We rely on a stable, slowly-varying version of Cabal, not on the leading edge. That take pressure off the Cabal developers, and means that GHC can use a version of Cabal that has survived quite a bit of testing.
  • Development is easier, because 'make' will preprocess files too. Right now if you modify a .y or .hsc file, you need to tell Cabal to preprocess again before saying 'make' (this is a regression from pre-Cabal).
  • We can make improvements that would be hard in Cabal, such as making libraries depend on each other.
  • It ought to be easier to reinstate HC bootstrapping, since we rely less on Cabal to get us to a .a file.
  • Compared to the pre-Cabal build system, we're not duplicating the package metadata or the code that processes it, only the build rules.

Detailed plan

  • Modify cabal-bin.hs with a new command to generate the Makefile bindings for a package, into a file e.g. ghc-build.mk. That is, cabal-bin.hs imports the Cabal libraries and uses them to support all of Cabal's existing features, plus the new one.
  • There is just one cabal-bin executable for the whole GHC build tree.
  • Question: should we rename cabal-bin.hs to ghc-cabal.hs? (Simon, Manuel say: yes.)
  • The version of Cabal used by cabal-bin.hs does not need to be an up-to-the-minute bleeding-edge version. It should be stable and vary slowly. We suck a new version of Cabal into the GHC build system manually, rather than mirroring the Cabal HEAD.
  • The makefile-generation stuff (which Duncan dislikes) can be removed from Cabal itself by the Cabal crew at their leisure. In effect, that code now lives in cabal-bin.hs.
  • libraries/Makefile puts a GNUmakefile into each library subdir, with identical contents, something like
    TOP=../..
    -include ghc-build.mk
    include $(TOP)/cabal-package.mk
    
    The boilerplate in cabal-package.mk is written by hand (not generated), and contains all the make rules required implement the desired make targets.
  • In each subdir we support various make targets, e.g.
    • make configure, configures the package and uses cabal-bin to generate ghc-build.mk
    • make all, builds the .a, and registers (using cabal-bin). Builds dependencies automatically (or perhaps not: calculating dependencies in GHC takes a while, and traditionally we've done this on demand only).
    • make install uses cabal-bin.
  • libraries/Makefile just invokes make in the subdirs in the appropriate order.

Improvements for later

  • We want dependencies from every object file on the .a files of the packages that it depends on. This way we can make it possible to modify a library module and say 'make' and have everything rebuilt that needs to be rebuilt (including the stage2 compiler). Note that we need to know about indirect as well as direct package dependencies.
  • Build multiple libraries in parallel
  • It should be possible to "make distclean" without configuring first. Mostly this just means that we need to, for example, remove both prog and prog.exe when cleaning, as we don't know if we are on Windows or not.
  • The "vanilla" way should actually have a name. Currently, there's no nice way to /only/ build something the profiling way, as you can't just not put "v" in WAYS.
  • We could simplify a lot of stuff if all of the inplace installations went into TOP/inplace, and that had the same layout as an installed tree. The main thing holding us back is that both stage1 and stage2 ghc currently want a binary called "ghc".