|Version 42 (modified by simonpj, 5 years ago) (diff)|
Using the Build System
- Source trees and build trees
- Booting, configuring, cleaning
- Steps to prepare for building
- Building things
- Developing in a GHC build tree
- Installing extra packages
- Standard Targets
NOTE: this page now describes the new GHC build system, due to be added to GHC in April 2009. For documentation of the "old" build system, see Building/Old/Using?.
Using the Build System
This part of the guide is about using the build system, that is, how to interact with the build system when you're developing some part of GHC, its libraries or tools. The section aims to be comprehensive; for a quick start, read Building/Hacking first.
Source trees and build trees
Sometimes we want to separate the build tree from the source tree. There are a few advantages to doing this:
- You can make multiple different builds from the same sources, perhaps for testing different build settings, or for building on different platforms.
- You might want to put the source tree on a remote, backed-up, filesystem, but keep your build tree on a local fast unbacked-up drive (this is a configuration we use regularly at GHC HQ). It doesn't matter if you lose the build tree: it can easily be regenerated.
- It's easy to blow away a build tree and start again, without modifying your source tree. make maintainer-clean is usually good for this too, but it can miss files that it doesn't know about, or files that are remnants from older versions of GHC.
- It helps to avoid mistakes whereby you edit a file that happens to be automatically generated, instead of the original source file (e.g. editing config.mk instead of config.mk.in). If you only edit files in the source tree, then this can't happen.
However, if you just want to build the software once on a single platform, then your source tree can also be your build tree, and you can skip the rest of this section.
Windows users: so far as we know, symbolic links do not work right on MSYS at least, so we never use separate source and build trees on Windows.
A build tree is just an exact copy of the source tree, except that every file in it is a symbolic link to the appropriate file in the source tree. There are "standard" Unix utilities that make such copies, so standard that they go by different names: lndir and mkshadowdir are two (If you don't have either, the GHC source tree contains sources for the X11 lndir check out utils/lndir).
You need to be a bit careful, though, that any new files you create (if you do any development work) are in the source tree, not the build tree!
Booting, configuring, cleaning
GHC uses the autoconf tools in the standard Unixy way, described in more detail in subsequent subsections on this page:
--------------- | | | Source code | <--------------------- | | | --------------- maintainer-clean | | sh boot (runs autoconf) | | | v | ---------------------------- | | Source tar-ball |>--------| | ./configure | | | libraries/*/GNUmakefile | | | libraries/*/ghc.mk |<-- | ---------------------------- | | | | | ./configure distclean | | | | v | | ------------------ | | | Configured |>------------ | | mk/config.mk | | | | |>------------------- ------------------ | | | | | make | | | | | v | | ------------------ | | | Built |>------------ | | | | | |>------------------- ------------------
Steps to prepare for building
We gave a quick getting-started introduction to building GHC in Building/QuickStart, the following sections describe each step in more detail.
In the build system, the path to the top of your build tree is referred to as $(TOP), and we will use that convention in the following sections.
Generate configure scripts
NOTE: if you're starting from a source distribution that you downloaded from the GHC web site, rather than darcs sources, you can skip this step. Source distributions come with configure scripts pre-generated for you.
Change directory to $(TOP) and issue the command
$ sh boot
(Note: the $ is the prompt. You don't type that bit.) This runs autoreconf on the various configure.ac scripts in the GHC build tree, generating configure scripts and other miscellaneous files.
You have to re-do this step if (and only if) you ever change one of the files that autoconf uses to generate the configure scripts, such as configure.ac, or aclocal.m4.
Run the configure script
Run the configure script, thus:
$ ./configure <args>
configure's mission is to scurry round your computer working out what architecture it has, what operating system, whether it has the vfork system call, where tar is kept, whether gcc is available, where various obscure #include files are, whether it's a leap year, and what the systems manager had for lunch. It communicates these snippets of information in two ways:
- It translates various files such as mk/config.mk.in to mk/config.mk, substituting for things between "@" brackets. So, "@HaveGcc@" will be replaced by "YES" or "NO" depending on what configure finds. mk/config.mk is included by every Makefile (directly or indirectly), so the configuration information is thereby communicated to all Makefiles.
- It translates mk/config.h.in to mk/config.h. The latter is #included by various C source files, which can thereby make use of configuration information.
configure takes some optional arguments. Use ./configure --help to get a list of the available arguments. Here are some of the ones you might need:
- Specifies the "build platform" (see platform names). This is usually only necessary on Windows, where you should set it to --build=i386-unknown-mingw32.
- Set the "host platform" (see platform names). Usually not necessary.
- Set the "target platform" (see platform names). Usually not necessary.
- Specifies the path to an installed GHC which you would like to use. This compiler will be used for compiling GHC-specific code (eg. GHC itself). This option cannot be specified using build.mk (see later), because configure needs to auto-detect the version of GHC you're using. The default is to look for a compiler named ghc in your path.
- Specifies the path to the installed GCC. This compiler will be used to compile all C files, except any generated by the installed Haskell compiler, which will have its own idea of which C compiler (if any) to use. The default is to use gcc. On Windows, this should be set to the gcc that comes with MinGW, which by default is c:/mingw/bin/gcc.
- Specifies which ld program to use. Normally only necessary on Windows, where you should set it to the ld that comes with MinGW, which is usually c:/mingw/bin/ld.
- Tells the build system where you would like GHC to be eventually installed. You don't have to install GHC in order to use it: it is entirely possible to work on GHC and test it without ever modifying anything outside the build tree. However, if you do with to install GHC, then the --prefix directory is the root of the install tree. Typciallly on Unix systems the default for --prefix is /usr/local. See also Building/Installing for more details.
Next, you say how this build of GHC is to differ from the standard defaults by creating a new file mk/build.mk in the build tree. This file is the one and only file you edit in the build tree, precisely because it says how this build differs from the source. (Just in case your build tree does die, you might want to keep a private directory of build.mk files, and use a symbolic link in each build tree to point to the appropriate one.)
mk/build.mk is purely for overriding settings that are found in mk/config.mk. You should never edit mk/config.mk directly, since it is automatically generated from mk/config.mk.in by configure. Neither should you edit mk/config.mk.in; just provide your settings in mk/build.mk.
We provide a sample file, mk/build.mk.sample, which you can copy to mk/build.mk and edit. It provides a number of pre-defined configurations, such as a "fast build" or a "development build". Take a look in the file itself for details.
To understand more about what you can put in mk/build.mk, read on.
Common build.mk options
The following are some common variables that you might want to set in your mk/build.mk. For other variables that you can override, take a look in mk/config.mk.in.
(default: -H32m -O)
Options passed to GHC for all Haskell compilations.
Options added when compiling GHC (all stages)
Options added when compiling the stage 1 GHC.
Options added when compiling the stage 2 GHC.
Options added when compiling the stage 3 GHC.
(default: -O2 -XGenerics)
Options added when compiling the libraries.
Set to YES to enable profiling for GHC itself (stage 2). You'll also need to add GhcLibWays += p to make this work.
Set to YES to pass -debug when building GHC (stage 2).
(default: v p)
Ways in which to build the libraries. Must contain at least v (the vanilla way). Also contains p by default (profiling). For other ways, see mk/config.mk.in.
(default: YES if supported, NO otherwise)
When set to YES, static library object files are split into smaller pieces. This means that less of the library code needs to be linked into the final application, which makes smaller binaries. It takes longer to build libraries this way, though.
(default: YES if supported, NO otherwise)
When set to YES, these build the docbook documentation (e.g. the users guide) as HTML, PS and PDF respectively.
How to make GHC build quickly
The GHC build tree is set up so that, by default, it builds a compiler ready for installing and using. That means full optimisation, and the build can take a long time. If you unpack your source tree and right away say ./configure; make, expect to have to wait a while. For hacking, you want the build to be quick - quick to build in the first place, and quick to rebuild after making changes. Tuning your build setup can make the difference between several hours to build GHC, and less than an hour.
Here are the build.mk settings that we use to build fast:
# My build settings for hacking on stage 2 SRC_HC_OPTS = -H32m -O -fasm -Rghc-timing GhcStage1HcOpts = -O -fasm GhcStage2HcOpts = -O0 -DDEBUG -Wall GhcLibHcOpts = -O -fasm -XGenerics GhcLibWays = v SplitObjs = NO GhcBootLibs = YES
What do these options do?
- SRC_HC_OPTS = -H32m -O -fasm -Rghc-timing
- These options are added to the command line for all Haskell compilations. We turn on -fasm, because that halves compilation time at the expense of a few percent performance. -Rghc-timing prints out a line of timing info about each compilation. It's handy to keep an eye on. -Wall turns on all the warnings; GHC is meant to be warning-clean with -Wall.
- GhcStage1HcOpts = -O -fasm
- Build stage 1 optimised: we're going to be rebuilding stage 2 a lot, so we want the compiler that does the building to be fast.
- GhcStage2HcOpts = -O0 -DDEBUG -Wall
We turn off optimisation here, assuming you'll be modifying and
testing stage 2. With optimisation off, rebuilding GHC after
modifying it will be much quicker, not only because the
individual compilations will be quicker, but also there will be
fewer dependencies between modules, so much less stuff is recompiled
after each modification.
Also we turn on -DDEBUG, because that enables assertions and debugging code in the compiler itself. Turning on DEBUG makes the compiler about 30% slower.
- GhcLibHcOpts = -O -fasm -XGenerics
- You almost certainly want optimisation on when building libraries, otherwise the code you build with this compiler goes really slowly. -fgenerics add generics support to the libraries - you can turn this off if you like (it'll make the libraries a bit smaller), but you won't be able to use Generics in the code you build against these libraries.
- GhcLibWays = v
- Normally the profiled libraries are built. Setting GhcLibWays to just "v" disables this, so you only build the normal libs.
- SplitObjs = NO
- Object splitting causes each module to be split into smaller pieces in the final library, to reduce executable sizes when linking against the library. It can be quite time and memory-consuming, so turn it off when you're hacking.
- GhcBootLibs = YES
- If you're just interested in working on GHC, then you probably don't want to build the "extralibs" libraries that we normally ship with GHC. So when getting the sources, run darcs-all without the --extra option. Alternatively, even if you have the libraries in your tree, you can stop them being built by setting GhcBootLibs in your build.mk.
At this point you have made yourself a fully-configured build tree, so you are ready to start building real things.
The first thing you need to know is that you must use GNU make. On some systems (eg. FreeBSD) this is called gmake, whereas on others it is the standard make command. In this document we will always refer to it as make; please substitute with gmake if your system requires it. If you use a the wrong make you will get all sorts of error messages (but no damage) because the GHC Makefiles use GNU make's facilities extensively.
To just build the whole thing, cd to the top of your build tree and type make. This will prepare the tree and build the various parts in the correct order, resulting in a complete build of GHC that can even be used directly from the tree (as inplace/bin/ghc-stage2), without being installed first.
Order of the build
Here is a high level view of what happens when you build GHC:
- First, we build a few packages that GHC itself depends on, such as Cabal, and filepath, using your installed GHC. These packages are under libraries, and each is built in a subdirectory dist-boot; for example the bootstrap build of Cabal will be in libraries/Cabal/dist-boot.
- Then we build GHC against these packages, still using your installed GHC. This is called the "stage 1" compiler (see stages). You can run the stage 1 compiler by invoking inplace/bin/ghc-stage1. The stage 1 build of GHC happens in compiler/stage1.
- The stage 1 compiler is now used to build all the packages in the libraries subdirectory, and the runtime system in rts.
- Finally, the stage 1 compiler is used to build GHC itself again, this time against the libraries we just built. This GHC is called stage 2, and can be invokes as inplace/bin/ghc-stage2.
There's an optional final stage, using the stage 2 compiler to build a stage 3 compiler, but this isn't strictly necessary, and is only used to check that the stage 2 compiler is working properly.
What to do after make has finished
If make completes successfully, then it should have created inplace/bin/ghc-stage2, which is a GHC binary you can run directly. It supports all the usual features, including GHCi if you pass the --interactive flag.
In fact, the inplace directory looks a lot like an installed copy of GHC (see Building/Installing): there is a bin subdirectory containing various programs that can be run, including ghc-pkg, Haddock and hsc2hs.
You can now run the testsuite, see Building/RunningTests. You can now install GHC, by typing make install.
What to do if you get a build failure
GHC is a complex system, with many platform-dependent components. We try our best to make sure it builds out of the box as often as possible, but build failures are not uncommon. If you get some kind of failure, don't panic.
The chances are, someone else already encountered the same problem as you and has reported it on a mailing list or as a ticket. Search for the error message using your favourite search engine, and if that doesn't turn anything up then search directly on this Trac. Finally, if you don't find a solution:
Developing in a GHC build tree
This section describes how to make changes to the sources of GHC, or its libraries and tools, and use the build system to build and test those changes.
Bringing the whole tree up to date
The easy way to bring the tree up to date after making some changes is simply to go to the top of the tree and say make. The build system has enough dependencies encoded into it that this should rebuild everything that needs to be rebuilt, in the correct order. The downside is that it will build everything right though to the stage2 compiler and including all the documentation, which might be overkill if all you wanted to do was to test a small change to GHC.
Building a single sub-component
Each subdirectory of the source tree has a stub makefile, most of which follow this pattern:
dir = libraries/base TOP = ../.. include $(TOP)/mk/sub-makefile.mk
the main purpose of the stub makefile is to invoke make at the top-level of the tree: GHC's build system is non-recursive, so in effect there is really just one Makefile, at the top level. However, the stub makefile has another purpose: it passes a target to the top-level Makefile, telling it to build just the components of the system in this directory. For example, when you say make in the rts directory, this is actually equivalent to
$ make -C .. all_rts
where "-C .." tells make to invoke the Makefile in the directory "..", and all_rts is the target that makes everything in the rts subdirectory. Equivalently, make all_libraries/base at the top level would build everything in the libraries/base subdirectory. To understand how the all targets are defined, see standard targets.
You can also clean a single component of the tree, just by saying make clean in a subdirectory. Again this is equivalent to issuing a command at the top of the tree of the form make clean_libraries/base.
Rebuilding the GHC binary after making changes
Suppose you want to make a small change to GHC itself and test it. Assuming that your tree is already up to date, the best way to do this is as follows:
$ cd ghc $ make stage=2
This will bring the stage 2 compiler up to date only. Setting stage=2 has the effect of disabling all the rules that build the stage 1 compiler, so the build system will ignore the fact that the stage 1 compiler is also out of date, and hence all the libraries are also potentially out of date. If you did make from the top-level, all of these dependencies would be taken into account, and a lot of rebuilding would probably ensue. There's another target that takes an even quicker shortcut:
$ cd ghc $ make 2
This is like make stage=2, except that it omits the dependency-building phase (make 2 is in fact just shorthand for make stage=2 FAST=YES; see Fast Rebuilding below). If you have changed the imports in any modules, those new dependencies will not be taken into account by the build system, so you might get a build failure. On the other hand, this shortcut usually works and the few seconds it saves can make GHC development a much more interactive experience. There are also targets
- make 1
- make 3
to make the stage 1 and stage 3 compilers respectively. These targets work in both the ghc and compiler subdirectories (Commentary/SourceTree).
Building a single file
It's possible to tell make to build a single file, from any subdirectory in the tree. For example, suppose I want to build just the module Control.Monad in the base package, I can do it like this:
$ make libraries/base/dist-install/build/Control/Monad.o
(you have to know that dist-install is the distdir for a package, and object files are put in the subdirectory build). It's also possible to do this from the libraries/base subdirectory:
$ cd libraries/base $ make dist-install/build/Control/Monad.o
suppose you wanted to build this module with a few extra flags, perhaps because you want to see what GHC's optimiser is doing on this module:
$ rm dist-install/build/Control/Monad.o $ make dist-install/build/Control/Monad.o EXTRA_HC_OPTS=-dcore-lint
you could also cut-and-paste the command-line to add flags, but sometimes the EXTRA_HC_OPTS method is more convenient.
Often when you're working in a particular part of the tree, the rest of the tree is up to date, and you just want to rebuild the component you're working on after making changes. To speed things up, we'd like to avoid having make check that everything else in the tree is up-to-date before building the component we're working on. So the GHC build system provides a couple of ways to do this:
- make FAST=YES
- Only has an effect in a subdirectory. Setting FAST=YES causes make to omit rebuilding the .depend file (if any), and also omits some of the phases. FAST=YES is allowed in conjunction with any other target; for example, it makes sense when rebuilding a single file, as in the previous section.
- make fast
- Shorthand for make all FAST=YES.
Another useful trick is
- make stage=0
- Does not build any GHC stages at all. stage=0 can be used in combination with other targets and settings.
Installing extra packages
The boot libraries are built as part of building GHC; they are built with the stage1 compiler, and imported when the stage2 compiler is compiled with stage1.
All other libraries are stand-alone Cabal packages, and the build system knows nothing about them. Nevertheless, it is common to want to install extra packages for the GHC in your build tree. Here's how to do it.
The easiest way is to just use cabal-install:
$ cabal install --with-ghc=$(TOP)/inplace/bin/ghc-stage2 <package>
where $(TOP) stands for the root directory of your build tree, and <package> is the name of the package you want to install. This will install the package in your home directory (e.g. somewhere under ~/.cabal/lib on a Unix system) , so you'll probably want to remove it by hand when you've finished.
There's also a way to build and register a package with GHC but without actually installing it in your home directory. For example, if you want to install a package from its darcs repository:
$ cd $(TOP)/libraries $ darcs get http://darcs.haskell.org/packages/parallel
(You don't have to download the package to $(TOP)/libraries; it can go anywhere.) Now build it using Cabal, telling Cabal to use the GHC from your build tree:
$ runhaskell Setup configure --with-ghc=$(TOP)/inplace/bin/ghc-stage2 $ runhaskell Setup build $ runhaskell Setup register --inplace
The --inplace flag to register tells Cabal not to copy the compiled package, but rather to leave it right where it is, and register this location in the package database in your GHC build tree.
The following targets work both at the top level, and in any subdirectory of the tree. When used in a subdirectory, they apply only to the components of the system in that directory.
- Invoking make help in any directory of the build tree will elicit a message describing the targets available in that directory.
- (default target, can be omitted). Builds all the targets for this directory. At the top level, builds everything that needs to be built for a GHC installation, including the stage 2 GHC, all libraries and documentation. After make, make install will not need to do any further rebuilding.
- Delete all files from the current directory that are normally created by make. Don't delete the files that record the configuration.
- html, pdf, ps
- Build documentation below the current directory (if any) in HTML, PDF, or Postscript format respectively.
To see how these targets are defined: Building/Architecture/Idiom/StandardTargets.
The following targets are accepted only by the top-level Makefile:
- installs the things built by all. Where does it install them? In the places you specified when running configure, principally set by the --prefix flag; see Building/Installing.
- The same as clean, but also removes files created by configure. distclean preserves files that would normally go in a source distrubtion. Note that after make distclean you will have to run ./configure before building again.
- The same as distclean, but also removes files generated by sh boot. The result of make maintainer-clean should be a completely clean source tree, containing only revision-controlled files.
- Builds a binary distribution. A binary distribution is a copy of the build tree with the source files removed, such that it can be installed on a different system simply by issuing `./configure; make install`.
- Builds a source distribution. A source distribution is a source tree for GHC, complete with some extra files included for convenience so that the user doesn't need to install autoconf, Happy, or Alex in order to build it. Hence make sdist only works in a completely built tree.