Changes between Version 14 and Version 15 of Building/Using


Ignore:
Timestamp:
Mar 30, 2009 1:30:22 PM (6 years ago)
Author:
simonmar
Comment:

Start documenting the new build system

Legend:

Unmodified
Added
Removed
Modified
  • Building/Using

    v14 v15  
    11
    22[[PageOutline]]
     3
     4'''NOTE''': this page now describes the new GHC build system, due to
     5be added to GHC in April 2009.  For documentation of the "old" build
     6system, see [wiki:Building/Old/Using].
    37
    48= Using the Build System =
    59   
    6 This rest of this guide is designed to help you even if you aren't
    7 really interested in Makefiles and systems configurations, but you
    8 need a mental model of the interlocking pieces so that they can make
    9 them work, extend them consistently when adding new software, and lay
    10 hands on them gently when they don't work.
    11 
    12 == History ==
    13 
    14 First, a historical note.  The GHC build system used to be
    15 called "fptools": a generic build system used to build multiple
    16 projects (GHC, Happy, !GreenCard, H/Direct, etc.).  It had a
    17 concept of the generic project-independent parts, and
    18 project-specific parts that resided in a project
    19 subdirectory.
    20 
    21 Nowadays, most of these other projects are using
    22 [http://www.haskell.org/cabal/ Cabal], or have faded away, and GHC is
    23 the only regular user of the fptools build system.  We decided
    24 therefore to simplify the situation for developers, and specialise the
    25 build system for GHC.  This resulted in a simpler organisation of the
    26 source tree and the build system, which hopefully makes the whole
    27 thing easier to understand.
    28 
    29 You might find old comments that refer to "projects" or "fptools" in
    30 the documentation and/or source; please let us know if you do.
    31 
    32 == Build trees ==
    33 
    34 If you just want to build the software once on a single platform, then
    35 your source tree can also be your build tree, and you can skip the
    36 rest of this section.
    37 
    38 We often want to build multiple versions of our software for different
    39 architectures, or with different options (e.g. profiling).  It's very
    40 desirable to share a single copy of the source code among all these
    41 builds.
    42 
    43 So for every source tree we have zero or more ''build trees''.  Each
    44 build tree is initially an exact copy of the source tree, except that
    45 each file is a symbolic link to the source file, rather than being a
    46 copy of the source file.  There are "standard" Unix utilities that
    47 make such copies, so standard that they go by different names:
    48 {{{lndir}}} and {{{mkshadowdir}}} are two (If you don't have either, the
    49 source distribution includes sources for the X11
    50 {{{lndir}}} — check out {{{utils/lndir}}}). See
    51 [[ref(The story so far)]]
    52 for a typical invocation.
    53 
    54 The build tree does not need to be anywhere near the source tree in
    55 the file system.  Indeed, one advantage of separating the build tree
    56 from the source is that the build tree can be placed in a
    57 non-backed-up partition, saving your systems support people from
    58 backing up untold megabytes of easily-regenerated, and
    59 rapidly-changing, gubbins.  The golden rule is that (with a single
    60 exception — [[ref(Getting the build you want)]]) ''absolutely
    61 everything in the build tree is either a symbolic link to the source
    62 tree, or else is mechanically generated''.  It should be perfectly OK
    63 for your build tree to vanish overnight; an hour or two compiling and
    64 you're on the road again.
     10This part of the guide is about ''using'' the build system, that is,
     11how to interact with the build system when you're developing some part
     12of GHC, its libraries or tools.
     13
     14== Source trees and build trees ==
     15
     16Sometimes we want to separate the build tree from the source tree.
     17There are a few advantages to doing this:
     18
     19 * You can make multiple different builds from the same sources,
     20   perhaps for testing different build settings, or for building
     21   on different platforms.
     22
     23 * You might want to put the source tree on a remote, backed-up,
     24   filesystem, but keep your build tree on a local fast unbacked-up
     25   drive (this is a configuration we use regularly at GHC HQ).  It
     26   doesn't matter if you lose the build tree: it can easily be
     27   regenerated.
     28
     29 * It's easy to blow away a build tree and start again, without
     30   modifying your source tree.  `make maintainer-clean` is usually
     31   good for this too, but it can miss files that it doesn't know
     32   about, or files that are remnants from older versions of GHC.
     33
     34However, if you just want to build the software once on a single
     35platform, then your source tree can also be your build tree, and you
     36can skip the rest of this section.
     37
     38A ''build tree'' is just an exact copy of the source tree, except that
     39every file in it is a symbolic link to the appropriate file in the
     40source tree.  There are "standard" Unix utilities that make such
     41copies, so standard that they go by different names: {{{lndir}}} and
     42{{{mkshadowdir}}} are two (If you don't have either, the GHC source
     43tree contains sources for the X11 {{{lndir}}} check out
     44{{{utils/lndir}}}).
    6545
    6646You need to be a bit careful, though, that any new files you create
    67 (if you do any development work) are in the source tree, not a build
     47(if you do any development work) are in the source tree, not the build
    6848tree!
    6949
    70 Remember, that the source files in the build tree are ''symbolic
    71 links'' to the files in the source tree.  (The build tree soon
    72 accumulates lots of built files like {{{Foo.o}}}, as well.)  You can
    73 ''delete'' a source file from the build tree without affecting the
    74 source tree (though it's an odd thing to do).  On the other hand, if
    75 you ''edit'' a source file from the build tree, you'll edit the
    76 source-tree file directly.  (You can set up Emacs so that if you edit
    77 a source file from the build tree, Emacs will silently create an
    78 edited copy of the source file in the build tree, leaving the source
    79 file unchanged; but the danger is that you think you've edited the
    80 source file whereas actually all you've done is edit the build-tree
    81 copy.  More commonly you do want to edit the source file.)
    82 
    83 
    84 Like the source tree, the top level of your build tree must be (a
    85 linked copy of) the root directory of the GHC source tree.  Inside
    86 Makefiles, the root of your build tree is called
    87 {{{$(TOP)}}}.  In
    88 the rest of this document path names are relative to {{{$(TOP)}}}
    89 unless otherwise stated.  For example, the file {{{mk/target.mk}}} is
    90 actually {{{$(TOP)/mk/target.mk}}}.
    91 
    92 == Getting the build you want ==
    93 
    94 When you build GHC you will be compiling code on a particular ''host
    95 platform'', to run on a particular ''target platform'' (usually the
    96 same as the host platform).  The difficulty is that there are minor
    97 differences between different platforms; minor, but enough that the
    98 code needs to be a bit different for each.  There are some big
    99 differences too: for a different architecture we need to build GHC
    100 with a different native-code generator.
    101 
    102 There are also knobs you can turn to control how the software is
    103 built.  For example, you might want to build GHC optimised (so that it
    104 runs fast) or unoptimised (so that you can compile it fast after
    105 you've modified it.  Or, you might want to compile it with debugging
    106 on (so that extra consistency-checking code gets included) or off.
    107 And so on.
    108 
    109 All of this stuff is called the ''configuration'' of your build.  You
    110 set the configuration using a three-step process.
    111 
    112 === Step 1: get ready for configuration ===
    113 
    114 NOTE: if you're starting from a source distribution, rather than darcs
    115 sources, you can skip this step.
    116 
    117 Change directory to {{{$(TOP)}}} and issue the command
     50== Steps for building GHC ==
     51
     52We gave a quick getting-started introduction to building GHC in
     53[wiki:Building/QuickStart], the following sections describe each step
     54in more detail.
     55
     56In the build system, the path to the top of your build tree is
     57referred to as `$(TOP)`, and we will use that convention in the
     58following sections.
     59
     60=== Generate configure scripts ===
     61
     62NOTE: if you're starting from a source distribution that you
     63downloaded from the GHC web site, rather than darcs sources, you can
     64skip this step.  Source distributions come with configure scripts
     65pre-generated for you.
     66
     67Change directory to `$(TOP)` and issue the command
    11868{{{
    11969$ sh boot
    12070}}}
    12171
    122 (with no arguments). This GNU program (recursively) converts
    123 {{{$(TOP)/configure.ac}}} and {{{$(TOP)/aclocal.m4}}} to a
    124 shell script called {{{$(TOP)/configure}}}.  If {{{boot}}}
    125 bleats that it can't write the file {{{configure}}}, then delete the
    126 latter and try again.  Note that you must use {{{sh boot}}}, and
    127 not the old {{{autoreconf}}} or {{{autoconf}}}!  If you erroneously
    128 use {{{autoreconf}}} then building the libraries will fail, and it
    129 you use {{{autoconf}}} you'll
    130 get a message like {{{No rule to make target 'mk/config.h.in'}}}.
    131 
    132 Some parts of the source tree, particularly libraries, have their own
    133 configure script.  {{{sh boot}}} takes care of that, too, so all
    134 you have to do is calling {{{sh boot}}} in the top-level directory
    135 {{{$(TOP)}}}.
    136 
    137 These steps are completely platform-independent; they just mean that
    138 the human-written files ({{{configure.ac}}} and {{{aclocal.m4}}}) can
    139 be short, although the resulting files (the {{{configure}}} shell
    140 scripts and the C header template {{{mk/config.h.in}}}) are long.
    141 
    142 === Step 2: system configuration. ===
    143 
    144 Run the newly-created {{{configure}}} script, thus:
     72(Note: the `$` is the prompt.  You don't type that bit.)  This runs
     73`autoreconf` on the various `configure.ac` scripts in the GHC build
     74tree, generating `configure` scripts and other miscellaneous files.
     75
     76You have to re-do this step if (and only if) you ever change one of
     77the files that `autoconf` uses to generate the `configure` scripts,
     78such as `configure.ac`, or `aclocal.m4`.
     79
     80=== Run the configure script ===
     81
     82Run the `configure` script, thus:
    14583{{{
    14684$ ./configure <args>
    14785}}}
    14886
    149 {{{configure}}}'s mission is to scurry round your computer working out
     87`configure`'s mission is to scurry round your computer working out
    15088what architecture it has, what operating system, whether it has the
    151 {{{vfork}}} system call, where {{{tar}}} is kept, whether {{{gcc}}} is
    152 available, where various obscure {{{#include}}} files are, whether
     89`vfork` system call, where `tar` is kept, whether `gcc` is
     90available, where various obscure `#include` files are, whether
    15391it's a leap year, and what the systems manager had for lunch.  It
    15492communicates these snippets of information in two ways:
    15593
    156  * It translates {{{mk/config.mk.in}}} to {{{mk/config.mk}}},
    157    substituting for things between "{{{@}}}" brackets.  So,
    158    "{{{@HaveGcc@}}}" will be replaced by "{{{YES}}}" or "{{{NO}}}"
    159    depending on what {{{configure}}} finds.  {{{mk/config.mk}}} is
    160    included by every Makefile (directly or indirectly), so the
    161    configuration information is thereby communicated to all Makefiles.
    162 
    163  * It translates {{{mk/config.h.in}}} to {{{mk/config.h}}}.  The
    164    latter is {{{#include}}}d by various C programs, which can
     94 * It translates various files such as `mk/config.mk.in` to
     95   `mk/config.mk`, substituting for things between "`@`"
     96   brackets.  So, "`@HaveGcc@`" will be replaced by "`YES`" or
     97   "`NO`" depending on what `configure` finds.
     98   `mk/config.mk` is included by every Makefile (directly or
     99   indirectly), so the configuration information is thereby
     100   communicated to all Makefiles.
     101
     102 * It translates `mk/config.h.in` to `mk/config.h`.  The
     103   latter is `#include`d by various C source files, which can
    165104   thereby make use of configuration information.
    166105
    167 {{{configure}}} takes some optional arguments.  Use {{{./configure --help}}}
     106`configure` takes some optional arguments.  Use `./configure --help`
    168107to get a list of the available arguments.  Here are some of
    169108the ones you might need:
    170109
    171  {{{--with-ghc=<path>}}}::
     110 `--build=<platform>`::
     111  Specifies the "build platform" (see [wiki:Building/BuildSystem#Platformsettings]).
     112  This is usually only necessary on Windows, where you should set it
     113  to `--build=i386-unknown-mingw32`.
     114
     115 `--host=<platform>`::
     116  Set the "host platform" (see [wiki:Building/BuildSystem#Platformsettings]).
     117  Usually not necessary.
     118
     119 `--target=<platform>`::
     120  Set the "target platform" (see [wiki:Building/BuildSystem#Platformsettings]).
     121  Usually not necessary.
     122
     123 `--with-ghc=<path>`::
    172124  Specifies the path to an installed GHC which you would like to use.
    173125  This compiler will be used for compiling GHC-specific code (eg. GHC
    174   itself).  This option ''cannot'' be specified using {{{build.mk}}}
    175   (see later), because {{{configure}}} needs to auto-detect the
     126  itself).  This option ''cannot'' be specified using `build.mk`
     127  (see later), because `configure` needs to auto-detect the
    176128  version of GHC you're using.  The default is to look for a compiler
    177   named {{{ghc}}} in your path.
    178              
    179  {{{--with-hc=<path>}}}::
    180   Specifies the path to any installed Haskell compiler.  This compiler
    181   will be used for compiling generic Haskell code.  The default is to
    182   use {{{ghc}}}. (NOTE: I'm not sure it actually works to specify a
    183   compiler other than GHC here; unless you really know what you're
    184   doing I suggest not using this option at all.)
    185              
    186  {{{--with-gcc=<path>}}}::
     129  named `ghc` in your path.
     130
     131 `--with-gcc=<path>`::
    187132  Specifies the path to the installed GCC. This compiler will be used
    188133  to compile all C files, ''except'' any generated by the installed
    189134  Haskell compiler, which will have its own idea of which C compiler
    190   (if any) to use.  The default is to use {{{gcc}}}.
    191 
    192 === Step 3: build configuration. ===
     135  (if any) to use.  The default is to use `gcc`.  On Windows, this
     136  should be set to the gcc that comes with MinGW, which by default
     137  is `c:/mingw/bin/gcc`.
     138
     139 `--with-ld=<path>`::
     140  Specifies which `ld` program to use.  Normally only necessary on
     141  Windows, where you should set it to the `ld` that comes with MinGW,
     142  which is usually `c:/mingw/bin/ld`.
     143
     144 `--prefix`::
     145  Tells the build system where you would like GHC to be eventually
     146  installed.  You don't have to install GHC in order to use it: it is
     147  entirely possible to work on GHC and test it without ever modifying
     148  anything outside the build tree.  However, if you do with to install
     149  GHC, then the `--prefix` directory is the root of the install tree.
     150  Typciallly on Unix systems the default for `--prefix` is
     151  `/usr/local`.  See also [wiki:Building/Installing Installing] for more details.
     152
     153=== Build configuration ===
    193154
    194155Next, you say how this build of GHC is to differ from the standard
    195 defaults by creating a new file {{{mk/build.mk}}} ''in the build
    196 tree''.  This file is the one and only file you edit in the build
    197 tree, precisely because it says how this build differs from the
    198 source.  (Just in case your build tree does die, you might want to
    199 keep a private directory of {{{build.mk}}} files, and use a symbolic
    200 link in each build tree to point to the appropriate one.)  So
    201 {{{mk/build.mk}}} never exists in the source tree — you create one
    202 in each build tree from the template.  We'll discuss what to put in it
    203 shortly.
    204 
    205 And that's it for configuration. Simple, eh?
    206 
    207 What do you put in your build-specific configuration file
    208 {{{mk/build.mk}}}?  ''For almost all purposes all you will do is put
    209 make variable definitions that override those in''
    210 {{{mk/config.mk.in}}}.  The whole point of
    211 {{{mk/config.mk.in}}} — and its derived counterpart
    212 {{{mk/config.mk}}} — is to define the build configuration. It is
    213 heavily commented, as you will see if you look at it.  So generally,
    214 what you do is look at {{{mk/config.mk.in}}}, and add definitions in
    215 {{{mk/build.mk}}} that override any of the {{{config.mk}}} definitions
    216 that you want to change.  (The override occurs because the main
    217 boilerplate file, {{{mk/boilerplate.mk}}}, includes {{{build.mk}}}
    218 after {{{config.mk}}}.)
    219 
    220 For your convenience, there's a file called {{{build.mk.sample}}} that
    221 can serve as a starting point for your {{{build.mk}}}.
    222 
    223 For example, {{{config.mk.in}}} contains the definition:
    224 
    225  {{{GhcHcOpts = -Rghc-timing}}}::
    226   The accompanying comment explains that this is the list of flags
    227   passed to GHC when building GHC itself.  For doing development, it
    228   is wise to add {{{-DDEBUG}}}, to enable debugging code.  So you
    229   would add the following to {{{build.mk}}}:
    230      
    231  {{{GhcHcOpts += -DDEBUG}}}::
    232   GNU {{{make}}} allows existing definitions to have new text appended
    233   using the "{{{+=}}}" operator, which is quite a convenient feature.
     156defaults by creating a new file `mk/build.mk` ''in the build tree''.
     157This file is the one and only file you edit in the build tree,
     158precisely because it says how this build differs from the source.
     159(Just in case your build tree does die, you might want to keep a
     160private directory of `build.mk` files, and use a symbolic link in each
     161build tree to point to the appropriate one.)
     162
     163`mk/build.mk` is purely for overriding settings that are found in
     164`mk/config.mk`.  You should never edit `mk/config.mk` directly, since
     165it is automatically generated from `mk/config.mk.in` by `configure`.
     166Neither should you edit `mk/config.mk.in`; just provide your settings
     167in `mk/build.mk`.
     168
     169We provide a sample file, `mk/build.mk.sample`, which you can copy to
     170`mk/build.mk` and edit.  It provides a number of pre-defined
     171configurations, such as a "fast build" or a "development build".  Take
     172a look in the file itself for details.
     173
     174To understand more about what you can put in `mk/build.mk`, read on.
     175
     176==== Common build.mk options ====
     177
     178The following are some common variables that you might want to set in
     179your `mk/build.mk`.  For other variables that you can override,
     180take a look in [[GhcFile(mk/config.mk.in)]].
     181
     182 `SRC_HC_OPTS`::
     183  (default: `-H32m -O`)[[br]]
     184   Options passed to GHC for all Haskell compilations.
     185
     186 `GhcHcOpts`::
     187  (default: `-Rghc-timing`)[[br]]
     188  Options added when compiling GHC (all
     189  [wiki:Building/Architecture#Idiom:stages stages])
     190
     191 `GhcStage1HcOpts`::
     192  (default: ''empty'')[[br]]
     193  Options added when compiling the stage 1 GHC.
     194
     195 `GhcStage2HcOpts`::
     196  (default: `-O2`)[[br]]
     197  Options added when compiling the stage 2 GHC.
     198
     199 `GhcStage3HcOpts`::
     200  (default: `-O2`)[[br]]
     201  Options added when compiling the stage 3 GHC.
     202
     203 `GhcLibHcOpts`::
     204  (default: `-O2 -XGenerics`)[[br]]
     205  Options added when compiling the libraries.
     206
     207 `GhcProfiled`::
     208  (default: `NO`)[[br]]
     209  Set to `YES` to enable profiling for GHC itself (stage 2).
     210
     211 `GhcDebugged`::
     212  (default: `NO`)[[br]]
     213  Set to `YES` to pass `-debug` when building GHC (stage 2).
     214
     215 `GhcLibWays`::
     216  (default: `p`)[[br]]
     217  Ways in which to build the libraries.  Must contain
     218  at least `v` ([wiki:Building/Architecture#Idiom:thevanillaway the
     219  vanilla way]).  Also contains `p` by default (profiling).  For other
     220  ways, see `mk/config.mk.in`.
     221
     222 `SplitObjs`::
     223  (default: `YES` if supported, `NO` otherwise)[[br]]
     224  When set to `YES`, static library object files are split into smaller
     225  pieces.  This means that less of the library code needs to be linked
     226  into the final application, which makes smaller binaries.  It takes
     227  longer to build libraries this way, though.
     228 
     229
     230==== How to make GHC build quickly ====
     231
     232The GHC build tree is set up so that, by default, it builds a compiler
     233ready for installing and using.  That means full optimisation, and the
     234build can take a ''long'' time.  If you unpack your source tree and
     235right away say {{{./configure; make}}}, expect to have to wait a while.
     236For hacking, you want the build to be quick - quick to build in the
     237first place, and quick to rebuild after making changes.  Tuning your
     238build setup can make the difference between several hours to build
     239GHC, and less than an hour. 
     240
     241Here are the `build.mk` settings that we use to build fast:
     242
     243{{{
     244# My build settings for hacking on stage 2
     245SRC_HC_OPTS     = -H32m -O -fasm -Rghc-timing
     246GhcStage1HcOpts = -O -fasm
     247GhcStage2HcOpts = -O0 -DDEBUG -Wall
     248GhcLibHcOpts    = -O -fasm -XGenerics
     249GhcLibWays      =
     250SplitObjs       = NO
     251GhcBootLibs     = YES
     252}}}
     253
     254What do these options do?
     255
     256 {{{SRC_HC_OPTS = -H32m -O -fasm -Rghc-timing}}}::
     257  These options are added to the command line for all Haskell
     258  compilations.  We turn on `-fasm`, because that halves compilation
     259  time at the expense of a few percent performance. `-Rghc-timing`
     260  prints out a line of timing info about each compilation.  It's handy
     261  to keep an eye on.  `-Wall` turns on all the warnings; GHC is
     262  meant to be warning-clean with `-Wall`.
     263
     264 {{{GhcStage1HcOpts = -O -fasm}}}::
     265  Build stage 1 optimised: we're going to be rebuilding stage 2 a lot,
     266  so we want the compiler that does the building to be fast.
     267
     268 {{{GhcStage2HcOpts = -O0 -DDEBUG -Wall}}}::
     269  We turn off optimisation here, assuming you'll be modifying and
     270  testing stage 2.  With optimisation off, rebuilding GHC after
     271  modifying it will be ''much'' quicker, not only because the
     272  individual compilations will be quicker, but also there will be
     273  fewer dependencies between modules, so much less stuff is recompiled
     274  after each modification.
    234275  [[br]][[br]]
    235   Haskell compilations by default have {{{-O}}} turned on, by virtue
    236   of this setting from {{{config.mk}}}:
    237 
    238  {{{SRC_HC_OPTS += -H16m -O}}}::
    239   {{{SRC_HC_OPTS}}} means "options for HC from the source tree", where
    240   HC stands for Haskell Compiler.  {{{SRC_HC_OPTS}}} are added to
    241   every Haskell compilation.  To turn off optimisation, you could add
    242   this to {{{build.mk}}}:
    243 
    244  {{{SRC_HC_OPTS = -H16m -O0}}}::
    245   Or you could just add {{{-O0}}} to {{{GhcHcOpts}}} to turn off
    246   optimisation for the compiler.  See [wiki:Building/Hacking]
    247   for some more suggestions.
    248   [[br]][[br]]
    249   When reading {{{config.mk.in}}}, remember that anything between
    250   "@...@" signs is going to be substituted by {{{configure}}} later.
    251   You ''can'' override the resulting definition if you want, but you
    252   need to be a bit surer what you are doing.  For example, there's a
    253   line that says:
    254 
    255  {{{TAR = @TarCmd@}}}::
    256   This defines the Make variables {{{TAR}}} to the pathname for a
    257   {{{tar}}} that {{{configure}}} finds somewhere.  If you have your
    258   own pet {{{tar}}} you want to use instead, that's fine. Just add
    259   this line to {{{mk/build.mk}}}:
    260 
    261  {{{TAR = mytar}}}::
    262   You do not ''have'' to have a {{{mk/build.mk}}} file at all; if you
    263   don't, you'll get all the default settings from
    264   {{{mk/config.mk.in}}}.
    265 
    266   You can also use {{{build.mk}}} to override anything that
    267   {{{configure}}} got wrong.  One place where this happens often is
    268   with the definition of {{{FPTOOLS_TOP_ABS}}}: this variable is supposed
    269   to be the canonical path to the top of your source tree, but if your
    270   system uses an automounter then the correct directory is hard to
    271   find automatically.  If you find that {{{configure}}} has got it
    272   wrong, just put the correct definition in {{{build.mk}}}.
    273 
    274 == The story so far ==
    275 
    276 Let's summarise the steps you need to carry to get yourself a
    277 fully-configured build tree from scratch.
    278 
    279  * Get your source tree from somewhere (darcs repository or source
    280    distribution).  Say you call the root directory {{{myghc}}} (it
    281    does not have to be called {{{ghc}}}).
    282 
    283  * (Optional) Use {{{lndir}}} or {{{mkshadowdir}}} to create a build
    284    tree.
    285 {{{
    286 $ cd myghc
    287 $ mkshadowdir . /scratch/joe-bloggs/myghc-x86
    288 }}}
    289   (N.B. {{{mkshadowdir}}}'s first argument is taken relative to its
    290   second.) You probably want to give the build tree a name that
    291   suggests its main defining characteristic (in your mind at least),
    292   in case you later add others.
    293 
    294  * Change directory to the build tree.  Everything is going to happen
    295    there now.
    296 {{{
    297 $ cd /scratch/joe-bloggs/myghc-x86
    298 }}}
    299 
    300  * Prepare for system configuration:
    301 {{{
    302 $ sh boot
    303 }}}
    304   (You can skip this step if you are starting from a source
    305   distribution, and you already have {{{configure}}} and
    306   {{{mk/config.h.in}}}.)
    307 
    308  * Do system configuration:
    309 {{{
    310 $ ./configure
    311 }}}
    312   Don't forget to check whether you need to add any arguments to
    313   {{{configure}}}; for example, a common requirement is to specify
    314   which GHC to use with
    315   {{{--with-ghc=<path>}}}.
    316 
    317  * Create the file {{{mk/build.mk}}}, adding definitions for your
    318    desired configuration options.
    319 
    320 You can make subsequent changes to {{{mk/build.mk}}} as often as you
    321 like.  You do not have to run any further configuration programs to
    322 make these changes take effect. In theory you should, however, say
    323 {{{make clean; make}}}, because configuration option changes could
    324 affect anything — but in practice you are likely to know what's
    325 affected.
    326 
    327 == Making things ==
     276  Also we turn on {{{-DDEBUG}}}, because that enables assertions and
     277  debugging code in the compiler itself.  Turning on DEBUG makes
     278  the compiler about 30% slower.
     279
     280 {{{GhcLibHcOpts = -O -fasm -XGenerics}}}::
     281  You almost certainly want optimisation ''on'' when building
     282  libraries, otherwise the code you build with this compiler
     283  goes really slowly.  {{{-fgenerics}}} add generics support to the
     284  libraries - you can turn this off if you like (it'll make the
     285  libraries a bit smaller), but you won't be able to use Generics in
     286  the code you build against these libraries.
     287
     288 {{{GhcLibWays =}}}::
     289  Normally the profiled libraries are built.  Setting {{{GhcLibWays}}} to
     290  empty disables this, so you only build the normal libs.
     291
     292 {{{SplitObjs = NO}}}::
     293  Object splitting causes each module to be split into smaller
     294  pieces in the final library, to reduce executable sizes when
     295  linking against the library.  It can be quite time and
     296  memory-consuming, so turn it off when you're hacking.
     297
     298 {{{GhcBootLibs = YES}}}::
     299  If you're just interested in working on GHC, then you probably don't want
     300  to build the "extralibs" libraries that we normally ship with GHC. 
     301  So when [wiki:Building/GettingTheSources getting the sources],
     302  run `darcs-all` without the `--extra` option.  Alternatively, even if you have
     303  the libraries in your tree, you can stop them being built by setting
     304  `GhcBootLibs` in your `build.mk`.
     305
     306=== Making things ===
    328307
    329308At this point you have made yourself a fully-configured build tree, so
     
    331310
    332311The first thing you need to know is that ''you must use GNU
    333 {{{make}}}''.  On some systems (eg. FreeBSD) this is called
    334 {{{gmake}}}, whereas on others it is the standard {{{make}}} command.
    335 In this document we will always refer to it as {{{make}}}; please
    336 substitute with {{{gmake}}} if your system requires it.  If you use a
    337 the wrong {{{make}}} you will get all sorts of error messages (but no
    338 damage) because the GHC {{{Makefiles}}} use GNU {{{make}}}'s
     312`make`''.  On some systems (eg. FreeBSD) this is called
     313`gmake`, whereas on others it is the standard `make` command.
     314In this document we will always refer to it as `make`; please
     315substitute with `gmake` if your system requires it.  If you use a
     316the wrong `make` you will get all sorts of error messages (but no
     317damage) because the GHC `Makefiles` use GNU `make`'s
    339318facilities extensively.
    340319
    341 To just build the whole thing, {{{cd}}} to the top of your build tree
    342 and type {{{make}}}.  This will prepare the tree and build the various
    343 parts in the correct order, resulting in a complete build of GHC that
    344 can even be used directly from the tree, without being installed
    345 first.
    346 
    347 == Bootstrapping GHC ==
    348 
    349 GHC requires a 2-stage bootstrap in order to provide full
    350 functionality, including GHCi.  By a 2-stage bootstrap, we mean that
    351 the compiler is built once using the installed GHC, and then again
    352 using the compiler built in the first stage.  You can also build a
    353 stage 3 compiler, but this normally isn't necessary except to verify
    354 that the stage 2 compiler is working properly.
    355 
    356 Note that when doing a bootstrap, the stage 1 compiler must be built,
    357 followed by the runtime system and libraries, and then the stage 2
    358 compiler.  The correct ordering is implemented by the top-level
    359 {{{Makefile}}}, so if you want everything to work automatically it's
    360 best to start {{{make}}} from the top of the tree.  The top-level
    361 {{{Makefile}}} is set up to do a 2-stage bootstrap by default (when
    362 you say {{{make}}}).  Some other targets it supports are:
    363 
    364  {{{stage1}}}::
    365   Build everything as normal, including the stage 1 compiler.
    366 
    367  {{{stage2}}}::
    368   Build the stage 2 compiler only.
    369 
    370  {{{stage3}}}::
    371   Build the stage 3 compiler only.
    372 
    373  {{{bootstrap}}}, {{{bootstrap2}}}::
    374   Build stage 1 followed by stage 2.
    375 
    376  {{{bootstrap3}}}::
    377   Build stages 1, 2 and 3.
    378 
    379  {{{install}}}::
    380   Install everything, including the compiler built in stage 2.  To
    381   override the stage, say {{{make install stage=n}}} where {{{n}}} is
    382   the stage to install.
    383 
    384  {{{binary-dist}}}::
    385   make a binary distribution.  This is the target we
    386   use to build the binary distributions of GHC.
    387 
    388  {{{dist}}}::
    389   make a source distribution.  Note that this target
    390   does {{{make distclean}}} as part of its work;
    391   don't use it if you want to keep what you've built.
    392 
    393 The top-level {{{Makefile}}} also arranges
    394 to do the appropriate {{{make boot}}} steps (see
    395 below) before actually building anything.
    396 
    397 The {{{stage1}}}, {{{stage2}}} and {{{stage3}}} targets also work in
    398 the {{{compiler}}} directory, but don't forget that each stage
    399 requires its own {{{make boot}}} step: for example, you must do
     320To just build the whole thing, `cd` to the top of your build tree and
     321type `make`.  This will prepare the tree and build the various parts
     322in the correct order, resulting in a complete build of GHC that can
     323even be used directly from the tree (as `inplace/bin/ghc-stage2`),
     324without being installed first.
     325
     326=== What to do after `make` has finished ===
     327
     328If `make` completes successfully, then it should have created
     329`inplace/bin/ghc-stage2`, which is a GHC binary you can run directly.
     330It supports all the usual features, including GHCi if you pass the
     331`--interactive` flag. 
     332
     333In fact, the `inplace` directory looks a lot like an installed copy of
     334GHC (see [wiki:Building/Installing]): there is a `bin`
     335subdirectory containing various programs that can be run, including
     336`ghc-pkg`, Haddock and `hsc2hs`.
     337
     338You can now run the testsuite, see [wiki:Building/RunningTests].
     339 
     340You can now install GHC, by typing `make install`.
     341
     342== What to do if you get a build failure ==
     343
     344GHC is a complex system, with many platform-dependent components.  We
     345try our best to make sure it builds out of the box as often as
     346possible, but build failures are not uncommon.  If you get some kind
     347of failure, don't panic.
     348
     349The chances are, someone else already encountered the same problem as
     350you and has reported it on a mailing list or as a ticket.  Search for
     351the error message using your favourite search engine, and if that
     352doesn't turn anything up then search directly on this Trac.  Finally,
     353if you don't find a solution:
     354
     355 * Ask someone: [wiki:MailingListsAndIRC]
     356 * or just [wiki:ReportABug]
     357
     358== Components of the build ==
     359
     360Here is a high level view of what happens when you build GHC:
     361
     362 * First, we build a few packages that GHC itself depends on, such as
     363   `Cabal`, and `filepath`, using your installed GHC.  These packages
     364   are under `libraries`, and each is built in a subdirectory
     365   `dist-boot`; for example the bootstrap build of Cabal will be in
     366   `libraries/Cabal/dist-boot`.
     367
     368 * Then we build GHC against these packages, still using your
     369   installed GHC.  This is called the "stage 1" compiler (see
     370   [wiki:Building/Architecture#Idiom:stages stages]).  You can run the
     371   stage 1 compiler by invoking `inplace/bin/ghc-stage1`.  The stage 1
     372   build of GHC happens in `compiler/stage1`.
     373
     374 * The stage 1 compiler is now used to build all the packages in the
     375   `libraries` subdirectory, and the runtime system in `rts`.
     376
     377 * Finally, the stage 1 compiler is used to build GHC itself again,
     378   this time against the libraries we just built.  This GHC is called
     379   stage 2, and can be invokes as `inplace/bin/ghc-stage2`.
     380
     381There's an optional final stage, using the stage 2 compiler to build a
     382stage 3 compiler, but this isn't strictly necessary, and is only used
     383to check that the stage 2 compiler is working properly.
     384
     385
     386== Developing in a GHC build tree ==
     387
     388This section describes how to make changes to the sources of GHC, or
     389its libraries and tools, and use the build system to build and test
     390those changes.
     391
     392=== Bringing the whole tree up to date ===
     393
     394The easy way to bring the tree up to date after making some changes is
     395simply to go to the top of the tree and say `make`.  The build system
     396has enough dependencies encoded into it that this should rebuild
     397everything that needs to be rebuilt, in the correct order.  The
     398downside is that it will build everything right though to the stage2
     399compiler and including all the documentation, which might be overkill
     400if all you wanted to do was to test a small change to GHC.
     401
     402=== Just building a single component ===
     403
     404Suppose you want to make a small change to GHC itself and test it.
     405Assuming that your tree is already up to date, the best way to do this
     406is as follows:
     407
    400408{{{
    401 $ make boot stage=2
     409$ cd ghc
     410$ make stage2
    402411}}}
    403 before {{{make stage2}}} in {{{compiler}}}.
     412
     413This will bring the stage2 compiler up to date only.  In particular,
     414it will ignore the fact that by modifying GHC you have thereby made
     415the stage 1 compiler out of date, and hence possibly all the libraries
     416and the whole of stage 2 are now also out of date.  If you did `make`
     417from the top-level, all of these dependencies would be taken into
     418account, and a lot of rebuilding would probably ensue.
     419
     420Each subdirectory of the source tree has a
     421[wiki:BuildSystem/Architecture#Idiom:stubmakefiles stub makefile],
     422most of which follow this pattern:
     423
     424{{{
     425dir = libraries/base
     426TOP = ../..
     427include $(TOP)/mk/sub-makefile.mk
     428}}}
     429
     430the main purpose of the stub makefile is to invoke `make` at the
     431top-level of the tree: GHC's build system is
     432[wiki:Building/Architecture#Idiom:non-recursivemake non-recursive], so
     433in effect there is really just one `Makefile`, at the top level.
     434However, the stub makefile has another purpose: it passes a target to
     435the top-level `Makefile`, telling it to build just the components of
     436the system in this directory.  For example, when you say `make` in the
     437`rts` directory, this is actually equivalent to
     438
     439{{{
     440$ (cd ..; make all_rts)
     441}}}
     442
     443where `make all_rts` makes every target in the `rts` subdirectory.
     444Equivalently, `make all_libraries/base` at the top level would build
     445everything in the `libraries/base` subdirectory.  To understand how
     446the `all` targets are defined, see
     447[wiki:Building/Architecture#Idiom:standardtargetsallcleanetc. standard targets].
     448
     449You can also clean a single component of the tree, just by saying
     450`make clean` in a subdirectory.  Again this is equivalent to issuing a
     451command at the top of the tree of the form `make clean_libraries/base`.
     452
     453The `stage2` target in the `ghc` directory does something else: it
     454also disables the dependencies that would otherwise cause the stage 1
     455compiler and the libraries to be rebuilt.  It does this by simply
     456setting `ASSUME_STAGE1=YES` and `ASSUME_LIBRARIES=YES` when invoking
     457the top-level `make`.
    404458
    405459== Standard Targets ==
    406460
    407 In any directory you should be able to make the following:
    408 
    409  {{{boot}}}::
    410   does the one-off preparation required to get ready for the real
    411   work, e.g. building the module dependency graph.
    412   [[br]][[br]]
    413   Invoking the {{{boot}}} target explicitly is not normally necessary.
    414   From the top-level directory, invoking {{{make}}} causes {{{make boot}}}
    415   to be invoked in various subdirectories, in the right
    416   order.  Unless you really know what you are doing, it is best to
    417   always say {{{make}}} from the top level first.
    418   [[br]][[br]]
    419   If you're working in a subdirectory somewhere and need to update the
    420   dependencies, {{{make boot}}} is a good way to do it.
    421 
    422  {{{all}}}::
    423   makes all the final target(s) for this Makefile.  Depending on which
    424   directory you are in a "final target" may be an executable program,
    425   a library archive, a shell script, or a Postscript file.  Typing
    426   {{{make}}} alone is generally the same as typing {{{make all}}}.
    427 
    428  {{{install}}}::
    429   installs the things built by {{{all}}} (except for the
    430   documentation).  Where does it install them?  That is specified by
    431   {{{mk/config.mk.in}}}; you can override it in {{{mk/build.mk}}}, or
    432   by running {{{configure}}} with command-line arguments like
    433   {{{--bindir=/home/simonpj/bin}}}; see {{{./configure --help}}} for
    434   the full details.
    435 
    436  {{{install-docs}}}::
    437   installs the documentation. Otherwise behaves just like
    438   {{{install}}}.
    439 
    440  {{{clean}}}::
     461The main targets understood by the top-level `Makefile` are as
     462follows:
     463
     464 `all`::
     465  (default target, can be omitted).  Builds everything that needs to
     466  be built for a GHC installation, including the stage 2 GHC, all
     467  libraries and documentation.  After `make`, `make install` will not
     468  need to do any further rebuilding.
     469
     470 `install`::
     471  installs the things built by `all`.  Where does it install them?  In
     472  the places you specified when running `configure`, principally set
     473  by the `--prefix` flag; see [wiki:Building/Installing].
     474
     475 `clean`::
    441476  Delete all files from the current directory that are normally
    442   created by building the program.  Don't delete the files that record
    443   the configuration, or files generated by {{{make boot}}}.  Also
    444   preserve files that could be made by building, but normally aren't
    445   because the distribution comes with them.
    446 
    447  {{{distclean}}}::
    448   Delete all files from the current directory that are created by
    449   configuring or building the program. If you have unpacked the source
    450   and built the program without creating any other files, {{{make distclean}}}
    451   should leave only the files that were in the distribution.
    452 
    453  {{{mostlyclean}}}::
    454   Like {{{clean}}}, but may refrain from deleting a few files that
    455   people normally don't want to recompile.
    456 
    457  {{{maintainer-clean}}}::
    458   Delete everything from the current directory that can be
    459   reconstructed with this Makefile.  This typically includes
    460   everything deleted by {{{distclean}}}, plus more: C source files
    461   produced by Bison, tags tables, Info files, and so on.
    462   [[br]][[br]]   
    463   One exception, however: {{{make maintainer-clean}}} should not
    464   delete {{{configure}}} even if {{{configure}}} can be remade using a
    465   rule in the {{{Makefile}}}. More generally, {{{make maintainer-clean}}}
    466   should not delete anything that needs to exist in order to run
    467   {{{configure}}} and then begin to build the program.
    468   [[br]][[br]]   
    469   After a {{{maintainer-clean}}}, a {{{configure}}} will be necessary
    470   before building again.
    471 
    472 All of these standard targets automatically recurse into
    473 sub-directories.  Certain other standard targets do not:
    474 
    475  {{{depend}}}::
    476   make a {{{.depend}}} file in each directory that needs it. This
    477   {{{.depend}}} file contains mechanically-generated dependency
    478   information; for example, suppose a directory contains a Haskell
    479   source module {{{Foo.lhs}}} which imports another module {{{Baz}}}.
    480   Then the generated {{{.depend}}} file will contain the dependency:
    481 {{{
    482 Foo.o : Baz.hi
    483 }}}
    484   which says that the object file {{{Foo.o}}} depends on the interface
    485   file {{{Baz.hi}}} generated by compiling module {{{Baz}}}.  The
    486   {{{.depend}}} file is automatically included by every Makefile.
    487   Now that we are using Cabal for most of the building, most directories
    488   don't support the `depend` target any more. Use `boot` instead.
    489 
    490 Some {{{Makefile}}}s have targets other
    491 than these.  You can discover them by looking in the
    492 {{{Makefile}}} itself.
    493 
    494 == Using GHC from the build tree ==
    495 
    496 If you want to build GHC and just use it direct from the build tree
    497 without doing {{{make install}}} first, you can run the in-place
    498 driver script.  To run the stage 1 compiler, use
    499 `ghc/stage1-inplace/ghc`, stage 2 is
    500 `ghc/stage2-inplace/ghc`, and so on.
    501 
    502 Utils like `ghc-pkg` can be found under
    503 `utils/ghc-pkg/install-inplace/bin/ghc-pkg` etc.
    504 
    505 == Fast Making ==
    506 
    507 Sometimes the dependencies get in the way: if you've made a small
    508 change to one file, and you're absolutely sure that it won't affect
    509 anything else, but you know that {{{make}}} is going to rebuild
    510 everything anyway, the following hack may be useful:
    511 
    512 {{{
    513 $ make FAST=YES
    514 }}}
    515 
    516 This tells the make system to ignore dependencies and just build what
    517 you tell it to.  In other words, it's equivalent to temporarily
    518 removing the {{{.depend}}} file in the current directory (where
    519 {{{mkdependHS}}} and friends store their dependency information).
    520 
    521 A bit of history: GHC used to come with a {{{fastmake}}} script that
    522 did the above job, but GNU make provides the features we need to do it
    523 without resorting to a script.  Also, we've found that fastmaking is
    524 less useful since the advent of GHC's recompilation checker (see the
    525 User's Guide section on "Separate Compilation").
     477  created by `make`.  Don't delete the files that record the
     478  configuration.
     479
     480 `distclean`::
     481  The same as `clean`, but also removes files created by `configure`.
     482  `distclean` preserves files that would normally go in a source
     483  distrubtion.  Note that after `make distclean` you will have to run
     484  `./configure` before building again.
     485
     486 `maintainer-clean`::
     487  The same as `distclean`, but also removes files generated by `sh boot`.
     488  The result of `make maintainer-clean` should be a completely clean
     489  source tree, containing only revision-controlled files.
     490
     491 `binary-dist`::
     492  Builds a binary distribution.  A binary distribution is a copy of
     493  the build tree with the source files removed, such that it can be
     494  installed on a different system simply by issuing `./configure; make
     495  install`.
     496
     497 `sdist`::
     498  Builds a source distribution.  A source distribution is a source
     499  tree for GHC, complete with some extra files included for
     500  convenience so that the user doesn't need to install `autoconf`,
     501  `Happy`, or `Alex` in order to build it.  Hence `make sdist` only
     502  works in a completely built tree.