Changes between Version 14 and Version 15 of Building/Using


Ignore:
Timestamp:
Mar 30, 2009 1:30:22 PM (6 years ago)
Author:
simonmar
Comment:

Start documenting the new build system

Legend:

Unmodified
Added
Removed
Modified
  • Building/Using

    v14 v15  
    11 
    22[[PageOutline]] 
     3 
     4'''NOTE''': this page now describes the new GHC build system, due to 
     5be added to GHC in April 2009.  For documentation of the "old" build 
     6system, see [wiki:Building/Old/Using]. 
    37 
    48= Using the Build System = 
    59     
    6 This rest of this guide is designed to help you even if you aren't 
    7 really interested in Makefiles and systems configurations, but you 
    8 need a mental model of the interlocking pieces so that they can make 
    9 them work, extend them consistently when adding new software, and lay 
    10 hands on them gently when they don't work. 
    11  
    12 == History == 
    13  
    14 First, a historical note.  The GHC build system used to be 
    15 called "fptools": a generic build system used to build multiple 
    16 projects (GHC, Happy, !GreenCard, H/Direct, etc.).  It had a 
    17 concept of the generic project-independent parts, and 
    18 project-specific parts that resided in a project 
    19 subdirectory. 
    20  
    21 Nowadays, most of these other projects are using 
    22 [http://www.haskell.org/cabal/ Cabal], or have faded away, and GHC is 
    23 the only regular user of the fptools build system.  We decided 
    24 therefore to simplify the situation for developers, and specialise the 
    25 build system for GHC.  This resulted in a simpler organisation of the 
    26 source tree and the build system, which hopefully makes the whole 
    27 thing easier to understand. 
    28  
    29 You might find old comments that refer to "projects" or "fptools" in 
    30 the documentation and/or source; please let us know if you do. 
    31  
    32 == Build trees == 
    33  
    34 If you just want to build the software once on a single platform, then 
    35 your source tree can also be your build tree, and you can skip the 
    36 rest of this section. 
    37  
    38 We often want to build multiple versions of our software for different 
    39 architectures, or with different options (e.g. profiling).  It's very 
    40 desirable to share a single copy of the source code among all these 
    41 builds. 
    42  
    43 So for every source tree we have zero or more ''build trees''.  Each 
    44 build tree is initially an exact copy of the source tree, except that 
    45 each file is a symbolic link to the source file, rather than being a 
    46 copy of the source file.  There are "standard" Unix utilities that 
    47 make such copies, so standard that they go by different names: 
    48 {{{lndir}}} and {{{mkshadowdir}}} are two (If you don't have either, the 
    49 source distribution includes sources for the X11 
    50 {{{lndir}}} — check out {{{utils/lndir}}}). See  
    51 [[ref(The story so far)]]  
    52 for a typical invocation. 
    53  
    54 The build tree does not need to be anywhere near the source tree in 
    55 the file system.  Indeed, one advantage of separating the build tree 
    56 from the source is that the build tree can be placed in a 
    57 non-backed-up partition, saving your systems support people from 
    58 backing up untold megabytes of easily-regenerated, and 
    59 rapidly-changing, gubbins.  The golden rule is that (with a single 
    60 exception — [[ref(Getting the build you want)]]) ''absolutely 
    61 everything in the build tree is either a symbolic link to the source 
    62 tree, or else is mechanically generated''.  It should be perfectly OK 
    63 for your build tree to vanish overnight; an hour or two compiling and 
    64 you're on the road again. 
     10This part of the guide is about ''using'' the build system, that is, 
     11how to interact with the build system when you're developing some part 
     12of GHC, its libraries or tools. 
     13 
     14== Source trees and build trees == 
     15 
     16Sometimes we want to separate the build tree from the source tree. 
     17There are a few advantages to doing this: 
     18 
     19 * You can make multiple different builds from the same sources, 
     20   perhaps for testing different build settings, or for building 
     21   on different platforms. 
     22 
     23 * You might want to put the source tree on a remote, backed-up, 
     24   filesystem, but keep your build tree on a local fast unbacked-up 
     25   drive (this is a configuration we use regularly at GHC HQ).  It 
     26   doesn't matter if you lose the build tree: it can easily be 
     27   regenerated. 
     28 
     29 * It's easy to blow away a build tree and start again, without 
     30   modifying your source tree.  `make maintainer-clean` is usually 
     31   good for this too, but it can miss files that it doesn't know 
     32   about, or files that are remnants from older versions of GHC. 
     33 
     34However, if you just want to build the software once on a single 
     35platform, then your source tree can also be your build tree, and you 
     36can skip the rest of this section. 
     37 
     38A ''build tree'' is just an exact copy of the source tree, except that 
     39every file in it is a symbolic link to the appropriate file in the 
     40source tree.  There are "standard" Unix utilities that make such 
     41copies, so standard that they go by different names: {{{lndir}}} and 
     42{{{mkshadowdir}}} are two (If you don't have either, the GHC source 
     43tree contains sources for the X11 {{{lndir}}} check out 
     44{{{utils/lndir}}}). 
    6545 
    6646You need to be a bit careful, though, that any new files you create 
    67 (if you do any development work) are in the source tree, not a build 
     47(if you do any development work) are in the source tree, not the build 
    6848tree! 
    6949 
    70 Remember, that the source files in the build tree are ''symbolic 
    71 links'' to the files in the source tree.  (The build tree soon 
    72 accumulates lots of built files like {{{Foo.o}}}, as well.)  You can 
    73 ''delete'' a source file from the build tree without affecting the 
    74 source tree (though it's an odd thing to do).  On the other hand, if 
    75 you ''edit'' a source file from the build tree, you'll edit the 
    76 source-tree file directly.  (You can set up Emacs so that if you edit 
    77 a source file from the build tree, Emacs will silently create an 
    78 edited copy of the source file in the build tree, leaving the source 
    79 file unchanged; but the danger is that you think you've edited the 
    80 source file whereas actually all you've done is edit the build-tree 
    81 copy.  More commonly you do want to edit the source file.) 
    82  
    83  
    84 Like the source tree, the top level of your build tree must be (a 
    85 linked copy of) the root directory of the GHC source tree.  Inside 
    86 Makefiles, the root of your build tree is called 
    87 {{{$(TOP)}}}.  In 
    88 the rest of this document path names are relative to {{{$(TOP)}}} 
    89 unless otherwise stated.  For example, the file {{{mk/target.mk}}} is 
    90 actually {{{$(TOP)/mk/target.mk}}}. 
    91  
    92 == Getting the build you want == 
    93  
    94 When you build GHC you will be compiling code on a particular ''host 
    95 platform'', to run on a particular ''target platform'' (usually the 
    96 same as the host platform).  The difficulty is that there are minor 
    97 differences between different platforms; minor, but enough that the 
    98 code needs to be a bit different for each.  There are some big 
    99 differences too: for a different architecture we need to build GHC 
    100 with a different native-code generator. 
    101  
    102 There are also knobs you can turn to control how the software is 
    103 built.  For example, you might want to build GHC optimised (so that it 
    104 runs fast) or unoptimised (so that you can compile it fast after 
    105 you've modified it.  Or, you might want to compile it with debugging 
    106 on (so that extra consistency-checking code gets included) or off. 
    107 And so on. 
    108  
    109 All of this stuff is called the ''configuration'' of your build.  You 
    110 set the configuration using a three-step process. 
    111  
    112 === Step 1: get ready for configuration === 
    113  
    114 NOTE: if you're starting from a source distribution, rather than darcs 
    115 sources, you can skip this step. 
    116  
    117 Change directory to {{{$(TOP)}}} and issue the command 
     50== Steps for building GHC == 
     51 
     52We gave a quick getting-started introduction to building GHC in 
     53[wiki:Building/QuickStart], the following sections describe each step 
     54in more detail. 
     55 
     56In the build system, the path to the top of your build tree is 
     57referred to as `$(TOP)`, and we will use that convention in the 
     58following sections. 
     59 
     60=== Generate configure scripts === 
     61 
     62NOTE: if you're starting from a source distribution that you 
     63downloaded from the GHC web site, rather than darcs sources, you can 
     64skip this step.  Source distributions come with configure scripts 
     65pre-generated for you. 
     66 
     67Change directory to `$(TOP)` and issue the command 
    11868{{{ 
    11969$ sh boot 
    12070}}} 
    12171 
    122 (with no arguments). This GNU program (recursively) converts 
    123 {{{$(TOP)/configure.ac}}} and {{{$(TOP)/aclocal.m4}}} to a 
    124 shell script called {{{$(TOP)/configure}}}.  If {{{boot}}} 
    125 bleats that it can't write the file {{{configure}}}, then delete the 
    126 latter and try again.  Note that you must use {{{sh boot}}}, and 
    127 not the old {{{autoreconf}}} or {{{autoconf}}}!  If you erroneously 
    128 use {{{autoreconf}}} then building the libraries will fail, and it 
    129 you use {{{autoconf}}} you'll 
    130 get a message like {{{No rule to make target 'mk/config.h.in'}}}. 
    131  
    132 Some parts of the source tree, particularly libraries, have their own 
    133 configure script.  {{{sh boot}}} takes care of that, too, so all 
    134 you have to do is calling {{{sh boot}}} in the top-level directory 
    135 {{{$(TOP)}}}. 
    136  
    137 These steps are completely platform-independent; they just mean that 
    138 the human-written files ({{{configure.ac}}} and {{{aclocal.m4}}}) can 
    139 be short, although the resulting files (the {{{configure}}} shell 
    140 scripts and the C header template {{{mk/config.h.in}}}) are long. 
    141  
    142 === Step 2: system configuration. === 
    143  
    144 Run the newly-created {{{configure}}} script, thus: 
     72(Note: the `$` is the prompt.  You don't type that bit.)  This runs 
     73`autoreconf` on the various `configure.ac` scripts in the GHC build 
     74tree, generating `configure` scripts and other miscellaneous files. 
     75 
     76You have to re-do this step if (and only if) you ever change one of 
     77the files that `autoconf` uses to generate the `configure` scripts, 
     78such as `configure.ac`, or `aclocal.m4`. 
     79 
     80=== Run the configure script === 
     81 
     82Run the `configure` script, thus: 
    14583{{{ 
    14684$ ./configure <args> 
    14785}}} 
    14886 
    149 {{{configure}}}'s mission is to scurry round your computer working out 
     87`configure`'s mission is to scurry round your computer working out 
    15088what architecture it has, what operating system, whether it has the 
    151 {{{vfork}}} system call, where {{{tar}}} is kept, whether {{{gcc}}} is 
    152 available, where various obscure {{{#include}}} files are, whether 
     89`vfork` system call, where `tar` is kept, whether `gcc` is 
     90available, where various obscure `#include` files are, whether 
    15391it's a leap year, and what the systems manager had for lunch.  It 
    15492communicates these snippets of information in two ways: 
    15593 
    156  * It translates {{{mk/config.mk.in}}} to {{{mk/config.mk}}}, 
    157    substituting for things between "{{{@}}}" brackets.  So, 
    158    "{{{@HaveGcc@}}}" will be replaced by "{{{YES}}}" or "{{{NO}}}" 
    159    depending on what {{{configure}}} finds.  {{{mk/config.mk}}} is 
    160    included by every Makefile (directly or indirectly), so the 
    161    configuration information is thereby communicated to all Makefiles. 
    162  
    163  * It translates {{{mk/config.h.in}}} to {{{mk/config.h}}}.  The 
    164    latter is {{{#include}}}d by various C programs, which can 
     94 * It translates various files such as `mk/config.mk.in` to 
     95   `mk/config.mk`, substituting for things between "`@`" 
     96   brackets.  So, "`@HaveGcc@`" will be replaced by "`YES`" or 
     97   "`NO`" depending on what `configure` finds. 
     98   `mk/config.mk` is included by every Makefile (directly or 
     99   indirectly), so the configuration information is thereby 
     100   communicated to all Makefiles. 
     101 
     102 * It translates `mk/config.h.in` to `mk/config.h`.  The 
     103   latter is `#include`d by various C source files, which can 
    165104   thereby make use of configuration information. 
    166105 
    167 {{{configure}}} takes some optional arguments.  Use {{{./configure --help}}} 
     106`configure` takes some optional arguments.  Use `./configure --help` 
    168107to get a list of the available arguments.  Here are some of 
    169108the ones you might need: 
    170109 
    171  {{{--with-ghc=<path>}}}:: 
     110 `--build=<platform>`:: 
     111  Specifies the "build platform" (see [wiki:Building/BuildSystem#Platformsettings]). 
     112  This is usually only necessary on Windows, where you should set it 
     113  to `--build=i386-unknown-mingw32`. 
     114 
     115 `--host=<platform>`:: 
     116  Set the "host platform" (see [wiki:Building/BuildSystem#Platformsettings]). 
     117  Usually not necessary. 
     118 
     119 `--target=<platform>`:: 
     120  Set the "target platform" (see [wiki:Building/BuildSystem#Platformsettings]). 
     121  Usually not necessary. 
     122 
     123 `--with-ghc=<path>`:: 
    172124  Specifies the path to an installed GHC which you would like to use. 
    173125  This compiler will be used for compiling GHC-specific code (eg. GHC 
    174   itself).  This option ''cannot'' be specified using {{{build.mk}}} 
    175   (see later), because {{{configure}}} needs to auto-detect the 
     126  itself).  This option ''cannot'' be specified using `build.mk` 
     127  (see later), because `configure` needs to auto-detect the 
    176128  version of GHC you're using.  The default is to look for a compiler 
    177   named {{{ghc}}} in your path. 
    178                
    179  {{{--with-hc=<path>}}}:: 
    180   Specifies the path to any installed Haskell compiler.  This compiler 
    181   will be used for compiling generic Haskell code.  The default is to 
    182   use {{{ghc}}}. (NOTE: I'm not sure it actually works to specify a 
    183   compiler other than GHC here; unless you really know what you're 
    184   doing I suggest not using this option at all.) 
    185                
    186  {{{--with-gcc=<path>}}}:: 
     129  named `ghc` in your path. 
     130 
     131 `--with-gcc=<path>`:: 
    187132  Specifies the path to the installed GCC. This compiler will be used 
    188133  to compile all C files, ''except'' any generated by the installed 
    189134  Haskell compiler, which will have its own idea of which C compiler 
    190   (if any) to use.  The default is to use {{{gcc}}}. 
    191  
    192 === Step 3: build configuration. === 
     135  (if any) to use.  The default is to use `gcc`.  On Windows, this 
     136  should be set to the gcc that comes with MinGW, which by default 
     137  is `c:/mingw/bin/gcc`. 
     138 
     139 `--with-ld=<path>`:: 
     140  Specifies which `ld` program to use.  Normally only necessary on 
     141  Windows, where you should set it to the `ld` that comes with MinGW, 
     142  which is usually `c:/mingw/bin/ld`. 
     143 
     144 `--prefix`:: 
     145  Tells the build system where you would like GHC to be eventually 
     146  installed.  You don't have to install GHC in order to use it: it is 
     147  entirely possible to work on GHC and test it without ever modifying 
     148  anything outside the build tree.  However, if you do with to install 
     149  GHC, then the `--prefix` directory is the root of the install tree. 
     150  Typciallly on Unix systems the default for `--prefix` is 
     151  `/usr/local`.  See also [wiki:Building/Installing Installing] for more details. 
     152 
     153=== Build configuration === 
    193154 
    194155Next, you say how this build of GHC is to differ from the standard 
    195 defaults by creating a new file {{{mk/build.mk}}} ''in the build 
    196 tree''.  This file is the one and only file you edit in the build 
    197 tree, precisely because it says how this build differs from the 
    198 source.  (Just in case your build tree does die, you might want to 
    199 keep a private directory of {{{build.mk}}} files, and use a symbolic 
    200 link in each build tree to point to the appropriate one.)  So 
    201 {{{mk/build.mk}}} never exists in the source tree — you create one 
    202 in each build tree from the template.  We'll discuss what to put in it 
    203 shortly. 
    204  
    205 And that's it for configuration. Simple, eh? 
    206  
    207 What do you put in your build-specific configuration file 
    208 {{{mk/build.mk}}}?  ''For almost all purposes all you will do is put 
    209 make variable definitions that override those in'' 
    210 {{{mk/config.mk.in}}}.  The whole point of 
    211 {{{mk/config.mk.in}}} — and its derived counterpart 
    212 {{{mk/config.mk}}} — is to define the build configuration. It is 
    213 heavily commented, as you will see if you look at it.  So generally, 
    214 what you do is look at {{{mk/config.mk.in}}}, and add definitions in 
    215 {{{mk/build.mk}}} that override any of the {{{config.mk}}} definitions 
    216 that you want to change.  (The override occurs because the main 
    217 boilerplate file, {{{mk/boilerplate.mk}}}, includes {{{build.mk}}} 
    218 after {{{config.mk}}}.) 
    219  
    220 For your convenience, there's a file called {{{build.mk.sample}}} that 
    221 can serve as a starting point for your {{{build.mk}}}. 
    222  
    223 For example, {{{config.mk.in}}} contains the definition: 
    224  
    225  {{{GhcHcOpts = -Rghc-timing}}}:: 
    226   The accompanying comment explains that this is the list of flags 
    227   passed to GHC when building GHC itself.  For doing development, it 
    228   is wise to add {{{-DDEBUG}}}, to enable debugging code.  So you 
    229   would add the following to {{{build.mk}}}: 
    230        
    231  {{{GhcHcOpts += -DDEBUG}}}:: 
    232   GNU {{{make}}} allows existing definitions to have new text appended 
    233   using the "{{{+=}}}" operator, which is quite a convenient feature. 
     156defaults by creating a new file `mk/build.mk` ''in the build tree''. 
     157This file is the one and only file you edit in the build tree, 
     158precisely because it says how this build differs from the source. 
     159(Just in case your build tree does die, you might want to keep a 
     160private directory of `build.mk` files, and use a symbolic link in each 
     161build tree to point to the appropriate one.) 
     162 
     163`mk/build.mk` is purely for overriding settings that are found in 
     164`mk/config.mk`.  You should never edit `mk/config.mk` directly, since 
     165it is automatically generated from `mk/config.mk.in` by `configure`. 
     166Neither should you edit `mk/config.mk.in`; just provide your settings 
     167in `mk/build.mk`. 
     168 
     169We provide a sample file, `mk/build.mk.sample`, which you can copy to 
     170`mk/build.mk` and edit.  It provides a number of pre-defined 
     171configurations, such as a "fast build" or a "development build".  Take 
     172a look in the file itself for details. 
     173 
     174To understand more about what you can put in `mk/build.mk`, read on. 
     175 
     176==== Common build.mk options ==== 
     177 
     178The following are some common variables that you might want to set in 
     179your `mk/build.mk`.  For other variables that you can override, 
     180take a look in [[GhcFile(mk/config.mk.in)]]. 
     181 
     182 `SRC_HC_OPTS`:: 
     183  (default: `-H32m -O`)[[br]] 
     184   Options passed to GHC for all Haskell compilations. 
     185 
     186 `GhcHcOpts`:: 
     187  (default: `-Rghc-timing`)[[br]] 
     188  Options added when compiling GHC (all 
     189  [wiki:Building/Architecture#Idiom:stages stages]) 
     190 
     191 `GhcStage1HcOpts`:: 
     192  (default: ''empty'')[[br]] 
     193  Options added when compiling the stage 1 GHC. 
     194 
     195 `GhcStage2HcOpts`:: 
     196  (default: `-O2`)[[br]] 
     197  Options added when compiling the stage 2 GHC. 
     198 
     199 `GhcStage3HcOpts`:: 
     200  (default: `-O2`)[[br]] 
     201  Options added when compiling the stage 3 GHC. 
     202 
     203 `GhcLibHcOpts`:: 
     204  (default: `-O2 -XGenerics`)[[br]] 
     205  Options added when compiling the libraries. 
     206 
     207 `GhcProfiled`:: 
     208  (default: `NO`)[[br]] 
     209  Set to `YES` to enable profiling for GHC itself (stage 2). 
     210 
     211 `GhcDebugged`:: 
     212  (default: `NO`)[[br]] 
     213  Set to `YES` to pass `-debug` when building GHC (stage 2). 
     214 
     215 `GhcLibWays`:: 
     216  (default: `p`)[[br]] 
     217  Ways in which to build the libraries.  Must contain 
     218  at least `v` ([wiki:Building/Architecture#Idiom:thevanillaway the 
     219  vanilla way]).  Also contains `p` by default (profiling).  For other 
     220  ways, see `mk/config.mk.in`. 
     221 
     222 `SplitObjs`:: 
     223  (default: `YES` if supported, `NO` otherwise)[[br]] 
     224  When set to `YES`, static library object files are split into smaller 
     225  pieces.  This means that less of the library code needs to be linked 
     226  into the final application, which makes smaller binaries.  It takes 
     227  longer to build libraries this way, though. 
     228   
     229 
     230==== How to make GHC build quickly ==== 
     231 
     232The GHC build tree is set up so that, by default, it builds a compiler 
     233ready for installing and using.  That means full optimisation, and the 
     234build can take a ''long'' time.  If you unpack your source tree and 
     235right away say {{{./configure; make}}}, expect to have to wait a while. 
     236For hacking, you want the build to be quick - quick to build in the 
     237first place, and quick to rebuild after making changes.  Tuning your 
     238build setup can make the difference between several hours to build 
     239GHC, and less than an hour.   
     240 
     241Here are the `build.mk` settings that we use to build fast: 
     242 
     243{{{ 
     244# My build settings for hacking on stage 2 
     245SRC_HC_OPTS     = -H32m -O -fasm -Rghc-timing 
     246GhcStage1HcOpts = -O -fasm 
     247GhcStage2HcOpts = -O0 -DDEBUG -Wall 
     248GhcLibHcOpts    = -O -fasm -XGenerics 
     249GhcLibWays      = 
     250SplitObjs       = NO 
     251GhcBootLibs     = YES 
     252}}} 
     253 
     254What do these options do? 
     255 
     256 {{{SRC_HC_OPTS = -H32m -O -fasm -Rghc-timing}}}:: 
     257  These options are added to the command line for all Haskell 
     258  compilations.  We turn on `-fasm`, because that halves compilation 
     259  time at the expense of a few percent performance. `-Rghc-timing` 
     260  prints out a line of timing info about each compilation.  It's handy 
     261  to keep an eye on.  `-Wall` turns on all the warnings; GHC is 
     262  meant to be warning-clean with `-Wall`. 
     263 
     264 {{{GhcStage1HcOpts = -O -fasm}}}:: 
     265  Build stage 1 optimised: we're going to be rebuilding stage 2 a lot, 
     266  so we want the compiler that does the building to be fast. 
     267 
     268 {{{GhcStage2HcOpts = -O0 -DDEBUG -Wall}}}:: 
     269  We turn off optimisation here, assuming you'll be modifying and 
     270  testing stage 2.  With optimisation off, rebuilding GHC after 
     271  modifying it will be ''much'' quicker, not only because the 
     272  individual compilations will be quicker, but also there will be 
     273  fewer dependencies between modules, so much less stuff is recompiled 
     274  after each modification. 
    234275  [[br]][[br]] 
    235   Haskell compilations by default have {{{-O}}} turned on, by virtue 
    236   of this setting from {{{config.mk}}}: 
    237  
    238  {{{SRC_HC_OPTS += -H16m -O}}}:: 
    239   {{{SRC_HC_OPTS}}} means "options for HC from the source tree", where 
    240   HC stands for Haskell Compiler.  {{{SRC_HC_OPTS}}} are added to 
    241   every Haskell compilation.  To turn off optimisation, you could add 
    242   this to {{{build.mk}}}: 
    243  
    244  {{{SRC_HC_OPTS = -H16m -O0}}}:: 
    245   Or you could just add {{{-O0}}} to {{{GhcHcOpts}}} to turn off 
    246   optimisation for the compiler.  See [wiki:Building/Hacking] 
    247   for some more suggestions. 
    248   [[br]][[br]] 
    249   When reading {{{config.mk.in}}}, remember that anything between 
    250   "@...@" signs is going to be substituted by {{{configure}}} later. 
    251   You ''can'' override the resulting definition if you want, but you 
    252   need to be a bit surer what you are doing.  For example, there's a 
    253   line that says: 
    254  
    255  {{{TAR = @TarCmd@}}}:: 
    256   This defines the Make variables {{{TAR}}} to the pathname for a 
    257   {{{tar}}} that {{{configure}}} finds somewhere.  If you have your 
    258   own pet {{{tar}}} you want to use instead, that's fine. Just add 
    259   this line to {{{mk/build.mk}}}: 
    260  
    261  {{{TAR = mytar}}}:: 
    262   You do not ''have'' to have a {{{mk/build.mk}}} file at all; if you 
    263   don't, you'll get all the default settings from 
    264   {{{mk/config.mk.in}}}. 
    265  
    266   You can also use {{{build.mk}}} to override anything that 
    267   {{{configure}}} got wrong.  One place where this happens often is 
    268   with the definition of {{{FPTOOLS_TOP_ABS}}}: this variable is supposed 
    269   to be the canonical path to the top of your source tree, but if your 
    270   system uses an automounter then the correct directory is hard to 
    271   find automatically.  If you find that {{{configure}}} has got it 
    272   wrong, just put the correct definition in {{{build.mk}}}. 
    273  
    274 == The story so far == 
    275  
    276 Let's summarise the steps you need to carry to get yourself a 
    277 fully-configured build tree from scratch. 
    278  
    279  * Get your source tree from somewhere (darcs repository or source 
    280    distribution).  Say you call the root directory {{{myghc}}} (it 
    281    does not have to be called {{{ghc}}}). 
    282  
    283  * (Optional) Use {{{lndir}}} or {{{mkshadowdir}}} to create a build 
    284    tree. 
    285 {{{ 
    286 $ cd myghc 
    287 $ mkshadowdir . /scratch/joe-bloggs/myghc-x86 
    288 }}} 
    289   (N.B. {{{mkshadowdir}}}'s first argument is taken relative to its 
    290   second.) You probably want to give the build tree a name that 
    291   suggests its main defining characteristic (in your mind at least), 
    292   in case you later add others. 
    293  
    294  * Change directory to the build tree.  Everything is going to happen 
    295    there now. 
    296 {{{ 
    297 $ cd /scratch/joe-bloggs/myghc-x86 
    298 }}} 
    299  
    300  * Prepare for system configuration: 
    301 {{{ 
    302 $ sh boot 
    303 }}} 
    304   (You can skip this step if you are starting from a source 
    305   distribution, and you already have {{{configure}}} and 
    306   {{{mk/config.h.in}}}.) 
    307  
    308  * Do system configuration: 
    309 {{{ 
    310 $ ./configure 
    311 }}} 
    312   Don't forget to check whether you need to add any arguments to 
    313   {{{configure}}}; for example, a common requirement is to specify 
    314   which GHC to use with 
    315   {{{--with-ghc=<path>}}}. 
    316  
    317  * Create the file {{{mk/build.mk}}}, adding definitions for your 
    318    desired configuration options. 
    319  
    320 You can make subsequent changes to {{{mk/build.mk}}} as often as you 
    321 like.  You do not have to run any further configuration programs to 
    322 make these changes take effect. In theory you should, however, say 
    323 {{{make clean; make}}}, because configuration option changes could 
    324 affect anything — but in practice you are likely to know what's 
    325 affected. 
    326  
    327 == Making things == 
     276  Also we turn on {{{-DDEBUG}}}, because that enables assertions and 
     277  debugging code in the compiler itself.  Turning on DEBUG makes 
     278  the compiler about 30% slower. 
     279 
     280 {{{GhcLibHcOpts = -O -fasm -XGenerics}}}:: 
     281  You almost certainly want optimisation ''on'' when building 
     282  libraries, otherwise the code you build with this compiler 
     283  goes really slowly.  {{{-fgenerics}}} add generics support to the 
     284  libraries - you can turn this off if you like (it'll make the 
     285  libraries a bit smaller), but you won't be able to use Generics in 
     286  the code you build against these libraries. 
     287 
     288 {{{GhcLibWays =}}}:: 
     289  Normally the profiled libraries are built.  Setting {{{GhcLibWays}}} to 
     290  empty disables this, so you only build the normal libs. 
     291 
     292 {{{SplitObjs = NO}}}:: 
     293  Object splitting causes each module to be split into smaller 
     294  pieces in the final library, to reduce executable sizes when 
     295  linking against the library.  It can be quite time and 
     296  memory-consuming, so turn it off when you're hacking. 
     297 
     298 {{{GhcBootLibs = YES}}}:: 
     299  If you're just interested in working on GHC, then you probably don't want 
     300  to build the "extralibs" libraries that we normally ship with GHC.   
     301  So when [wiki:Building/GettingTheSources getting the sources],  
     302  run `darcs-all` without the `--extra` option.  Alternatively, even if you have 
     303  the libraries in your tree, you can stop them being built by setting 
     304  `GhcBootLibs` in your `build.mk`. 
     305 
     306=== Making things === 
    328307 
    329308At this point you have made yourself a fully-configured build tree, so 
     
    331310 
    332311The first thing you need to know is that ''you must use GNU 
    333 {{{make}}}''.  On some systems (eg. FreeBSD) this is called 
    334 {{{gmake}}}, whereas on others it is the standard {{{make}}} command. 
    335 In this document we will always refer to it as {{{make}}}; please 
    336 substitute with {{{gmake}}} if your system requires it.  If you use a 
    337 the wrong {{{make}}} you will get all sorts of error messages (but no 
    338 damage) because the GHC {{{Makefiles}}} use GNU {{{make}}}'s 
     312`make`''.  On some systems (eg. FreeBSD) this is called 
     313`gmake`, whereas on others it is the standard `make` command. 
     314In this document we will always refer to it as `make`; please 
     315substitute with `gmake` if your system requires it.  If you use a 
     316the wrong `make` you will get all sorts of error messages (but no 
     317damage) because the GHC `Makefiles` use GNU `make`'s 
    339318facilities extensively. 
    340319 
    341 To just build the whole thing, {{{cd}}} to the top of your build tree 
    342 and type {{{make}}}.  This will prepare the tree and build the various 
    343 parts in the correct order, resulting in a complete build of GHC that 
    344 can even be used directly from the tree, without being installed 
    345 first. 
    346  
    347 == Bootstrapping GHC == 
    348  
    349 GHC requires a 2-stage bootstrap in order to provide full 
    350 functionality, including GHCi.  By a 2-stage bootstrap, we mean that 
    351 the compiler is built once using the installed GHC, and then again 
    352 using the compiler built in the first stage.  You can also build a 
    353 stage 3 compiler, but this normally isn't necessary except to verify 
    354 that the stage 2 compiler is working properly. 
    355  
    356 Note that when doing a bootstrap, the stage 1 compiler must be built, 
    357 followed by the runtime system and libraries, and then the stage 2 
    358 compiler.  The correct ordering is implemented by the top-level 
    359 {{{Makefile}}}, so if you want everything to work automatically it's 
    360 best to start {{{make}}} from the top of the tree.  The top-level 
    361 {{{Makefile}}} is set up to do a 2-stage bootstrap by default (when 
    362 you say {{{make}}}).  Some other targets it supports are: 
    363  
    364  {{{stage1}}}:: 
    365   Build everything as normal, including the stage 1 compiler. 
    366  
    367  {{{stage2}}}:: 
    368   Build the stage 2 compiler only. 
    369  
    370  {{{stage3}}}:: 
    371   Build the stage 3 compiler only. 
    372  
    373  {{{bootstrap}}}, {{{bootstrap2}}}:: 
    374   Build stage 1 followed by stage 2. 
    375  
    376  {{{bootstrap3}}}:: 
    377   Build stages 1, 2 and 3. 
    378  
    379  {{{install}}}:: 
    380   Install everything, including the compiler built in stage 2.  To 
    381   override the stage, say {{{make install stage=n}}} where {{{n}}} is 
    382   the stage to install. 
    383  
    384  {{{binary-dist}}}:: 
    385   make a binary distribution.  This is the target we 
    386   use to build the binary distributions of GHC. 
    387  
    388  {{{dist}}}:: 
    389   make a source distribution.  Note that this target 
    390   does {{{make distclean}}} as part of its work; 
    391   don't use it if you want to keep what you've built. 
    392  
    393 The top-level {{{Makefile}}} also arranges 
    394 to do the appropriate {{{make boot}}} steps (see 
    395 below) before actually building anything. 
    396  
    397 The {{{stage1}}}, {{{stage2}}} and {{{stage3}}} targets also work in 
    398 the {{{compiler}}} directory, but don't forget that each stage 
    399 requires its own {{{make boot}}} step: for example, you must do 
     320To just build the whole thing, `cd` to the top of your build tree and 
     321type `make`.  This will prepare the tree and build the various parts 
     322in the correct order, resulting in a complete build of GHC that can 
     323even be used directly from the tree (as `inplace/bin/ghc-stage2`), 
     324without being installed first. 
     325 
     326=== What to do after `make` has finished === 
     327 
     328If `make` completes successfully, then it should have created 
     329`inplace/bin/ghc-stage2`, which is a GHC binary you can run directly. 
     330It supports all the usual features, including GHCi if you pass the 
     331`--interactive` flag.   
     332 
     333In fact, the `inplace` directory looks a lot like an installed copy of 
     334GHC (see [wiki:Building/Installing]): there is a `bin` 
     335subdirectory containing various programs that can be run, including 
     336`ghc-pkg`, Haddock and `hsc2hs`. 
     337 
     338You can now run the testsuite, see [wiki:Building/RunningTests]. 
     339  
     340You can now install GHC, by typing `make install`. 
     341 
     342== What to do if you get a build failure == 
     343 
     344GHC is a complex system, with many platform-dependent components.  We 
     345try our best to make sure it builds out of the box as often as 
     346possible, but build failures are not uncommon.  If you get some kind 
     347of failure, don't panic. 
     348 
     349The chances are, someone else already encountered the same problem as 
     350you and has reported it on a mailing list or as a ticket.  Search for 
     351the error message using your favourite search engine, and if that 
     352doesn't turn anything up then search directly on this Trac.  Finally, 
     353if you don't find a solution: 
     354 
     355 * Ask someone: [wiki:MailingListsAndIRC] 
     356 * or just [wiki:ReportABug] 
     357 
     358== Components of the build == 
     359 
     360Here is a high level view of what happens when you build GHC: 
     361 
     362 * First, we build a few packages that GHC itself depends on, such as 
     363   `Cabal`, and `filepath`, using your installed GHC.  These packages 
     364   are under `libraries`, and each is built in a subdirectory 
     365   `dist-boot`; for example the bootstrap build of Cabal will be in 
     366   `libraries/Cabal/dist-boot`. 
     367 
     368 * Then we build GHC against these packages, still using your 
     369   installed GHC.  This is called the "stage 1" compiler (see 
     370   [wiki:Building/Architecture#Idiom:stages stages]).  You can run the 
     371   stage 1 compiler by invoking `inplace/bin/ghc-stage1`.  The stage 1 
     372   build of GHC happens in `compiler/stage1`. 
     373 
     374 * The stage 1 compiler is now used to build all the packages in the 
     375   `libraries` subdirectory, and the runtime system in `rts`. 
     376 
     377 * Finally, the stage 1 compiler is used to build GHC itself again, 
     378   this time against the libraries we just built.  This GHC is called 
     379   stage 2, and can be invokes as `inplace/bin/ghc-stage2`. 
     380 
     381There's an optional final stage, using the stage 2 compiler to build a 
     382stage 3 compiler, but this isn't strictly necessary, and is only used 
     383to check that the stage 2 compiler is working properly. 
     384 
     385 
     386== Developing in a GHC build tree == 
     387 
     388This section describes how to make changes to the sources of GHC, or 
     389its libraries and tools, and use the build system to build and test 
     390those changes. 
     391 
     392=== Bringing the whole tree up to date === 
     393 
     394The easy way to bring the tree up to date after making some changes is 
     395simply to go to the top of the tree and say `make`.  The build system 
     396has enough dependencies encoded into it that this should rebuild 
     397everything that needs to be rebuilt, in the correct order.  The 
     398downside is that it will build everything right though to the stage2 
     399compiler and including all the documentation, which might be overkill 
     400if all you wanted to do was to test a small change to GHC. 
     401 
     402=== Just building a single component === 
     403 
     404Suppose you want to make a small change to GHC itself and test it. 
     405Assuming that your tree is already up to date, the best way to do this 
     406is as follows: 
     407 
    400408{{{ 
    401 $ make boot stage=2 
     409$ cd ghc 
     410$ make stage2 
    402411}}} 
    403 before {{{make stage2}}} in {{{compiler}}}. 
     412 
     413This will bring the stage2 compiler up to date only.  In particular, 
     414it will ignore the fact that by modifying GHC you have thereby made 
     415the stage 1 compiler out of date, and hence possibly all the libraries 
     416and the whole of stage 2 are now also out of date.  If you did `make` 
     417from the top-level, all of these dependencies would be taken into 
     418account, and a lot of rebuilding would probably ensue. 
     419 
     420Each subdirectory of the source tree has a 
     421[wiki:BuildSystem/Architecture#Idiom:stubmakefiles stub makefile], 
     422most of which follow this pattern: 
     423 
     424{{{ 
     425dir = libraries/base 
     426TOP = ../.. 
     427include $(TOP)/mk/sub-makefile.mk 
     428}}} 
     429 
     430the main purpose of the stub makefile is to invoke `make` at the 
     431top-level of the tree: GHC's build system is 
     432[wiki:Building/Architecture#Idiom:non-recursivemake non-recursive], so 
     433in effect there is really just one `Makefile`, at the top level. 
     434However, the stub makefile has another purpose: it passes a target to 
     435the top-level `Makefile`, telling it to build just the components of 
     436the system in this directory.  For example, when you say `make` in the 
     437`rts` directory, this is actually equivalent to 
     438 
     439{{{ 
     440$ (cd ..; make all_rts) 
     441}}} 
     442 
     443where `make all_rts` makes every target in the `rts` subdirectory. 
     444Equivalently, `make all_libraries/base` at the top level would build 
     445everything in the `libraries/base` subdirectory.  To understand how 
     446the `all` targets are defined, see 
     447[wiki:Building/Architecture#Idiom:standardtargetsallcleanetc. standard targets]. 
     448 
     449You can also clean a single component of the tree, just by saying 
     450`make clean` in a subdirectory.  Again this is equivalent to issuing a 
     451command at the top of the tree of the form `make clean_libraries/base`. 
     452 
     453The `stage2` target in the `ghc` directory does something else: it 
     454also disables the dependencies that would otherwise cause the stage 1 
     455compiler and the libraries to be rebuilt.  It does this by simply 
     456setting `ASSUME_STAGE1=YES` and `ASSUME_LIBRARIES=YES` when invoking 
     457the top-level `make`. 
    404458 
    405459== Standard Targets == 
    406460 
    407 In any directory you should be able to make the following: 
    408  
    409  {{{boot}}}:: 
    410   does the one-off preparation required to get ready for the real 
    411   work, e.g. building the module dependency graph. 
    412   [[br]][[br]] 
    413   Invoking the {{{boot}}} target explicitly is not normally necessary. 
    414   From the top-level directory, invoking {{{make}}} causes {{{make boot}}} 
    415   to be invoked in various subdirectories, in the right 
    416   order.  Unless you really know what you are doing, it is best to 
    417   always say {{{make}}} from the top level first. 
    418   [[br]][[br]] 
    419   If you're working in a subdirectory somewhere and need to update the 
    420   dependencies, {{{make boot}}} is a good way to do it. 
    421  
    422  {{{all}}}:: 
    423   makes all the final target(s) for this Makefile.  Depending on which 
    424   directory you are in a "final target" may be an executable program, 
    425   a library archive, a shell script, or a Postscript file.  Typing 
    426   {{{make}}} alone is generally the same as typing {{{make all}}}. 
    427  
    428  {{{install}}}:: 
    429   installs the things built by {{{all}}} (except for the 
    430   documentation).  Where does it install them?  That is specified by 
    431   {{{mk/config.mk.in}}}; you can override it in {{{mk/build.mk}}}, or 
    432   by running {{{configure}}} with command-line arguments like 
    433   {{{--bindir=/home/simonpj/bin}}}; see {{{./configure --help}}} for 
    434   the full details. 
    435  
    436  {{{install-docs}}}:: 
    437   installs the documentation. Otherwise behaves just like 
    438   {{{install}}}. 
    439  
    440  {{{clean}}}:: 
     461The main targets understood by the top-level `Makefile` are as 
     462follows: 
     463 
     464 `all`:: 
     465  (default target, can be omitted).  Builds everything that needs to 
     466  be built for a GHC installation, including the stage 2 GHC, all 
     467  libraries and documentation.  After `make`, `make install` will not 
     468  need to do any further rebuilding. 
     469 
     470 `install`:: 
     471  installs the things built by `all`.  Where does it install them?  In 
     472  the places you specified when running `configure`, principally set 
     473  by the `--prefix` flag; see [wiki:Building/Installing]. 
     474 
     475 `clean`:: 
    441476  Delete all files from the current directory that are normally 
    442   created by building the program.  Don't delete the files that record 
    443   the configuration, or files generated by {{{make boot}}}.  Also 
    444   preserve files that could be made by building, but normally aren't 
    445   because the distribution comes with them. 
    446  
    447  {{{distclean}}}:: 
    448   Delete all files from the current directory that are created by 
    449   configuring or building the program. If you have unpacked the source 
    450   and built the program without creating any other files, {{{make distclean}}} 
    451   should leave only the files that were in the distribution. 
    452  
    453  {{{mostlyclean}}}:: 
    454   Like {{{clean}}}, but may refrain from deleting a few files that 
    455   people normally don't want to recompile. 
    456  
    457  {{{maintainer-clean}}}:: 
    458   Delete everything from the current directory that can be 
    459   reconstructed with this Makefile.  This typically includes 
    460   everything deleted by {{{distclean}}}, plus more: C source files 
    461   produced by Bison, tags tables, Info files, and so on. 
    462   [[br]][[br]]     
    463   One exception, however: {{{make maintainer-clean}}} should not 
    464   delete {{{configure}}} even if {{{configure}}} can be remade using a 
    465   rule in the {{{Makefile}}}. More generally, {{{make maintainer-clean}}} 
    466   should not delete anything that needs to exist in order to run 
    467   {{{configure}}} and then begin to build the program. 
    468   [[br]][[br]]     
    469   After a {{{maintainer-clean}}}, a {{{configure}}} will be necessary 
    470   before building again. 
    471  
    472 All of these standard targets automatically recurse into 
    473 sub-directories.  Certain other standard targets do not: 
    474  
    475  {{{depend}}}:: 
    476   make a {{{.depend}}} file in each directory that needs it. This 
    477   {{{.depend}}} file contains mechanically-generated dependency 
    478   information; for example, suppose a directory contains a Haskell 
    479   source module {{{Foo.lhs}}} which imports another module {{{Baz}}}. 
    480   Then the generated {{{.depend}}} file will contain the dependency: 
    481 {{{ 
    482 Foo.o : Baz.hi 
    483 }}} 
    484   which says that the object file {{{Foo.o}}} depends on the interface 
    485   file {{{Baz.hi}}} generated by compiling module {{{Baz}}}.  The 
    486   {{{.depend}}} file is automatically included by every Makefile. 
    487   Now that we are using Cabal for most of the building, most directories 
    488   don't support the `depend` target any more. Use `boot` instead. 
    489  
    490 Some {{{Makefile}}}s have targets other 
    491 than these.  You can discover them by looking in the 
    492 {{{Makefile}}} itself. 
    493  
    494 == Using GHC from the build tree == 
    495  
    496 If you want to build GHC and just use it direct from the build tree 
    497 without doing {{{make install}}} first, you can run the in-place 
    498 driver script.  To run the stage 1 compiler, use 
    499 `ghc/stage1-inplace/ghc`, stage 2 is 
    500 `ghc/stage2-inplace/ghc`, and so on. 
    501  
    502 Utils like `ghc-pkg` can be found under 
    503 `utils/ghc-pkg/install-inplace/bin/ghc-pkg` etc. 
    504  
    505 == Fast Making == 
    506  
    507 Sometimes the dependencies get in the way: if you've made a small 
    508 change to one file, and you're absolutely sure that it won't affect 
    509 anything else, but you know that {{{make}}} is going to rebuild 
    510 everything anyway, the following hack may be useful: 
    511  
    512 {{{ 
    513 $ make FAST=YES 
    514 }}} 
    515  
    516 This tells the make system to ignore dependencies and just build what 
    517 you tell it to.  In other words, it's equivalent to temporarily 
    518 removing the {{{.depend}}} file in the current directory (where 
    519 {{{mkdependHS}}} and friends store their dependency information). 
    520  
    521 A bit of history: GHC used to come with a {{{fastmake}}} script that 
    522 did the above job, but GNU make provides the features we need to do it 
    523 without resorting to a script.  Also, we've found that fastmaking is 
    524 less useful since the advent of GHC's recompilation checker (see the 
    525 User's Guide section on "Separate Compilation"). 
     477  created by `make`.  Don't delete the files that record the 
     478  configuration. 
     479 
     480 `distclean`:: 
     481  The same as `clean`, but also removes files created by `configure`. 
     482  `distclean` preserves files that would normally go in a source 
     483  distrubtion.  Note that after `make distclean` you will have to run 
     484  `./configure` before building again. 
     485 
     486 `maintainer-clean`:: 
     487  The same as `distclean`, but also removes files generated by `sh boot`. 
     488  The result of `make maintainer-clean` should be a completely clean 
     489  source tree, containing only revision-controlled files. 
     490 
     491 `binary-dist`:: 
     492  Builds a binary distribution.  A binary distribution is a copy of 
     493  the build tree with the source files removed, such that it can be 
     494  installed on a different system simply by issuing `./configure; make 
     495  install`. 
     496 
     497 `sdist`:: 
     498  Builds a source distribution.  A source distribution is a source 
     499  tree for GHC, complete with some extra files included for 
     500  convenience so that the user doesn't need to install `autoconf`, 
     501  `Happy`, or `Alex` in order to build it.  Hence `make sdist` only 
     502  works in a completely built tree.