|Version 21 (modified by 7 years ago) (diff),|
Buildbot is currently down, and we are working on a replacement here: http://darcs.haskell.org/builder/
Setting up a nightly build
The GHC buildbot builds GHC on various platforms in various different ways each night, runs the test suite and performance benchmarks, and mails the results to the
email@example.com mailing list. We're always keen to add more build slaves to the setup, especially if you have a platform that doesn't already have a build slave, so if you'd like to join the fun, please let us know at cvs-ghc@…. If a platform is represented in the nightly builds, it's more likely we'll be able to identify and fix problems specific to that platform quickly.
To see the current status of the builds:
To create a new build slave
First you, as a buildbot client, need to agree a buildbot username (
myUser) and password (
myPass) with the buildbot admins (just pick a username and password and send it to
firstname.lastname@example.org). You'll also need to decide:
- when the build(s) should happen (time and timezone!)
- build HEAD, STABLE, or (the default) alternate between the two?
- full build (up to stage 3, with extra-libs, full testsuite, and 5 nofib runs) or a fast build (stage 2, no extra-libs, fast testsuite, no nofib runs), or (the default) something in-between
Finally, if there is anything special that needs to be done for the client (e.g. if gcc is in an unusual place) then you'll need to let the admins know.
Then you'll need to install buildbot and its dependencies on the machine that will be doing the nightly build; see the BuildBot website for details. NB. if you're on Windows, you'll need to install BuildBot under Cygwin using the Cygwin Python; there are various problems getting the GHC build to work via BuildBot using the native Win32 Python, so we've given up on that route for now.
In order to actually do the build, you'll also need the prerequisite tools for a GHC build, see Building/Preparation, and you should have darcs installed too (we recommend at least version 0.9).
Now create and enter the directory you want the buildbot client to work in
$ mkdir /buildbot/ghc $ cd /buildbot/ghc
and tell buildbot to set up a slave there
$ buildbot create-slave . darcs.haskell.org:9989 myUser myPass
This will print a few lines asking you to fill in
info/host. In the latter file, please include information on what operating system and architecture the machine is running.
By default it seems that buildbot uses a umask of 077, this probably isn't what you want. For example, if you intend to upload distributions, they'll have restricted permissions. You can change it in the
buildbot.tac file, we set it to 002 for example:
umask = 002
It also created
Makefile.sample; we recommend renaming this to
Makefile. You can now start the buildbot client with
make start and stop it with
You can watch what your slave is doing by looking at the
twistd.log file in the directory in which you're running your slave.
Automating startup: Unix
The easiest way to make the client start up automatically is to use
crontab -e, and add this line to your crontab file:
@reboot cd <buildbotdir> && make start
Remember to change
<buildbotdir> to your buildbot directory.
Cron will run the command in a minimal environment: it won't execute your normal shell startup files, so you won't have your usual
PATH settings, for example. To get the right
PATH and other environment variables, we suggest adding them to the
make start rule in
<buildbotdir>/Makefile. For example, my start rule looks something like this:
start: PATH=/usr/bin:/bin:/home/simonmar/bin \ http_proxy=http://18.104.22.168:80 \ twistd --no_save -y buildbot.tac
It might be a good idea to have the buildbot restarted once a day before your build is due to start, just in case it has died for any reason. I have another line in my crontab that looks like this:
0 17 * * * cd <buildbotdir> && (make stop; make start)
To restart the client at 17.00, before the builds start at 18.00.
It's a good idea to test that running the client via a cron job actually works, so test it: setup a temporary cron job to start the client in a couple of minutes time, check that the client is up and running, and maybe force a build via the status page to check that the build environment is working.
Automating startup: Windows
I did it the following way. Create a script in
PATH=/bin:/usr/bin cd <buildbotdir> make stop make start
(don't forget to create the script as a Unix text file, not a DOS text file, otherwise strange things will probably happen, they did to me anyway).
Create a new "Scheduled Task" via Control Panel->Scheduled Tasks. The command you want to run is
Schedule the task to run (a) at startup and possibly also (b) once a day, before your build is due to start. You can add multiple schedulers for a task by checking the box at the bottom of the "Schedule" page of the scheduled task settings.
(for the admins only...)
Pull the buildbot master configuration:
$ darcs get email@example.com:/home/buildbot/master $ cd master
master.cfg. Add new entries to
builders as necessary. Record and push the changes. Then restart the build master:
$ ssh firstname.lastname@example.org "cd master; make reconfig"
If there is anything unusual about the machine the build is being run on, e.g. the path to
gcc is different, then you will need to add a field for the unusual thing to GhcDefaultConfig and alter the build steps to make use of it. Then make a special factory for the build client you are adding with this field changed as appropriate.
Did it work?
Once the master is reconfiged and the client is started, the client should become visible on http://darcs.haskell.org:8010/
At present there is no way to force an immediate test build.
On MacOS X, when the user who runs the buildbot process logs out, the process keeps running, but it loses the ability to do DNS queries! This shows up as failures like this:
darcs failed: Not a repository: http://darcs.haskell.org/ghc (Failed to download URL http://darcs.haskell.org/ghc/_darcs/inventory: couldn't resolve host name)
To resolve this, there is some extra work to do when you configure your buildbot client: http://buildbot.net/trac/wiki/UsingLaunchd