Opened 3 months ago

Last modified 8 weeks ago

#15890 new feature request

Provide a way for hadrian users to always pass some options to hadrian itself

Reported by: alpmestan Owned by: alpmestan
Priority: normal Milestone: 8.10.1
Component: Build System (Hadrian) Version: 8.7
Keywords: Cc: RyanGlScott, snowleopard
Operating System: Unknown/Multiple Architecture: Unknown/Multiple
Type of failure: None/Unknown Test Case:
Blocked By: Blocking:
Related Tickets: Differential Rev(s): Phab:D5454
Wiki Page:

Description

And more generally, is there any chance that we could make something like our builder Args/predicate logic work for specifying arguments to hadrian itself?

Regardless of the answer to that last question, it seems only natural to have a simple/trivial solution for the case where we are always passing the same options to hadrian, over and over again, while working on GHC. "Yes, hadrian, I still want to build with the perf flavour, still with integer-simple, and you should still put all the build artifacts under ./ghc-prof-build/ while using all of my 4 cores", is something that I often told hadrian myself, while investigating a particular problem a few weeks ago, for example.

Ryan in particular has expressed interested in having a slightly cleaner solution than writing a tiny wrapper script that just does hadrian/build.sh --flavour=perf -oghc-prof-build $@ or something along those lines. Especially given that our Args infrastructure seems perfect to specify that kind of thing. It does seem a little bit circular though, of course.

An alternative would be look at a build.mk-ish file, without support for any Make construct, just the good old key=value syntax + comments, and reconstruct a Flavour and everything else out of it. This way, instead of "implicitly" passing the same arguments on the CLI, we would "implicitly" look them up from some file, without having to create a wrapper script.

(This build.mk-ish file is something I discussed with hvr some time ago, which he would like so as to programmatically specify how he wants his GHCs. He can't reasonably programmatically edit the settings file.)

Thoughts?

Change History (11)

comment:1 Changed 3 months ago by snowleopard

I think we can already almost do this, have a look at this section in the docs:

https://github.com/ghc/ghc/blob/master/hadrian/doc/user-settings.md#command-line-arguments

For example, it's possible to add the following to UserSetting.hs:

ryanFlavour :: Flavour
ryanFlavour = performanceFlavour -- Note that we inherit all performanceFlavour settings
    { name           = "ryan"
    , args           = args performanceFlavour <> ryanArgs -- Further tweaks to Args
    , integerLibrary = pure integerSimple }

ryanArgs :: Args
ryanArgs = builder Ghc ? input "//Prelude.hs"? pure ["-ddump-simpl", "-ddump-to-file"]

And then invoke Hadrian with --flavour=ryan flag.

There are several further improvements:

  • Make it possible to override the default flavour, so that it's not necessary to pass --flavour=ryan.
  • Add a new setting buildThreads to Flavour, so that one could set buildThreads = 4 as part of the user flavour, instead of setting it separately via command line.
  • Similarly, add the build root setting to Flavour to support buildRoot = "ghc-prof-build", which can still be overriden from the command line if need be.

Alp, Ryan: Does this sound like the right thing to do? I'm happy to implement.

comment:2 Changed 3 months ago by alpmestan

I'd be happy with this but my needs are less demanding than Ryan's, so I'll just let Ryan comment on your proposal.

comment:3 Changed 3 months ago by RyanGlScott

I'm not quite sure if I understand the proposal. Is the basic idea to just make it so that more things (e.g., the number of threads and the output directory) can be customized within Flavour? And, once that's in place, to allow overriding which build flavour is the default? If so, that sounds swell to me.

There's one more thing that I'd request be customizable, however, that wasn't mentioned in this ticket. I currently have to pass --progress-colour=never on the command line since there doesn't appear to be a way to set this in UserSettings.hs itself. Having this be customizable in the user settings would be quite nice.

comment:4 Changed 3 months ago by snowleopard

Is the basic idea to just make it so that more things (e.g., the number of threads and the output directory) can be customized within Flavour? And, once that's in place, to allow overriding which build flavour is the default? If so, that sounds swell to me.

Yes, that's the plan.

There's one more thing [...]

Sure, adding --progress-colour to Flavour shouldn't be difficult, I'll do that too.

comment:5 Changed 2 months ago by alpmestan

Have you made any progress on this, or should I pick it up (I will have the bandwidth for this soon) ?

comment:6 Changed 2 months ago by snowleopard

Alp, if you've got bandwidth for this, I suggest you go ahead. Alternatively, I will be able to implement this when this semester is finally over :)

comment:7 Changed 2 months ago by alpmestan

Differential Rev(s): Phab:D5454
Owner: set to alpmestan

I have one part done: the ability to override the default flavour (not the flavour whose name is default, I instead mean the flavour that we use when no --flavour=... argument is passed to hadrian). See https://phabricator.haskell.org/D5454, feedback welcome!

Regarding buildThreads, I'm finding it a little hard to get a chance to set the parallelism (through the shakeThreads field of ShakeOptions) when 1/ it's not overriden on the command line with -j 2/ using the default value specified by that buildThreads field I'm adding, since I don't have a handle on the flavour just yet. The same applies to the build root, I think, even though I haven't looked into that one a lot yet.

comment:8 Changed 2 months ago by mpickering

Another problem with UserSettings.hs is that it's tracked by version control.

comment:9 Changed 2 months ago by snowleopard

Matthew, see this comment on top:

-- If you want to customise your build you should copy this file from
-- hadrian/src/UserSettings.hs to hadrian/UserSettings.hs and edit your copy.
-- If you don't copy the file your changes will be tracked by git and you can
-- accidentally commit them.

comment:10 Changed 2 months ago by Ben Gamari <ben@…>

In c42eb2e6/ghc:

Hadrian: introduce userDefaultFlavour, making default flavour overridable

This patch introduces the `userDefaultFlavour` user setting. It should
be the name of the default flavour to use when no --flavour argument is
passed. Before this patch, we would just always default to... the
`default` flavour. With this patch, we default to whatever Flavour whose
name is `userDefaultFlavour`, therefore providing a way for users to
"persist" their choice of flavour, not having to repeat --flavour=[...]
in every hadrian command.

Test Plan:
Set `userDefaultFlavour = "quickest"`, run `hadrian/build.sh`, check
that the quickest flavour is indeed picked.

Reviewers: snowleopard, bgamari

Reviewed By: snowleopard

Subscribers: mpickering, rwbarton, carter

GHC Trac Issues: #15890

Differential Revision: https://phabricator.haskell.org/D5454

comment:11 Changed 8 weeks ago by osa1

Milestone: 8.8.18.10.1

Bumping milestones of low-priority tickets.

Note: See TracTickets for help on using tickets.