Changes between Version 39 and Version 40 of NewPlugins


Ignore:
Timestamp:
May 10, 2011 3:50:55 PM (4 years ago)
Author:
thoughtpolice
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • NewPlugins

    v39 v40  
    55This page explains what the plug-in mechanism does, how to use it, and a little about the implementation.  For discussion, and the current state of play, see the ticket: #3843. If you're interested in writing plugins for GHC, '''please comment and give feedback, we want to do it right'''!
    66
    7 1/17/11: I (Austin Seipp) am working on getting the patch cleaned up a little more and tidying it up before it gets integrated. Still need testsuite patches.
     7 * 1/17/11: I (Austin Seipp) am working on getting the patch cleaned up a little more and tidying it up before it gets integrated. Still need testsuite patches.
     8 * 5/10/11: I have been busy, but there's a new git repository!
    89
    910NB. Ridiculously incomplete writing/documentation.
     
    1112== Current overview ==
    1213
    13 Get GHC from its HEAD repository, and apply this patch:
     14I currently have a branch of GHC with plugins support, that is occasionally (once a week or so) merged with master for the latest fixes/updates. You can find it here:
    1415
    15 http://hackage.haskell.org/trac/ghc/raw-attachment/ticket/3843/ghc_plugins_support_2010_11_19.2.dpatch
     16https://github.com/thoughtpolice/ghc/tree/plugins
    1617
    17 Then build GHC like normal.
     18so just run:
     19
     20{{{
     21$ git remote add aseipp [email protected]:thoughtpolice/ghc.git
     22$ git pull --all
     23$ git checkout -b plugins_temp aseipp/plugins
     24}}}
     25
     26Then build GHC like normal. You don't need to check out branches of any libraries, because the compiler work only touches the code under `./compiler`.
    1827
    1928Now GHC understands the `-fplugin` and `-fplugin-arg` options. You essentially install plugins for GHC by `cabal install`ing them, and then calling GHC in the form of:
     
    8594=== Reflections on the current API for Core passes ===
    8695
    87 Scala's compiler has a plugin API described by [1], with examples at [2]. Scala is a bit of a different beast, but the compiler fully supports compilation plugins in the same manner we would like GHC to - more to the point, we want to make sure we have a '''good API for specifying when plugins are used and executed'''. This is a part of the API that is currently rather simplistic and ad-hoc: we just modify the entire list of compiler passes and return a new one for the optimizer to run. We would like a better mechanism for controlling when plugin passes happen.
     96Scala's compiler has a plugin API described by [1], with examples at [2]. Scala's compiler supports insertion of a plugin at almost any stage in the compiler, internal or external (so you can for example, write a 'check for division by zero' plugin that runs right after the parser/typechecker.) Furthermore, the Scala design actually involves annotating your plugin with a set of 'runsBefore' or 'runsAfter' constraints, which specify which other internal phases your plugin occurs before or after. After all plugins are loaded, internal and external (plugin-made) phases are constructed into a dependency graph and ordered using the constraints, to yield a sorted list of the final phases to run.
    8897
    89 Of course, GHC is changing all the time - authors of plugins should be ready to deal with differences and changes in versions (e.g. Core can change, don't doubt it for a second,) but if we will be providing a public API for writing plugins, we need to make sure it's sensible and usable for the future to come.
    90 
    91 Part of the purpose of the plugins work (in my vision) is to help lower the barrier to working with and on GHC, as well as piggyback off the work it's done. So we want to make sure plugins are done right, and look at what others have done right (and wrong.)
    92 
    93 TODO: expand on this bit and scala's compiler plugin design, primarily its `runsBefore` and `runsAfter` constraints, the old design Max had, and, by extension, on a better method of controlling when plugin passes are run, instead of just relying on a weak (and, I personally think somewhat fragile) installation function.
     98Barring the feature to let you insert a compiler plugin just *anywhere* (we only care about intermediate representations,) it would be nice to adopt this dependency-graph based interface. Mostly, because it saves us a little headache in having to potentially sift through the list of core passes to install ourselves. Furthermore resolving the dependencies and finalizing the list of plugins ends a question about 'which plugin loaded and installed itself first' (intuitively we would like to say left-to-right on the command line, but that's a little fragile to depend on.) It also makes it easier to have passes which may run before a pass that occurs multiple times, etc. And we can still compose `CoreToDo`s using `CoreDoPasses`.
    9499
    95100= The Future =
     
    98103Aside from manipulating the core language, we would also like to manipulate the C-- representation GHC generates for modules too.
    99104
    100 The [wiki:Commentary/Compiler/NewCodeGen new code generator] (more precisely, the optimizing conversion from STG to Cmm part of GHC) based on hoopl for dataflow analysis is going to be merged into HEAD Real Soon Now. It would be best perhaps to leave this part of the interface alone until it is merged - clients of the interface could then use hoopl to write dataflow passes for Cmm.
    101 
    102  * New code generator is based on several phases:
    103   * Conversion from STG to Cmm
    104   * Basic optimisations (basic block elim., liveness analysis, etc)
    105   * CPS Conversion
    106   * More basic optimisations on CPS form
    107   * Conversion to old Cmm representation, then passed to backend code generator.
    108 
    109  * We need some sort of interface to describe how to insert it into the pipeline - is the Core approach best here, that is, an installation function that just inserts itself into a list?
    110 
    111  * Add new interface to `Plugin` next to `installCoreToDos` i.e. `installCmmPass`, that installs a pass of type `CmmGraph -> CmmGraph` into the optimization pipeline somehow?
     105TODO: fixme
    112106
    113107=== Rough API possibilities ===
    114 
    115 NB. This section is based mostly on the upcoming NCG mega-patch, which re-engineers the C-- implementation to use Hoopl. If you want to follow along, you'll need the patch, which can be seen and downloaded here:
    116 
    117 http://www.haskell.org/pipermail/cvs-ghc/2011-January/059015.html
    118108
    119109TODO: fixme
     
    121111==== Composing a hoopl analysis ====
    122112
    123 [http://hackage.haskell.org/package/hoopl Hoopl] is the main workhorse behind the new GHC code generator - it is a sophisticated, higher order and highly polymorphic library for writing optimization and dataflow passes over imperative code graphs. It is based on the work of Lerner, Grove and Chambers [3], and one of the nice properties of the design is that it makes it easy to compose independent dataflow analysis to create a super-analysis that is more powerful than either analysis on its own, without the tedium of having to manually write such a super-analysis.
    124 
    125 API question here: how should we compose analysis that plugins provide with ones that GHC provides?
    126 
    127 TODO: fixme. check and explain current Hoopl combinators for deep/shallow passes (deepFwdRw and deepBwdRw,) as well as composing them (thenFwdRw and thenBwdRw) and how they should fit into this part of the API
    128113
    129114= References =
     
    131116 * [1] "Scala Compiler Phase and Plug-In Initialization for Scala 2.8" (PDF) - http://www.scala-lang.org/sites/default/files/sids/nielsen/Thu,%202009-05-28,%2008:13/compiler-phases-sid.pdf
    132117 * [2] "Writing Scala Compiler Plugins" http://www.scala-lang.org/node/140
    133  * [3] "Composing dataflow analysis and transformation" (PDF) - http://cseweb.ucsd.edu/~lerner/UW-CSE-01-11-01.pdf