Changes between Version 1 and Version 2 of Status/Oct09

Nov 3, 2009 1:33:38 PM (5 years ago)



  • Status/Oct09

    v1 v2  
    11= GHC status October 2009 = 
    3 We are just about to make our main annual release, of GHC 6.12. 
     3We are just about to make our annual major release, of GHC 6.12.1 (in the following we will say "GHC 6.12" to refer to GHC 6.12.1 and future patch-level releases along the 6.12 branch). 
     5GHC continues to be very active, with many opportunities 
     6for others to get involved.  We are particularly eager to find partners 
     7who are willing to take responsibility for a particular platform  
     8(e.g. '''Simon M fill in the most popular Tier 2 plaform'''); see [Platforms]. 
     10== The GHC 6.12 release ==  
     12We usually try to make a major release of GHC immediately after ICFP. 
     13It has been somewhat delayed this year, but we expect to release  
     14GHC 6.12 during November or December 2009.  Apart from the myriad of 
     15new bug fixes and minor enhancements, the big new things in 6.12 are: 
     17 * Considerably improved support for parallel execution.  GHC 6.10 
     18   would execute parallel Haskell programs, but performance was often 
     19   not very good.  Simon Marlow has done lots of performance tuning 
     20   in 6.12, removing many of the accidental (and largely invisible) 
     21   gotchas that made parallel programs run slowly. 
     23 * As part of this parallel-performance tuning, Satnam Singh and Simon 
     24   Marlow have developed ThreadScope, a GUI that lets you see what is 
     25   going on inside your parallel program.  It's a huge step forward 
     26   from "It takes 4 seconds with 1 processor, and 3 seconds with 8 
     27   processors; now what?".  ThreadScope will be released separately 
     28   from GHC, but at more or less the same time as GHC 6.12. 
     30 * Dynamic linking is now supported on Linux, and support for other 
     31   platforms will follow.  Thanks for this most recently go to 
     32   the [ Industrial Haskell Group] (thank you, IHG!) who 
     33   pushed it into a fully-working state; dynamic linking is the culmination of the work of 
     34   several people over recent years.[[BR]] 
     35   [[BR]] 
     36   One effect of dynamic linking is that binaries shrink dramatically, because the run-time 
     37   system and libraries are shared.  Perhaps more importantly, it is possible 
     38   to make dynamic plugins from Haskell code that can be used from other 
     39   applications. 
     41 * The I/O libraries are now Unicode-aware, so your Haskell programs 
     42   should now handle text files containing weird characters. 
     44 * The package system has been made more robust, by associating each 
     45   installed package with a unique identifier based on its exposed ABI. 
     46   Now, cases where the user re-installs a package without recompiling 
     47   its dependencies will be detected, and the broken dependencies will 
     48   be disabled.  Previously, this would lead to obscure compilation errors, 
     49   or worse, segfaulting programs.  This involved a large amount of 
     50   internal restructuring, but it paves the way for future improvements 
     51   to the way packages are handled.  For instance, in the future we 
     52   expect to track profiled packages independently of non-profiled ones, 
     53   and we hope to make it possible to upgrade a package in an ABI-compatible 
     54   way, without recompiling the packages that depend on it.  This latter 
     55   facility will be especially important as we move towards using 
     56   more shared libraries. 
     58 * A variety of small improvements to data types: record punning, 
     59   declararing constructors with class constraints, GADT syntax for 
     60   type fammilies etc. 
     62 * You can omit the "`$`" in a top-level Template Haskell splice, which 
     63   makes the TH call look more like an ordinary top-level declaration with 
     64   a new keyword. 
     66 * We're are deprecating `mdo` for recursive do-notation, in favour of 
     67   the more expressive `rec` statement. 
     69 * We've concluded that the implementation of impredicative 
     70   polymorphism is unsustainably complicated, so we are re-trenching. 
     71   It'll be depreceated in 6.12 (but will still work), and will be either 
     72   removed or replaced with something simpler in 6.14. 
     74For more detail, see the release notes in the  6.12 User manual [UM],  
     75which mention many things skipped over here. 
     77Another big change with GHC 6.12 is that Hackage and the Haskell Platform is 
     78allowing GHC HQ to get out of the libraries business.  So the plan is 
     80 * We release GHC 6.12 with very few libraries 
     81 * Bill Library Author downloads GHC 6.12 and tests his libraries 
     82 * The next Haskell Platform release packages GHC 6.12 with these tested libraries 
     83 * Joe User downloads the Haskell Platform. 
     84 * Four months later there's a new HP release, still with GHC 6.12, 
     85   but with more or better libraries.  The HP release cycle is 
     86   decoupled from GHC 
     88So if you are Joe User, you want to wait for the HP release.  Don't 
     89grab the GHC 6.12 release.  It'll be perfectly usable, but only if you 
     90use (an up to date) cabal-install to download libraries, and accept that 
     91they may not be tested with GHC 6.12. 
     93Lastly, GHC 6.12 has a totally re-engineered build system, with much-improved 
     94dependency tracking [BuildSystem].  While there have been lots of teething problems, the  
     95old system had become increasingly troublesome, so we had to make a change. 
     97== What's hot for the next year == 
     99GHC continues to be a great substrate for research.  Here are the main things 
     100we are working on at the moment. 
     102=== Type systems === 
     104Type families have proved a great success.  From the outside it might 
     105seem that they are done -- after all, they are in GHC 6.10 -- but the 
     106internals are quite fragile and it's amazing that it all works well as 
     107it does.  (Thanks to Manuel's work.)  Tom Schrijver, Dimitrios 
     108Vytiniotis, Martin Sulzmann, and Manuel Chakravarty have been working 
     109with Simon PJ to understand the fundamentals and, in the light of that 
     110insight, to re-engineer the implementation into something more robust. 
     111We have developed the "!OutsideIn" algorithm, which gives a much nicer 
     112account of type inference than our previous story of type inference 
     113[ICFP09].  More controversially, we now believe that local let/where 
     114bindings should not be generalised [LetGen].  Dimitrios is building a 
     115prototype that embodies these ideas, which we'll then transfer into 
     118Meanwhile, Dimitrios, Simon, and Stephanie Weirich are also working on 
     119fixing one of GHC's more embarassing bugs (Trac #1969 '''Simon check'''), 
     120whereby an interaction of type families and the newtype-deriving can 
     121persuade GHC to generate type-unsound code. It's remained un-fixed 
     122because the obvious approaches seem to be hacks, so the cure was as 
     123bad as the disease.  We think we are on to something; stay tuned. 
     125=== Interemediate language and optimisation === 
     127Although it is, by design, invisible to users, GHC's intermediate language 
     128and optimsation passes have been receiving quite a bit of attention. 
     129Some highlights 
     131 * Read Max Bolingbroke's paper on Strict Core [MaxB], a possible new 
     132   intermediate language for GHC.  Adopting Strict Core would be a Big  
     133   Change, however, and we have not decided to do so (yet). 
     135 * Simon PJ totally re-engineered the way that INLINE pragmas are  
     136   implemented., with the goal of making them more robust and  
     137   predicatable [InlinePatch].  There's a new CONLIKE pragma which 
     138   affects rule matching. 
     140 * Peter Jonsson did an internship in which he made a start on turning 
     141   GHC into a supercompiler.  Neil Mitchell's terrific PhD thesis suggested 
     142   that supercompliation works well for Haskell [NeilM], and Peter has been working on 
     143   supercompilation for Timber as part of his own PhD [Peter J]. 
     144   The GHC version isn't ready for prime time yet, but Simon PJ (now 
     145   educated by Peter and Neil) is keen to pursue it. 
     147 * An internal change in GHC 6.12 is the addition of "annotations", a 
     148   general-purpose way for a programmer to add annotations to 
     149   top-level definitions that can be consulted by a core-to-core pass, 
     150   and for a core-to-core pass to pass information to its successors 
     151   [Annotations]. 
     152   We expect to use these annotations increasingly in GHC itself. 
     154=== Parallelism === 
     156SIMON MARLOW to write 
     158=== Data Parallelism === 
     160MANUEL CHAKRAVARTY to write 
     162=== Code generation === 
     164For the last two years we have been advertising a major upheaval in 
     165GHC's back end.  Currently a monolithic "code generator" converts 
     166lambda code (the STG language) into flat `C--`; "flat" in the sense 
     167that the stack is manifested, and there are no function calls. 
     168The upheaval splits this in to a pipeline of passes, with a relatively-simple 
     169conversion of lambda code into `C--` (with function calls), followed 
     170by a succession of passes that optimise this code, and flatten it 
     171(by manifesting the stack and removing calls). 
     173John Dias is the principal architect of this new path, and it is in GHC 
     174already; you can switch it on by saying `-fnew-codegen`.  What remains is 
     175(a) to make it work 100% (currently 99%, which is not good enough); (b) 
     176commit to it, which will allow us to remove gargantuan quantities of 
     177cruft; (c) exploit it, by implementing cool new optimisations at the `C--` level; 
     178(d) take it further by integrating the native code generators into the  
     179same pipeline.  You can read more on the wiki [CodeGen]. 
     181Several passes of the new code generation pipeline are supported by Hoopl, 
     182a Haskell library that makes it easy to write dataflow analyses and optimisations 
     183over `C--` code [Hoopl].  We think Hoopl is pretty cool, and have well-advanced 
     184ideas for how to improve it a lot more. 
     186All of this has taken longer than we hoped.  Once the new pipeline is in place 
     187we hope that others will join in.  For example, David Terei did an interesting 
     188undergraduate project on using LLVM as a back end for GHC [Terei], and  
     189Krzysztof Wos is just beginning an undergraduate project on optimisation in the new pipeline. 
     190We are particularly grateful to Ben Lippmeier for his work on the SPARC native code generator. 
     193== References == 
     195[Platforms]  '''A wiki ref''' 
     197[IHG]  Industrial Haskell Group  
     200[UM]   GHC 6.12 user manual  
     203[BuildSystem] '''A wiki ref''' 
     205[ICFP09] "Complete and Decidable Type Inference for GADTs", Tom Schrijvers,  
     206Simon Peyton Jones, Martin Sulzmann, and Dimitrios Vytiniotis. ICFP'09.  
     209[LetGen] "Let should not be generalised", Dimitrios Vytiniotis, Simon 
     210Peyton Jones, and Tom Schrijvers, submitted to TLDI'10.  
     213[Hoopl] "Hoopl: dataflow optimisation made simple", Norman Ramsey, John Dias, and Simon Peyton Jones,  
     214rejected by POPL 2010. 
     217[Terei] '''Manuel: what URL?''' 
     219[MaxB] "Types are calling conventions", Max Bolingbroke and Simon Peyton Jones, Haskell Symposium 2009. 
     222[InlinePatch] The big INLINE patch [] 
     224[NeilM] "Transformation and Analysis of Functional Programs", Neil Mitchelll,  
     225PhD thesis, University of York, 2009.  
     228[PeterJ] "Positive supercompliation for a higher order call-by-value language",  
     229Peter Jonsson and Johan Nordlander, POPL 2009. 
     232[Annotations] '''A wiki ref''' 
     234[CodeGen] '''A wiki ref'''