Changes between Version 3 and Version 4 of HackageTesting


Ignore:
Timestamp:
Apr 23, 2012 7:23:56 PM (3 years ago)
Author:
dterei
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • HackageTesting

    v3 v4  
    11= Hackage Testing = 
    22 
    3 We now have an automated program for performing regression tests against all of hackage. 
     3A great way to test changes to GHC (and Cabal) is to compile all packages on Hackage and compare the results to other runs that use different options or versions of GHC. 
    44 
    5 To check it out, run 
    6 {{{ 
    7 darcs get http://darcs.haskell.org/hackage-test/ 
    8 }}} 
     5We have an automated program for performing this, allowing all of Hackage to act as a regression test. At the moment we just compare which packages build to which ones don't under two different runs. In the future we could also run a packages testsuite and/or benchmark suite to get more information and also track performance regressions. 
    96 
    10 '''Caution:''' This program can cause arbitrary code to run on your machine. For example, if any modules use TH, then the TH will be executed. `configure` scripts will be run. Custom `Setup.hs` programs will be run. 
    11  
    12 == Example of usage == 
    13  
    14 Here's a run with the HEAD, no special options: 
    15 {{{ 
    16 $ ht normal ../cabal-install/dist/build/cabal/cabal  \ 
    17             /home/ian/ghc/darcs/ghc/inst/bin/ghc     \ 
    18             /home/ian/ghc/darcs/ghc/inst/bin/ghc-pkg \ 
    19             "" "" 
    20 }}} 
    21 [7.75 hours pass] 
    22  
    23 Then another run, this time using `-XAlternativeLayoutRule` to compile 
    24 each package (but not the build dependencies of the package): 
     7This tool is now hosted on [https://github.com/dterei/Hackager GitHub] and on [http://hackage.haskell.org/package/hackager Hackage]. 
    258 
    269{{{ 
    27 $ ht alternative ../cabal-install/dist/build/cabal/cabal  \ 
    28                  /home/ian/ghc/darcs/ghc/inst/bin/ghc     \ 
    29                  /home/ian/ghc/darcs/ghc/inst/bin/ghc-pkg \ 
    30                  ""                                       \ 
    31                  "--ghc-option=-XAlternativeLayoutRule" 
     10$ cabal install hackager 
     11$ ... 
     12$ hackager --help 
    3213}}} 
    33 [7 hours pass] 
    34  
    35 And finally a comparison of the results: 
    36  
    37 {{{ 
    38 $ htc normal alternative 
    39                             normal 
    40                          Buildable Build failed Deps failed Not tried 
    41 alternative Buildable          628            0           0         0 
    42             Build failed        73          215           0         0 
    43             Deps failed          0            0         170         0 
    44             Not tried            0            0           0         0 
    45 }}} 
    46 i.e. 73 packages became unbuildable when the alternative layout rule was 
    47 used.