Changes between Version 12 and Version 13 of Building/RunningTests


Ignore:
Timestamp:
Feb 6, 2008 1:19:47 PM (6 years ago)
Author:
simonmar
Comment:

update

Legend:

Unmodified
Added
Removed
Modified
  • Building/RunningTests

    v12 v13  
    8484        optc                    -- -O -fvia-C 
    8585        optasm                  -- -O -fasm 
    86         prof                    -- -O -prof -auto-all 
     86        profc                   -- -O -prof -auto-all -fvia-C 
    8787        profasm                 -- -O -prof -auto-all -fasm 
    88         unreg                   -- -unreg 
    8988        ghci                    -- (run only, not compile) run test under GHCi 
    9089        extcore                 -- -fext-core 
    9190        optextcore              -- -O -fext-core 
    92         threaded                -- -threaded 
     91        threaded1               -- -threaded -debug 
     92        threaded2               -- -threaded -O, and +RTS -N2 at run-time 
    9393        hpc                     -- -fhpc 
    9494}}} 
    9595certain ways are enabled automatically if the GHC build in the local 
    96 tree supports them.  Ways that are enabled this way are optasm, prof, 
    97 profasm, unreg, threaded, and ghci. 
     96tree supports them.  Ways that are enabled this way are optasm, profc, 
     97profasm, threaded1, threaded2, and ghci. 
    9898 
    9999= Updating tests when the output changes = 
     
    103103new output like so: 
    104104{{{ 
    105         make accept TESTS=<test-name> 
     105        make accept TEST=<test-name> 
    106106}}} 
    107107where <test-name> is the name of the test.  In a directory which 
    108108contains a single test, or if you want to update *all* the tests in 
    109 the current directory, just omit the 'TESTS=<test-name>' part. 
     109the current directory, just omit the 'TEST=<test-name>' part. 
    110110 
    111111= Adding a new test = 
     
    141141 
    142142 2. Having found a suitable place for the test, give the test a name. 
    143     Follow the convention for the directory in which you place the 
     143    For regression tests, we often just name the test after the bug number (e.g. 2047). 
     144    Alternatively, follow the convention for the directory in which you place the 
    144145    test: for example, in typecheck/should_compile, tests are named 
    145146    tc001, tc002, and so on.  Suppose you name your test T, then 
     
    172173 1. Edit all.T in the relevant directory and add a line for the test.  The line is always of the form 
    173174{{{ 
    174       test(<name>, <opt-fn>, <test-fn>, <args>) 
     175      test(<name>, <setup>, <test-fn>, <args>) 
    175176}}} 
    176177    where 
     
    178179    ''<name>'' is the name of the test, in quotes (' or "). 
    179180 
    180     ''<opt-fn>''  is a function (i.e. any callable object in Python) 
     181    ''<setup>''  is a function (i.e. any callable object in Python) 
    181182    which allows the options for this test to be changed. 
    182     There are several pre-defined functions which can be 
     183    There are many pre-defined functions which can be 
    183184    used in this field: 
    184185 
     
    187188        '''skip'''                  skip this test 
    188189 
     190        '''skip_if_no_ghci'''       skip unless GHCi is available 
     191 
     192        '''skip_if_fast'''          skip if "fast" is enabled 
     193 
     194        '''skip_if_platform(platform)'''  skip if we're on the named platform 
     195 
     196        '''skip_if_tag(tag)'''      skip if the compiler has a given tag 
     197 
     198        '''skip_unless_tag(tag)'''  skip unless the compiler has a given tag 
     199 
    189200        '''omit_ways(ways)'''       skip this test for certain ways 
    190201 
    191202        '''only_ways(ways)'''       do this test certain ways only 
    192203 
     204        '''extra_ways(ways)'''      add some ways which would normally be disabled 
     205 
    193206        '''omit_compiler_types(compilers)'''                           skip this test for certain compilers 
    194207 
     
    205218        '''set_stdin(file)'''       use a different file for stdin 
    206219 
     220        '''no_stdin'''              use no stdin at all (otherwise use `/dev/null`) 
     221 
    207222        '''exit_code(n)'''          expect an exit code of 'n' from the prog 
    208223 
     
    213228        '''no_clean'''              don't clean up after this test 
    214229 
    215     These functions should normally not be used; instead, use the `expect_broken*` 
     230        '''extra_clean(files)'''    extra files to clean after the test has completed 
     231 
     232        '''reqlib(P)'''             requires package P 
     233 
     234        '''req_profiling'''         requires profiling 
     235 
     236        '''ignore_output'''         don't try to compare output 
     237 
     238        '''alone'''                 don't run this test in parallel with anything else 
     239 
     240        '''literate'''              look for a `.lhs` file instead of a `.hs` file 
     241 
     242        '''c_src'''                 look for a `.c` file 
     243 
     244        '''cmd_prefix(string)'''    prefix this string to the command when run 
     245 
     246        '''normalise_slashes'''     convert backslashes to forward slashes before comparing the output 
     247 
     248    To use more than one modifier on a test, just put them in a list. 
     249    For example, to expect an exit code of 3 and omit way 'opt', we could use 
     250{{{ 
     251      [ omit_ways(['opt']), exit_code(3) ] 
     252}}} 
     253      as the `<setup>` argument. 
     254 
     255    The following should normally not be used; instead, use the `expect_broken*` 
    216256    functions above so that the problem doesn't get forgotten about, and when we 
    217257    come back to look at the test later we know whether current behaviour is why 
     
    225265 
    226266        '''expect_fail_if_compiler_type(compiler)'''   expect failure from a certain compiler 
    227  
    228       You can compose two of these functions together by 
    229       saying compose(f,g).  For example, to expect an exit 
    230       code of 3 and omit way 'opt', we could use 
    231 {{{ 
    232       compose(omit_ways(['opt']), exit_code(3)) 
    233 }}} 
    234       as the <opt-fn> argument.  Calls to compose() can of 
    235       course be nested, but it is simpler to use the composes 
    236       function which takes a list of functions to be composed, e.g.: 
    237 {{{ 
    238       composes([omit_ways(['opt']), exit_code(3), expect_broken(123)]) 
    239 }}} 
    240267 
    241268    ''<test-fn>'' 
     
    279306 
    280307      run_command 
    281             Just run an arbitrary command.  The 
    282             output is checked against T.stdout and 
    283             T.stderr, and the stdin and expected 
    284             exit code can be changed in the same 
    285             way as for compile_and_run. 
    286  
    287       run_command_ignore_output 
    288             Same as run_command, except the output 
    289             (both stdout and stderr) from the 
    290             command is ignored. 
     308            Just run an arbitrary command.  The output is checked 
     309            against `T.stdout` and `T.stderr` (unless `ignore_output` 
     310            is used), and the stdin and expected exit code can be 
     311            changed in the same way as for compile_and_run. 
    291312 
    292313      ghci_script 
     
    326347 
    327348For some examples, take a look in tests/ghc-regress/programs. 
     349 
     350= Sample output files = 
     351 
     352Normally, the sample `stdout` and `stderr` for a test T go in the 
     353files `T.stdout` and `T.stderr` respectively.  However, sometimes a 
     354test may generate different output depending on the platform, 
     355compiler, compiler version, or word-size.  For this reason the test 
     356driver looks for sample output files using this pattern: 
     357 
     358{{{ 
     359 T.stdout[-<compiler>][-<version>][-ws-<wordsize>][-<platform>] 
     360}}} 
     361 
     362Any combination of the optional extensions may be given, but they must 
     363be in the order specified.  The most specific output file that matches 
     364the current configuration will be selected; for example if the 
     365platform is `i386-unknown-mingw32` then `T.stderr-i386-unknown-mingw32` 
     366will be picked in preference to `T.stderr`. 
     367 
     368Another common example is to give different sample output for an older 
     369compiler version.  For example, the sample `stderr` for GHC 6.8.x would go in the file 
     370`T.stderr-ghc-6.8`. 
    328371 
    329372= The details =