Changes between Version 12 and Version 13 of Building/RunningTests


Ignore:
Timestamp:
Feb 6, 2008 1:19:47 PM (7 years ago)
Author:
simonmar
Comment:

update

Legend:

Unmodified
Added
Removed
Modified
  • Building/RunningTests

    v12 v13  
    8484        optc                    -- -O -fvia-C
    8585        optasm                  -- -O -fasm
    86         prof                    -- -O -prof -auto-all
     86        profc                   -- -O -prof -auto-all -fvia-C
    8787        profasm                 -- -O -prof -auto-all -fasm
    88         unreg                   -- -unreg
    8988        ghci                    -- (run only, not compile) run test under GHCi
    9089        extcore                 -- -fext-core
    9190        optextcore              -- -O -fext-core
    92         threaded                -- -threaded
     91        threaded1               -- -threaded -debug
     92        threaded2               -- -threaded -O, and +RTS -N2 at run-time
    9393        hpc                     -- -fhpc
    9494}}}
    9595certain ways are enabled automatically if the GHC build in the local
    96 tree supports them.  Ways that are enabled this way are optasm, prof,
    97 profasm, unreg, threaded, and ghci.
     96tree supports them.  Ways that are enabled this way are optasm, profc,
     97profasm, threaded1, threaded2, and ghci.
    9898
    9999= Updating tests when the output changes =
     
    103103new output like so:
    104104{{{
    105         make accept TESTS=<test-name>
     105        make accept TEST=<test-name>
    106106}}}
    107107where <test-name> is the name of the test.  In a directory which
    108108contains a single test, or if you want to update *all* the tests in
    109 the current directory, just omit the 'TESTS=<test-name>' part.
     109the current directory, just omit the 'TEST=<test-name>' part.
    110110
    111111= Adding a new test =
     
    141141
    142142 2. Having found a suitable place for the test, give the test a name.
    143     Follow the convention for the directory in which you place the
     143    For regression tests, we often just name the test after the bug number (e.g. 2047).
     144    Alternatively, follow the convention for the directory in which you place the
    144145    test: for example, in typecheck/should_compile, tests are named
    145146    tc001, tc002, and so on.  Suppose you name your test T, then
     
    172173 1. Edit all.T in the relevant directory and add a line for the test.  The line is always of the form
    173174{{{
    174       test(<name>, <opt-fn>, <test-fn>, <args>)
     175      test(<name>, <setup>, <test-fn>, <args>)
    175176}}}
    176177    where
     
    178179    ''<name>'' is the name of the test, in quotes (' or ").
    179180
    180     ''<opt-fn>''  is a function (i.e. any callable object in Python)
     181    ''<setup>''  is a function (i.e. any callable object in Python)
    181182    which allows the options for this test to be changed.
    182     There are several pre-defined functions which can be
     183    There are many pre-defined functions which can be
    183184    used in this field:
    184185
     
    187188        '''skip'''                  skip this test
    188189
     190        '''skip_if_no_ghci'''       skip unless GHCi is available
     191
     192        '''skip_if_fast'''          skip if "fast" is enabled
     193
     194        '''skip_if_platform(platform)'''  skip if we're on the named platform
     195
     196        '''skip_if_tag(tag)'''      skip if the compiler has a given tag
     197
     198        '''skip_unless_tag(tag)'''  skip unless the compiler has a given tag
     199
    189200        '''omit_ways(ways)'''       skip this test for certain ways
    190201
    191202        '''only_ways(ways)'''       do this test certain ways only
    192203
     204        '''extra_ways(ways)'''      add some ways which would normally be disabled
     205
    193206        '''omit_compiler_types(compilers)'''                           skip this test for certain compilers
    194207
     
    205218        '''set_stdin(file)'''       use a different file for stdin
    206219
     220        '''no_stdin'''              use no stdin at all (otherwise use `/dev/null`)
     221
    207222        '''exit_code(n)'''          expect an exit code of 'n' from the prog
    208223
     
    213228        '''no_clean'''              don't clean up after this test
    214229
    215     These functions should normally not be used; instead, use the `expect_broken*`
     230        '''extra_clean(files)'''    extra files to clean after the test has completed
     231
     232        '''reqlib(P)'''             requires package P
     233
     234        '''req_profiling'''         requires profiling
     235
     236        '''ignore_output'''         don't try to compare output
     237
     238        '''alone'''                 don't run this test in parallel with anything else
     239
     240        '''literate'''              look for a `.lhs` file instead of a `.hs` file
     241
     242        '''c_src'''                 look for a `.c` file
     243
     244        '''cmd_prefix(string)'''    prefix this string to the command when run
     245
     246        '''normalise_slashes'''     convert backslashes to forward slashes before comparing the output
     247
     248    To use more than one modifier on a test, just put them in a list.
     249    For example, to expect an exit code of 3 and omit way 'opt', we could use
     250{{{
     251      [ omit_ways(['opt']), exit_code(3) ]
     252}}}
     253      as the `<setup>` argument.
     254
     255    The following should normally not be used; instead, use the `expect_broken*`
    216256    functions above so that the problem doesn't get forgotten about, and when we
    217257    come back to look at the test later we know whether current behaviour is why
     
    225265
    226266        '''expect_fail_if_compiler_type(compiler)'''   expect failure from a certain compiler
    227 
    228       You can compose two of these functions together by
    229       saying compose(f,g).  For example, to expect an exit
    230       code of 3 and omit way 'opt', we could use
    231 {{{
    232       compose(omit_ways(['opt']), exit_code(3))
    233 }}}
    234       as the <opt-fn> argument.  Calls to compose() can of
    235       course be nested, but it is simpler to use the composes
    236       function which takes a list of functions to be composed, e.g.:
    237 {{{
    238       composes([omit_ways(['opt']), exit_code(3), expect_broken(123)])
    239 }}}
    240267
    241268    ''<test-fn>''
     
    279306
    280307      run_command
    281             Just run an arbitrary command.  The
    282             output is checked against T.stdout and
    283             T.stderr, and the stdin and expected
    284             exit code can be changed in the same
    285             way as for compile_and_run.
    286 
    287       run_command_ignore_output
    288             Same as run_command, except the output
    289             (both stdout and stderr) from the
    290             command is ignored.
     308            Just run an arbitrary command.  The output is checked
     309            against `T.stdout` and `T.stderr` (unless `ignore_output`
     310            is used), and the stdin and expected exit code can be
     311            changed in the same way as for compile_and_run.
    291312
    292313      ghci_script
     
    326347
    327348For some examples, take a look in tests/ghc-regress/programs.
     349
     350= Sample output files =
     351
     352Normally, the sample `stdout` and `stderr` for a test T go in the
     353files `T.stdout` and `T.stderr` respectively.  However, sometimes a
     354test may generate different output depending on the platform,
     355compiler, compiler version, or word-size.  For this reason the test
     356driver looks for sample output files using this pattern:
     357
     358{{{
     359 T.stdout[-<compiler>][-<version>][-ws-<wordsize>][-<platform>]
     360}}}
     361
     362Any combination of the optional extensions may be given, but they must
     363be in the order specified.  The most specific output file that matches
     364the current configuration will be selected; for example if the
     365platform is `i386-unknown-mingw32` then `T.stderr-i386-unknown-mingw32`
     366will be picked in preference to `T.stderr`.
     367
     368Another common example is to give different sample output for an older
     369compiler version.  For example, the sample `stderr` for GHC 6.8.x would go in the file
     370`T.stderr-ghc-6.8`.
    328371
    329372= The details =