Changes between Version 15 and Version 16 of Building/RunningTests/Adding


Ignore:
Timestamp:
Apr 8, 2013 11:48:04 PM (2 years ago)
Author:
afarmer
Comment:

Update performance tests section to reflect current test framework functions

Legend:

Unmodified
Added
Removed
Modified
  • Building/RunningTests/Adding

    v15 v16  
    277277{{{
    278278test('perf001',
    279      [ if_wordsize(32,
    280           compiler_stats_range_field('bytes allocated', 40000000, 10)),
    281        if_wordsize(64,
    282           compiler_stats_range_field('bytes allocated', 79110184, 10))
    283      ],
    284      compile,[''])
     279     [ compiler_stats_num_field('bytes allocated', [(wordsize(32), 40000000, 10), (wordsize(64), 79110184, 10)]) ],
     280     compile, [''])
    285281}}}
    286282
     
    289285The kinds of constraint that can be used are:
    290286
    291  * '''compiler_stats_range_field(stat, baseline, deviation)'''  tests the performance of GHC, and should be used with '''compile''' or '''compile_fail''' tests.  '''stat''' is one of the following: `'bytes allocated'`, `'peak_megabytes_allocated'`, or `'max_bytes_used'`; '''baseline''' is the baseline value obtained by running the benchmark, and '''deviation''' is the percentage deviation from the baseline that the framework will allow for the test to pass.
    292 
    293  * '''stats_range_field(stat, baseline, deviation)'''  is the same, but tests the performance of the ''program'', not the compiler.  It should be used in conjunction with a '''compile_and_run''' test.
     287 * '''compiler_stats_num_field(stat, expecteds)'''  tests the performance of GHC, and should be used with '''compile''' or '''compile_fail''' tests.  '''stat''' is one of the following: `'bytes allocated'`, `'peak_megabytes_allocated'`, or `'max_bytes_used'`; '''expecteds''' is a list of triples. Each triple has the form: `(predicate, baseline, deviation)`. '''predicate''' is a boolean value indicating which triple to use. In the above example, if the machine word size is 32 bits, the first triple's '''baseline''' and '''deviation''' values will be used. If the word size is 64 bits, the second triple's values will be used. '''baseline''' is the baseline value obtained by running the benchmark, and '''deviation''' is the percentage deviation from the baseline that the framework will allow for the test to pass.
     288
     289 * '''stats_num_field(stat, expecteds)'''  is the same, but tests the performance of the ''program'', not the compiler.  It should be used in conjunction with a '''compile_and_run''' test.
    294290
    295291=== The <test-fn> field ===