[PATCH 6 of 6] run-tests: introduce --random for running tests in random error

[Mads Kiilerich]

On 04/17/2013 05:14 AM, Augie Fackler wrote:
> On Apr 16, 2013, at 10:11 PM, Mads Kiilerich <mads at kiilerich.com> wrote:
>
>> # HG changeset patch
>> # User Mads Kiilerich <madski at unity3d.com>
>> # Date 1366162894 -7200
>> # Node ID 0dbf64735055e8dc2118ddea7eacb76118045235
>> # Parent  5dbf61fe581870e2c5b3a50c39dcc5f90cc161dc
>> run-tests: introduce --random for running tests in random error
>>
>> This is convenient when investigating failures one by one to avoid running the
>> same fixed tests first every time.
> Huh? I'm not quite sure I understand the merit of this one.

I "needed" this when testing a change that caused lots of tests to fail. 
I stopped run-tests whenever a test failed and tweaked code or test 
until it passed. Then I started run-tests again waiting for it to fail 
next time ... and I got bored of watching it testing the 'a' tests first 
every time. Using a random order worked better for me. Also because it 
often is convenient to attack hard problems from different angles.

My use case could also benefit from run-tests creating .err files up 
front for all tests it would consider running ... and remove them when 
tests passed. Then I could resume with -r.

But yeah, not a big and obvious feature. I will keep it in my queue and 
use it when I need it ;-)

/Mads
moving on to solving the problem of having seen more endings of TV films 
than beginnings and having heard the beginnings of albums more often 
than the ends


>
>> diff --git a/tests/run-tests.py b/tests/run-tests.py
>> --- a/tests/run-tests.py
>> +++ b/tests/run-tests.py
>> @@ -198,6 +198,8 @@ def parseargs():
>>          help="enable Py3k warnings on Python 2.6+")
>>      parser.add_option('--extra-config-opt', action="append",
>>                        help='set the given config opt in the test hgrc')
>> +    parser.add_option('--random', action="store_true",
>> +                      help='run tests in random order')
>>
>>      for option, (envvar, default) in defaults.items():
>>          defaults[option] = type(default)(os.environ.get(envvar, default))
>> @@ -1248,6 +1250,9 @@ def main():
>>
>>      tests = args
>>
>> +    if options.random:
>> +        random.shuffle(tests)
>> +
>>      # Reset some environment variables to well-known values so that
>>      # the tests produce repeatable output.
>>      os.environ['LANG'] = os.environ['LC_ALL'] = os.environ['LANGUAGE'] = 'C'
>> _______________________________________________
>> Mercurial-devel mailing list
>> Mercurial-devel at selenic.com
>> http://selenic.com/mailman/listinfo/mercurial-devel