[PATCH STABLE] tests: run test-check-code* last

Matt Harbison matt_harbison at yahoo.com
Tue Sep 11 23:03:18 CDT 2012


Adrian Buehlmann wrote:
> On 2012-09-11 03:31, Matt Harbison wrote:
>> Agreed, and I thought about running it first _and_ last.  But I figured
>> it runs relatively early now, and I don't recall anyone else mentioning
>> wanting it to run first
>
> I think I would appreciate if test-check* -- I constantly refuse to
> remember which one it is that I want, so I say '...check*' -- would be
> run first.

[snip (A) test-check* run followed by (B) full suite run]

> I have quite a couple of times forgotten to do (A), directly starting
> (B), leaving my room to do something else and then coming back, seeing
> that the code didn't even pass check-code (then thinking "arhg!").

Me too.  I agree that running first can be quite useful in most cases.

> I think I would even appreciate if the testsuite would terminate right
> after test-check* by default iff that fails.
>
> I think there is not much point in completing the whole testsuite (and
> wasting CPU cycles for it) anyway, if the code doesn't even pass check-code.

That I'm less sure about.  Most of the things I get bitten on are 
gratuitous whitespace, a commented out line that is too long or other 
fairly trivial stuff.  I know I'll have to change the code and run it 
again, but many of those times I'm just looking to see if some change is 
going to blow up the regular tests before I go about cleaning it up. 
Maybe quitting early is useful to people with a laptop on a battery?

As long as interactive mode doesn't quit early, I would have no 
objection though.  I've started running it as 'yes | python run-tests 
-i', walking away and sorting out the changes in an external diff program.

[snip]

> For example, I guess not that many people run the testsuite on x64
> Windows regularly. I wouldn't be surprised if I'm the only one. [1]

I usually run at least relevant pieces of the Windows test suite when 
developing, and then do the full run on Linux because it only skips 8 
tests there, but I sometimes do a full run on Windows too.

My problem is on Windows, I usually forget the --local and then a 
handful of tests fail.  I'm not sure why --local seems to be required on 
Windows, but since it seems to be, is it reasonable for run-tests.py to 
run in that mode on Windows even when not specified?  Or at least print 
a warning and/or abort?

My setup must be a bit different than yours because, because I don't 
have to set the path like you are doing- it is already there.  I 
probably did something to set that up, but I forget what.  It is just a 
straight open MinGW and then 'python run-tests.py --local', so it is 
just like a Linux run (except the --local).


Here's a small patch that illustrates the original problem by simulating 
two added tests with trivial output changes.  One test comes before 
check*, the other after.  Run it as 'python run-tests.py -i test-check* 
test-bundle.t test-rebase-collapse.t'.  If this happened to me in a 
non-contrived, final run before submission, I would accept the first 
change, quit when check* complains, fix the first test, and then rerun. 
  It's obvious the second test needs to be manually changed at the end 
if the first gets flagged, but the closer to the beginning check* is, 
the more tests don't get flagged the first time they are run.  And it 
isn't obvious the change is something that would run afoul of check* and 
benefit from a rerun.

Maybe check* should be run first (however that policy is implemented), 
and then conditionally at the end if in interactive mode and output was 
saved?  That seems better than hardcoding it unconditionally like I did.

# HG changeset patch
# Parent 3ee5d3c372fabcf57c305835dac98da78bdc1837
# User Matt Harbison <matt_harbison at yahoo.com>
# Date 1347416882 14400

diff --git a/tests/test-bundle.t b/tests/test-bundle.t
--- a/tests/test-bundle.t
+++ b/tests/test-bundle.t
@@ -600,4 +600,6 @@
    checking files
    4 files, 3 changesets, 5 total revisions

+  $ hg --config extensions.mq= strip -r tip -R part2
+
    $ cd ..
diff --git a/tests/test-rebase-collapse.t b/tests/test-rebase-collapse.t
--- a/tests/test-rebase-collapse.t
+++ b/tests/test-rebase-collapse.t
@@ -54,7 +54,6 @@
    $ hg phase --force --secret 3

    $ hg rebase --collapse --keepbranches
-  saved backup bundle to $TESTTMP/a1/.hg/strip-backup/*-backup.hg (glob)

    $ hg tglogp
    @  5:secret 'Collapsed revision


More information about the Mercurial-devel mailing list