[PATCH 0 of 4 V2] improve run-tests for simpler output writing for windows

Simon Heimberg simohe at besonet.ch
Wed Jul 17 20:34:54 CDT 2013

[1] and [3] have been reworked (Info instead of info, new doctest)
[2] did not change
[4] is new, failing on ANY mismatch by default (with proper output), allow to 
    enable warn only mode

@mpm: I hope [4] is closer to what you think about not "going soft on crime".

After this changes, we have more and simpler possibilities to write test output 
matching on windows. We can use buildbot (which could maybe report state "warning") 
or do it partially automated at a local machine.

The current situation to write output with paths matching on windows is 
troublesome. Most users can not run tests on windows. So they can not test 
themselves if the output is as expected.
When in doubt about a line they can append (glob) to it. Buildbot will show a 
Info line about it. Currently there are four of them (1). When they are not 
fixed, the quality of the test suite drops. (Is this also soft on crime?)
The user can also not write (glob). This will result in a failed test on 
buildbot. (This is not liked, but fixed soon.)
There are some check-code rules which warn about some cases where (glob) 
is necessary. But there are many missing. And it is difficult to write new 
ones. There are messages which are written with local or portable paths 
depending on whatever.
Example: "pushing to $TESTTMP/t" and "pushing to $TESTTMP/issue1852c"
(The 2nd is currently wrong and reported on buildbot.)
And we also have messages with inconsistent path formating.
Example: moving a\b to c/d
It is easy to write a wrong rule. (Currently there is one, telling that 
pushing to $TESTTMP/anypath always needs glob, which is wrong. But it looked
correct when I wrote it.)

This patches unify the handling of missing and unnecessary glob. With the last 
patch, selecting between warnings and failures is possible. The default is 
to fail tests, which is on the safe side.
Warning mode was planned for buildbot. But not sure if we want this. (Depends 
on how soon we like to fix this failures. Should we have a new final result 
"warnings"? How is this done for the failures in (2)?) If not we could get rid 
of this mode.
And the patches do write the output as it should look like (with or without glob). This simplifies and speeds up writing correct output. 

More thoughts about paths on windows, also in tests, are in (3)

(1) http://hgbuildbot.kublai.com/builders/Windows%202008%20R2%20hg%20tests/builds/554/steps/run-tests.py%20%28python2.6%29/logs/stdio
(2) http://hgbuildbot.kublai.com/builders/Windows%202008%20R2%20hg%20tests/builds/551/steps/run-tests.py%20%28python2.6%29/logs/warnings%20%283%29
(3) http://mercurial.selenic.com/wiki/PathSeparator#Tests

More information about the Mercurial-devel mailing list