Please read: only you can prevent test suite bloat

Matt Mackall mpm at selenic.com
Tue Oct 30 18:18:29 CDT 2012


The test suite already takes nearly 40 minutes to run on OS X, and over
an hour on Windows, so this is actually dated:

        The test suite is slow. And the test suite is slow because it is
        highly redundant. And it is highly redundant because for years
        we've been writing a completely new test for each issue that
        creates a new repo, adds a file, runs status, commits, does a
        merge, etc. 
        
        If we add a one-second test for each bug fix that shows up, very
        soon we'll have a test suite that takes an hour to run and thus
        is no longer useful to anyone.
        
        Therefore, if you want to add testing for a feature, you must
        either:
        
         *  add a short, fast doctest (where appropriate)
         *  fold your test into an appropriate existing test 
        
        When doing the latter, you should try to take advantage of work
        the test suite is already doing. For instance, if you're testing
        whether uppercase keywords work correctly, please adjust one of
        the many existing tests that uses a keyword to use an uppercase
        one. 
        
        http://mercurial.selenic.com/wiki/WritingTests#No_more_new_test_scripts.21

That's been there since last December, and I've rejected countless
patches after directing people to read it. And yet I'm seeing new
patches submitted on a nearly daily basis from subscribers to this list
that ignore this, some of them being accepted by crew members who
haven't read it either.

The last test to show up in crew looks like this:

--- a/tests/test-merge-tools.t
+++ b/tests/test-merge-tools.t
...
+  # hg update -C 1
+  $ echo "revision 4" > '"; exit 1; echo "'
+  $ hg commit -Am "revision 4"
+  adding "; exit 1; echo "
+  warning: filename contains '"', which is reserved on Windows: '"; exit 1; echo "'
+  $ hg update -C 1 > /dev/null
+  $ echo "revision 5" > '"; exit 1; echo "'
+  $ hg commit -Am "revision 5"
+  adding "; exit 1; echo "
+  warning: filename contains '"', which is reserved on Windows: '"; exit 1; echo "'
+  created new head
+  $ hg merge --config merge-tools.true.executable="true" -r 4
+  merging "; exit 1; echo "
+  0 files updated, 1 files merged, 0 files removed, 0 files unresolved
+  (branch merge, don't forget to commit)
+  $ hg update -C 1 > /dev/null

..which does exactly what the WritingTests page says not to do anymore.
There are already _32_ merges in that test script. There is absolutely
no reason to set up a new merge from scratch. Instead, take any previous
merge in the test and change a filename to have quoting issues. This
gives us the test case for -zero- additional overhead.

(Also note that this new test case is self-evidently hostile to running
on Windows.)

-- 
Mathematics is the supreme nostalgia of our time.




More information about the Mercurial-devel mailing list