Windows test suite timeouts

[Matt Mackall]

On Sat, 2013-01-05 at 21:35 -0500, Matt Harbison wrote:
> Mads Kiilerich wrote:
> > Dirkjan Ochtman wrote, On 01/05/2013 10:40 AM:
> >> On Fri, Jan 4, 2013 at 4:06 AM, Matt Harbison
> >> <matt_harbison at yahoo.com> wrote:
> >>> Every so often, I've noticed that some tests on Windows will take a
> >>> really
> >>> long time, and then timeout:
> >>>
> >>> $ python run-tests.py -i test-largefiles.t
> >>>
> >>> ERROR: c:\Users\Matt\Projects\hg\tests\test-largefiles.t timed out
> >>> t
> >>> Failed test-largefiles.t: timed out
> >>> # Ran 1 tests, 0 skipped, 1 failed.
> >> Actually, on Gentoo, test-largefiles.t and test-mq.t have been timing
> >> out for a bunch of users, so I'm guessing that test's problems aren't
> >> only on Windows.
> >
> > These tests _are_ big and will reach the timeout limit on slow or
> > overloaded hardware. The timeout had to be increased on some of the
> > buildbots.
> >
> > In both cases: Are you sure the tests really are hanging, or are they
> > just too slow?
> >
> > /Mads
> 
> It looks like that was the problem:
> 
>    $ time python run-tests.py -t 600 test-largefiles.t
>    .
>    # Ran 1 tests, 0 skipped, 0 failed.
> 
>    real    3m58.038s
>    user    0m0.000s
>    sys     0m0.062s
> 
> I'm really surprised it is that high above the 3 minute default timeout, 
> since it works so consistently until it fails consistently.

Perhaps you could time 20 runs in a row from a cold boot to look for a
trend.

> But this pretty much tracks with what I've seen- happens after a long 
> uptime, clears on reboot, and ran 3 times in a row with the raised 
> timeout (and both of your patches applied).  I'm not sure why it works 
> in a Linux virtual box on the same machine, or why the full .t.err is 
> generated, but at least we got to the bottom of it.

Have you looked at how long it takes to run this test on Linux?

$ time ./run-tests.py -l test-largefiles.t
.
# Ran 1 tests, 0 skipped, 0 failed.

real	0m32.142s
user	0m19.401s
sys	0m7.680s

That's on a virtual machine on the same box that serves
mercurial.selenic.com, which is under a steady load of mail and web
traffic. It's running on a $400 machine I built in 2008:

 model name	: Intel(R) Core(TM)2 Duo CPU     E7300  @ 2.66GHz

That's the slowest thing I've got convenient access to. If I had my
Raspberry Pi plugged in, it would probably also smoke Windows.

People assume that Windows and OS X and Linux are roughly comparable in
performance. It just ain't so, folks. Linux absolutely murders the other
two on fork/exec, syscall, and filesystem lookup intensive benchmarks,
which is what our test-suite amounts to.

-- 
Mathematics is the supreme nostalgia of our time.