run-tests.py refactor

[Gregory Szorc]

On 4/25/2014 6:10 AM, Martin Geisler wrote:
> Gregory Szorc <gregory.szorc at gmail.com> writes:
> 
>> On my i7-2600K mech HD w/ -l -j8:
>> 
>> Original:
>> 
>> real    6m40.558s user    31m31.405s sys     17m48.862s
> 
> I normally run with -j 20 and get a real time of 4m24s. I find that
> there's so much IO going on that I can start that many processes in
> parallel to really saturate the CPU (maybe -j 15 would be the
> same).

I created a tmpfs to isolate the impact of I/O from my mechanical HD.
My 4+4 cores were largely saturated during -j8. There were only brief
periods (maybe ~20s total) where idle CPU was below 5%. Raising it to
-j10 made the lulls go away.

Total execution time on the tmpfs was about the same as the mechanical
HD. I can only conclude that the tests are CPU bottlenecked. My
previous measurements indicate the bulk of that is inside `hg`.

I may very well write a feature that invokes the Python profiler on
all the `hg` invocations and sums the results. Could be interesting.