Speed on Windows for big repos

[Matt Mackall]

On Sun, 2008-01-27 at 15:58 +0100, Adrian Buehlmann wrote:
> The relative speed of Mercurial 0.9.5 on Windows XP compared to on Linux seems 
> to be rather weak for big repositories:
> 
> See the Netbeans repo test runs on:
> http://www.selenic.com/mercurial/wiki/index.cgi/RepoSamples
> 
>    Files: 74'713
>    Changesets: 64'888
>    Total Revisions: 325'131
> 
> Local "clone --pull":
> 
>    Ubuntu 7.10 on Core 2 Duo, 2GB RAM:             260 sec (< 5 min)
>    Windows XP SP2, 2.4 GHz Pentium 4, 1 GB RAM:  4'150 sec (> 1 h  )
> 
> Ok, not the same hardware, but a factor of nearly 16?
> 
> I did the Windows run using the batteries included installer (thanks to Jesse 
> for posting the Ubuntu test data). Windows CPU load was rather low, peak memory 
> usage far below 200 MB. Per my personal paranoia, I suspect that hg was mostly 
> waiting for disk writes to complete.
>
> Why is Mercurial on Windows really that much slower than on Linux?

I suspect it is indeed I/O bound. It's typically I/O bound in the Linux
case too!

What clone --pull is doing basically amounts to a glorified copy. We
read the changelog, and send each index entry along with its delta. Then
we do the same for the manifest and each file. And by "send", I mean we
use a Python generator. It's not even multi-threaded.

So odds are the operating system is failing to do one or more of the
following efficiently: a) write caching b) interleaving write-out I/O
with reads c) read-ahead.

-- 
Mathematics is the supreme nostalgia of our time.