Speed on Windows for big repos

[Adrian Buehlmann]

On 27.01.2008 17:43, Matt Mackall wrote:
> On Sun, 2008-01-27 at 15:58 +0100, Adrian Buehlmann wrote:
>> The relative speed of Mercurial 0.9.5 on Windows XP compared to on Linux seems 
>> to be rather weak for big repositories:
>>
>> See the Netbeans repo test runs on:
>> http://www.selenic.com/mercurial/wiki/index.cgi/RepoSamples
>>
>>    Files: 74'713
>>    Changesets: 64'888
>>    Total Revisions: 325'131
>>
>> Local "clone --pull":
>>
>>    Ubuntu 7.10 on Core 2 Duo, 2GB RAM:             260 sec (< 5 min)
>>    Windows XP SP2, 2.4 GHz Pentium 4, 1 GB RAM:  4'150 sec (> 1 h  )
>>
>> Ok, not the same hardware, but a factor of nearly 16?
>>
>> I did the Windows run using the batteries included installer (thanks to Jesse 
>> for posting the Ubuntu test data). Windows CPU load was rather low, peak memory 
>> usage far below 200 MB. Per my personal paranoia, I suspect that hg was mostly 
>> waiting for disk writes to complete.
>>
>> Why is Mercurial on Windows really that much slower than on Linux?
> 
> I suspect it is indeed I/O bound. It's typically I/O bound in the Linux
> case too!
> 
> What clone --pull is doing basically amounts to a glorified copy. We
> read the changelog, and send each index entry along with its delta. Then
> we do the same for the manifest and each file. And by "send", I mean we
> use a Python generator. It's not even multi-threaded.
> 
> So odds are the operating system is failing to do one or more of the
> following efficiently: a) write caching b) interleaving write-out I/O
> with reads c) read-ahead.
> 

Many thanks for the reply (and for your great work on Mercurial BTW!).

For reference, here is the time I measured for a local directory deep-copy on my 
Windows XP box for that Netbeans repo:

'''
 > timethis xcopy netbeans-main netbeans-xcopied /E /I /Q /Y /C /R

TimeThis :  Command Line :  xcopy netbeans-main netbeans-xcopied /E /I /Q /Y /C /R
TimeThis :    Start Time :  Sun Jan 27 18:29:37 2008

74806 File(s) copied

TimeThis :  Command Line :  xcopy netbeans-main netbeans-xcopied /E /I /Q /Y /C /R
TimeThis :    Start Time :  Sun Jan 27 18:29:37 2008
TimeThis :      End Time :  Sun Jan 27 18:37:30 2008
TimeThis :  Elapsed Time :  00:07:53.531

 > ver
Microsoft Windows XP [Version 5.1.2600]
'''

-- 
xcopy: http://www.ss64.com/nt/xcopy.html
timethis: 
http://www.microsoft.com/downloads/details.aspx?familyid=913795CD-7026-4143-AE85-1F5E096F9BE0&displaylang=en