[PATCH 2 of 2] hgext: add a new packed repository extension, packrepo

Bryan O'Sullivan bos at serpentine.com
Wed Aug 15 18:18:35 CDT 2012

On Fri, Jul 13, 2012 at 1:19 PM, Matt Mackall <mpm at selenic.com> wrote:

> I don't think we've fully understood the underlying issues here.

It seems you're right.

> My working theory has been that we're thrashing the journal.

That was mine, too.

Under such a theory, I would expect the following three tests to yield
> very similar times:
>  cp -a /ramfs/bare-repo /home/repo  (baseline)
>  hg clone -U /ramfs/bare-repo /home/repo
>  hg serve -R /ramfs/bare-repo; hg clone http://localhost:8000 /home/repo
> For the Mozilla-central repo, the first two come in at about 1m, with cp
> being marginally faster. But the hg serve case is coming in at over 2m
> without any sign of a CPU bottleneck.

Are you doing a sync afterwards?

On my machine:

   - "cp -a" takes 8 seconds
   - "tar | tar" takes about 4.5
   - "tar | nc; nc | tar" takes about 5.5 (just making sure that the
   network stack isn't at fault)
   - "hg serve; hg clone --uncompressed -U" takes ... 35 seconds !?

So that's rather interesting. It prompted me to dig a little deeper.

First observation: the files streamed are presented in a crazy random
order, which surprised me. The culprit: fncache iterating over the elements
of a set. http://selenic.com/repo/hg/file/c6f88e7f95b7/mercurial/store.py

Just fixing this improved the performance of uncompressed clone to 25
seconds, which is rather nice for a one-line change. The digging continues.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://selenic.com/pipermail/mercurial-devel/attachments/20120815/e477946a/attachment.html>

More information about the Mercurial-devel mailing list