MQ performance on large repo

Benoit Boissinot bboissin at gmail.com
Thu Mar 4 02:50:00 CST 2010


On Thu, Mar 4, 2010 at 4:42 AM, Greg Ward <greg at gerg.ca> wrote:
> OK, I think we've picked the low-hanging fruit of my "qrefresh"
> profile.  It's now down from ~9 sec to ~5.8 sec in an fncache repo.
> (Actually I have a clone of that repo in the older 'store' format, on
> the assumption that avoiding fncache would speed things up.
> Performance is very similar in the two repos.)
>
> So what's left?  First, here are the numbers again:
>
>  qpop       3.26  3.26  3.33  3.41  3.32
>  qpush      2.22  2.18  2.18  2.19  2.18
>  qrefresh   5.77  5.83  5.93  5.74  5.76
>  strip      1.99  2.02  1.98  2.00  2.00
>
> Each operation was repeated five times, and I'm reporting the runtime
> of all five runs in seconds.  The patch queue had 1 small patch which
> modifies 1 file.
>
> So here is the profile from that qrefresh operation:
>
> """
>   CallCount    Recursive    Total(ms)   Inline(ms) module:lineno(function)
>         347            0      0.7987      0.7987   <zlib.decompress>
>           3            0      2.0807      0.7816   mercurial.revlog:1313(strip)
>      320858            0      1.4716      0.7254   mercurial.revlog:269(__getitem__)

> If I'm interpreting this correctly, I think it means that the bulk of
> the runtime is from loading the revlog index for 00changelog.i and
> 00manifest.i into memory.  (We see 2*N calls to revlog.__getitem__()
> and __iter__() because 00changelog.i and 00manifest.i both have N
> revisions.)

What if you disable lazyindex? The C parser might be faster anyway.

cheers,

Benoit


More information about the Mercurial-devel mailing list