xrevlog: experimental reimplementation of revlog in C

Greg Ward greg at gerg.ca
Tue Nov 16 09:08:04 CST 2010


On Tue, Nov 16, 2010 at 1:45 AM, Matt Mackall <mpm at selenic.com> wrote:
>> Performance testing involves four tasks:
>>
>> 1) openclose: open the revlog and close (or discard) it
>
> Pretty meaningless comparison, as we generally read most of the revlog
> in this step.

Yes: this task is horribly unfair to the Mercurial API.  I just threw
it in out of curiosity.  And it has nothing to do with real-world
worflow; readtip() is the only task that does.

>> Python/Mercurial, medium changelog (12k revs):
>>   openclose     min   1496.8 ms
>>   readindex     min   6146.1 ms
>>   readtip       min   1521.0 ms
>>   readdata      min   3941.5 ms
>
> That's weird. readdata implies reading the whole index, so why is it
> faster?

Because I don't read the whole index.  At least with external data,
you only need to read/parse the required records, and that's exactly
what I do.  Granted, this is very "seeky" -- I just hop all over that
mmap()'ed array without regard for the consequences.  With inline
data, I do have to read the whole index -- no other way to know how
many revs are in the revlog.

>>   * I'm still not handling the node ID -> revnum mapping, which I
>> think will force me to read the whole index into memory much more
>> often -- so I'll lose some advantage there, I think
>
> Never more than once per open?

Definitely.  Possibly less than once per open: xrevlog uses the usual
Mercurial approach of delaying stuff as long as it possibly can, so I
would try to delay building the node->revnum mapping until it's
needed.  As soon as someone queries by node ID, we have to read the
whole index and build that hash, but until then, avoid it!

Note that my tasks are deliberately coded to include opening the
revlog.  That is meant to make the Python API look bad, because it
does so much work in the open.  ;-)  So if I build the node mapping up
front, xrevlog will pay that penalty 50 times for each run of the
performance test.  Only seems fair.

Greg


More information about the Mercurial-devel mailing list