[PATCH 0 of 3] Fast implemenation of parseindex in C

[Bernhard Leiner]

(Moving this discussion to the devel mailing list)

2008/10/6 Matt Mackall <mpm at selenic.com>:
>
> On Sun, 2008-09-21 at 22:24 +0200, Bernhard Leiner wrote:
>> Ok, here is the perf extension with a new perfparseindex test.
>>
>> parseindex implementation in C:
>> ! wall 0.032150 comb 0.030000 user 0.020000 sys 0.010000 (best of 100)
>> vs. hg-stable:
>> ! wall 0.065412 comb 0.060000 user 0.050000 sys 0.010000 (best of 100)
>
> (sorry, digging through mail backlog)
>
> Hmm, that's not as good as I hoped, but a good start! I was hoping to
> see something like 5x when we got around to doing this. Perhaps there
> are some more optimization opportunities.. In particular, Py_BuildObject
> is quite helpful.

Py_BuildObject? You are probably thinking of Py_BuildValue, don't you?

I just did a quick test with using Py_BuildValue to create the entries for
the index list. Unfortunately this does not speed things up but even slows
them down a bit. I will make some more experiments in the next days,
maybe I can find a faster way.

> It'd also be interesting to know what happens for very large indexes.

I will have a look

> I'd rather like to do away with the lazy parsing as it's quite slow, but
> it's also quite useful in the 20k-1M entry range. If we got to be about
> an order of magnitude faster, lazy parsing would be much less important.

Getting rid of lazy parsing would also make further optimizations easier.
(I have written a C implementation of revlog.parents and revlog._match
but both of them do not work with a lazy index.)


-- 
Bernhard Leiner           http://bernh.net