lazyparser problems (was Re: [PATCH 0 of 2 STABLE RFC] issue2137: revlog has stale data after unbundle())

[Benoit Boissinot]

On Wed, Apr 14, 2010 at 3:39 PM, Greg Ward <greg-hg at gerg.ca> wrote:
> On Wed, Apr 14, 2010 at 3:48 AM, Matt Mackall <mpm at selenic.com> wrote:
>> Oh, almost anything that looks like a bug with lazyparser probably is.
>> lazyparser is a sad thing. It's fat and slow and overly-complicated. If
>> some day we can figure out a way to cleanly move from lazyparser at
>> start-up to on-demand non-lazy lists and dicts, that'd be great.
>
> <bluesky>
> I was pondering this a few weeks ago, when digging into performance
> problems and wondering if anything can be done about the ~1 sec hit to
> read the changelog index of our big repo.  (There's also a pretty big
> RAM hit: the index is only ~6 MB on disk, but pretty much every hg
> process accessing that repo takes >= 45 MB of memory.)  My big crazy
> idea was to not use lists or dicts at all, but a custom data structure
> that is specifically designed for revlog and only reads the data that
> is absolutely needed.  (E.g. why should "hg tip" have to read the
> whole changelog index?)  Probably something written in C.  But talk is
> cheap and working code is not, so stop me if you've heard this one
> before.  ;-)
> </bluesky>

First step would be to benchmark it. The lazyparser was created before
we added the C parser. Now it might be possible that the C parser is
always faster...

Benoit