[PATCH 8 of 8] revlog: disable lazy parsers when running under PyPy

[Matt Mackall]

On Tue, 2011-01-04 at 09:08 -0500, Greg Ward wrote:
> On Fri, Dec 24, 2010 at 11:26 PM, Augie Fackler <durin42 at gmail.com> wrote:
> > On Dec 24, 2010, at 7:35 PM, Dan Villiom Podlaski Christiansen wrote:
> >>
> >> # HG changeset patch
> >> # User Dan Villiom Podlaski Christiansen <danchr at gmail.com>
> >> # Date 1293200636 -3600
> >> # Node ID 244e1f304faac009e4e55579f6c8a35141793be7
> >> # Parent  d281ff54afa4e11534aee921102706c477c13322
> >> revlog: disable lazy parsers when running under PyPy
> >>
> >> Using the lazy parsers mean that it becomes fairly difficult to
> >> determine when files should be closed. As PyPy tends to delay garbage
> >> collection of file descriptors just too long for our taste, we risk
> >> exhausting file descriptors when opening many revlogs. With this
> >> change, Mercurial running under PyPy is able to verify the Mercurial
> >> repository itself.
> >
> > Should we (in a later patch) look at having some kind of borged LRU available for revlogs so we can keep a fixed number open with lazy parsing for environments like PyPy?
> > (musing out loud since it might be generally helpful)
> 
> Catching up after vacation, this caught my eye.  I fixed a bug in the
> lazy parser several months ago, and at the time Matt made an offhand
> remark that I recall as "the lazy parser isn't as useful as it used to
> be and it's a source of bugs".  My own observation is that the lazy
> parser also makes revlog.py a lot more complex and difficult to
> follow.

The lazy index does two things quickly on large (greater than 1MB or 16k
revisions):

- looking up tip
- looking up a revision by number (not by hash)

Everything else (including looking up working directory parents!) it
does more slowly.

A quick test with the perf extension gives the following numbers on a
repo with 227k revisions :

                lazy          no lazy
just tip:       .008s         .551s
full index:     .769s         .551s

Given that 'tip' is now a more or less deprecated concept, it's probaby
a net win to kill the lazy index, as we'll increase performance for most
other operations by ~30%. I'll take a stab at it.

-- 
Mathematics is the supreme nostalgia of our time.