[PATCH 8 of 8] revlog: disable lazy parsers when running under PyPy

[Augie Fackler]

On Dec 24, 2010, at 7:35 PM, Dan Villiom Podlaski Christiansen wrote:
> 
> # HG changeset patch
> # User Dan Villiom Podlaski Christiansen <danchr at gmail.com>
> # Date 1293200636 -3600
> # Node ID 244e1f304faac009e4e55579f6c8a35141793be7
> # Parent  d281ff54afa4e11534aee921102706c477c13322
> revlog: disable lazy parsers when running under PyPy
> 
> Using the lazy parsers mean that it becomes fairly difficult to
> determine when files should be closed. As PyPy tends to delay garbage
> collection of file descriptors just too long for our taste, we risk
> exhausting file descriptors when opening many revlogs. With this
> change, Mercurial running under PyPy is able to verify the Mercurial
> repository itself.

Should we (in a later patch) look at having some kind of borged LRU available for revlogs so we can keep a fixed number open with lazy parsing for environments like PyPy?
(musing out loud since it might be generally helpful)

> 
> diff --git a/mercurial/revlog.py b/mercurial/revlog.py
> --- a/mercurial/revlog.py
> +++ b/mercurial/revlog.py
> @@ -455,10 +455,12 @@ class revlog(object):
>         if shallowroot:
>             v |= REVLOGSHALLOW
> 
> +        nonlazy = ("nonlazy" in getattr(self.opener, 'options', {})
> +                   or util.ispypy())
>         i = ''
>         try:
>             f = self.opener(self.indexfile)
> -            if "nonlazy" in getattr(self.opener, 'options', {}):
> +            if nonlazy:
>                 i = f.read()
>             else:
>                 i = f.read(_prereadsize)
> @@ -495,7 +497,7 @@ class revlog(object):
>             if not self._chunkcache:
>                 self._chunkclear()
> 
> -            if "nonlazy" in getattr(self.opener, 'options', {}):
> +            if nonlazy:
>                 f.close()
> 
>         # add the magic null revision at -1 (if it hasn't been done already)
> diff --git a/mercurial/util.py b/mercurial/util.py
> --- a/mercurial/util.py
> +++ b/mercurial/util.py
> @@ -1524,3 +1524,8 @@ def parsebool(s):
>     If s is not a valid boolean, returns None.
>     """
>     return _booleans.get(s.lower(), None)
> +
> +def ispypy():
> +    """Return True when running under the PyPy VM."""
> +    # The PyPy developers recommend this technique for detecting it.
> +    return '__pypy__' in sys.builtin_module_names
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel at selenic.com
> http://selenic.com/mailman/listinfo/mercurial-devel