[PATCH 8 of 8] revlog: disable lazy parsers when running under PyPy

Adrian Buehlmann adrian at cadifra.com
Fri Dec 24 19:12:03 CST 2010


On 2010-12-25 01:35, Dan Villiom Podlaski Christiansen wrote:
> # HG changeset patch
> # User Dan Villiom Podlaski Christiansen <danchr at gmail.com>
> # Date 1293200636 -3600
> # Node ID 244e1f304faac009e4e55579f6c8a35141793be7
> # Parent  d281ff54afa4e11534aee921102706c477c13322
> revlog: disable lazy parsers when running under PyPy
> 
> Using the lazy parsers mean that it becomes fairly difficult to
> determine when files should be closed. As PyPy tends to delay garbage
> collection of file descriptors just too long for our taste, we risk
> exhausting file descriptors when opening many revlogs. With this
> change, Mercurial running under PyPy is able to verify the Mercurial
> repository itself.
> 
> diff --git a/mercurial/revlog.py b/mercurial/revlog.py
> --- a/mercurial/revlog.py
> +++ b/mercurial/revlog.py
> @@ -455,10 +455,12 @@ class revlog(object):
>          if shallowroot:
>              v |= REVLOGSHALLOW
>  
> +        nonlazy = ("nonlazy" in getattr(self.opener, 'options', {})
> +                   or util.ispypy())
>          i = ''
>          try:
>              f = self.opener(self.indexfile)
> -            if "nonlazy" in getattr(self.opener, 'options', {}):
> +            if nonlazy:
>                  i = f.read()
>              else:
>                  i = f.read(_prereadsize)
> @@ -495,7 +497,7 @@ class revlog(object):
>              if not self._chunkcache:
>                  self._chunkclear()
>  
> -            if "nonlazy" in getattr(self.opener, 'options', {}):
> +            if nonlazy:
>                  f.close()
>  
>          # add the magic null revision at -1 (if it hasn't been done already)
> diff --git a/mercurial/util.py b/mercurial/util.py
> --- a/mercurial/util.py
> +++ b/mercurial/util.py
> @@ -1524,3 +1524,8 @@ def parsebool(s):
>      If s is not a valid boolean, returns None.
>      """
>      return _booleans.get(s.lower(), None)
> +
> +def ispypy():
> +    """Return True when running under the PyPy VM."""
> +    # The PyPy developers recommend this technique for detecting it.
> +    return '__pypy__' in sys.builtin_module_names

Just slightly off topic comment, but the lazy parser is causing troubles
on Windows as well since it keeps files open, causing various headaches
for Windows, especially when AV scanners are around.

See http://mercurial.selenic.com/wiki/UnlinkingFilesOnWindows

(Did you know?: On Windows, a process P, which has only read access
rights for a file F, is able to open F in such a way that no other
process Q can open F for reading again for as long as P holds F open. No
joke.)


More information about the Mercurial-devel mailing list