[PATCH 3 of 3] use per-directory clustered stat calls even in cases where known tree is walked
Benoit Boissinot
bboissin at gmail.com
Wed Oct 1 04:45:14 CDT 2008
On Wed, Oct 01, 2008 at 11:21:53AM +0200, Benoit Boissinot wrote:
> On Wed, Oct 01, 2008 at 10:51:34AM +0200, Christian Boos wrote:
> > Benoit Boissinot wrote:
> >> On Tue, Sep 30, 2008 at 10:48:29PM -0400, Petr Kodl wrote:
> >> ...
> >> I was thinking something more like this, could you test on windows?
> >> (the testsuite runs ok on linux)
> >> - some doc needs be added to statfiles()
> >> - the correct exception (ENOENT?) should be catched when listdir fails
> >>
> >> diff --git a/mercurial/dirstate.py b/mercurial/dirstate.p
> > (snip)
> >
> > Just a quick note to mention that the patch seemed to works fine - at
> > first.
>
> Ok I know what went wrong.
>
> Please change:
> ls = dict([(f.lower(), st) for
> f, st in osutil.listdir(dir, stat=True)])
> except:
>
> to (listdir returns a triplet, not a couple):
> ls = dict([(f.lower(), st) for
> f, kind, st in osutil.listdir(dir, stat=True)])
> except OSError:
>
> I really shouldn't have catched all errors.
There was another problem (it should remove the '/' when splitting, maybe I could
add strutil.rsplit()).
The following is better tested (I tried the same function minus the .lower() calls
on linux):
diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -532,17 +532,13 @@
results[nf] = None
# step 3: report unseen items in the dmap hash
- visit = [f for f in dmap if f not in results and match(f)]
- for nf in util.sort(visit):
- results[nf] = None
- try:
- st = lstat(join(nf))
+ visit = util.sort([f for f in dmap if f not in results and match(f)])
+ for nf, st in zip(visit, util.statfiles([join(f) for f in visit])):
+ if st is not None:
kind = getkind(st.st_mode)
- if kind == regkind or kind == lnkkind:
- results[nf] = st
- except OSError, inst:
- if inst.errno not in (errno.ENOENT, errno.ENOTDIR):
- raise
+ if kind != regkind and kind != lnkkind:
+ st = None
+ results[nf] = st
del results['.hg']
return results
diff --git a/mercurial/util.py b/mercurial/util.py
--- a/mercurial/util.py
+++ b/mercurial/util.py
@@ -1162,6 +1162,25 @@
except NameError:
pass
+ def statfiles(files):
+ dircache = {}
+ for fn in files:
+ pos = fn.rfind('/')
+ if pos != -1:
+ dir, base = fn[:pos].lower(), fn[pos+1:].lower()
+ else:
+ dir, base = '', fn.lower()
+ try:
+ ls = dircache[dir]
+ except KeyError:
+ try:
+ ls = dict([(f.lower(), st) for
+ f, kind, st in osutil.listdir(dir, stat=True)])
+ except OSError:
+ ls = {}
+ dircache[dir] = ls
+ yield ls.get(base, None)
+
try:
# override functions with win32 versions if possible
from util_win32 import *
@@ -1334,6 +1353,15 @@
def set_signal_handler():
pass
+
+ def statfiles(files):
+ for fn in files:
+ try:
+ yield os.lstat(fn)
+ except OSError, inst:
+ if inst.errno not in (errno.ENOENT, errno.ENOTDIR):
+ raise
+ yield None
def find_exe(name, default=None):
'''find path of an executable.
--
:wq
More information about the Mercurial-devel
mailing list