[PATCH STABLE V3] localrepo: discard objects in _filecache at transaction failure (issue4876)
FUJIWARA Katsunori
foozy at lares.dti.ne.jp
Sat Oct 24 10:04:16 UTC 2015
# HG changeset patch
# User FUJIWARA Katsunori <foozy at lares.dti.ne.jp>
# Date 1445680737 -32400
# Sat Oct 24 18:58:57 2015 +0900
# Branch stable
# Node ID b0408b0f349906cca7d5148bbfc57460f5e1865e
# Parent 39dbf495880b8a439d912091109427d27a7e616a
localrepo: discard objects in _filecache at transaction failure (issue4876)
'repo.invalidate()' deletes 'filecache'-ed properties by
'filecache.__delete__()' below via 'delattr(unfiltered, k)'. But
cached objects are still kept in 'repo._filecache'.
def __delete__(self, obj):
try:
del obj.__dict__[self.name]
except KeyError:
raise AttributeError(self.name)
If 'repo' object is reused even after failure of command execution,
referring 'filecache'-ed property may reuse one kept in
'repo._filecache', even if reloading from a file is expected.
Executing command sequence on command server is a typical case of this
situation (5c0f5db65c6b also tried to fix this issue). For example:
1. start a command execution
2. 'changelog.delayupdate()' is invoked in a transaction scope
This replaces own 'opener' by '_divertopener()' for additional
accessing to '00changelog.i.a' (aka "pending file").
3. transaction is aborted, and command (1) execution is ended
After 'repo.invalidate()' at releasing store lock, changelog
object above (= 'opener' of it is still replaced) is deleted from
'repo.__dict__', but still kept in 'repo._filecache'.
4. start next command execution with same 'repo'
5. referring 'repo.changelog' may reuse changelog object kept in
'repo._filecache' according to timestamp of '00changelog.i'
'00changelog.i' is truncated at transaction failure (even though
this truncation is unintentional one, as described later), and
'st_mtime' of it is changed. But 'st_mtime' doesn't have enough
resolution to always detect this truncation, and invalid
changelog object kept in 'repo._filecache' is reused
occasionally.
Then, "No such file or directory" error occurs for
'00changelog.i.a', which is already removed at (3).
This patch discards objects in '_filecache' other than dirstate at
transaction failure.
Changes in 'invalidate()' can't be simplified by 'self._filecache =
{}', because 'invalidate()' should keep dirstate in 'self._filecache'
'repo.invalidate()' at "hg qpush" failure is removed in this patch,
because now it is redundant.
This patch doesn't make 'repo.invalidate()' always discard objects in
'_filecache', because 'repo.invalidate()' is invoked also at unlocking
store lock.
- "always discard objects in filecache at unlocking" may cause
serious performance problem for subsequent procedures at normal
execution
- but it is impossible to "discard objects in filecache at unlocking
only at failure", because 'releasefn' of lock can't know whether a
lock scope is terminated normally or not
BTW, using "with" statement described in PEP343 for lock may
resolve this ?
After this patch, truncation of '00changelog.i' still occurs at
transaction failure, even though newly added revisions exist only in
'00changelog.i.a' and size of '00changelog.i' isn't changed by this
truncation.
Updating 'st_mtime' of '00changelog.i' implied by this redundant
truncation also affects cache behavior as described above.
This will be fixed by dropping '00changelog.i' at aborting from the
list of files to be truncated in transaction.
diff --git a/hgext/mq.py b/hgext/mq.py
--- a/hgext/mq.py
+++ b/hgext/mq.py
@@ -847,7 +847,6 @@ class queue(object):
try:
tr.abort()
finally:
- repo.invalidate()
self.invalidate()
raise
finally:
diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -1014,6 +1014,8 @@ class localrepository(object):
# out) in this transaction
repo.vfs.rename('journal.dirstate', 'dirstate')
+ repo.invalidate(clearfilecache=True)
+
tr = transaction.transaction(rp, self.svfs, vfsmap,
"journal",
"undo",
@@ -1205,13 +1207,15 @@ class localrepository(object):
pass
delattr(self.unfiltered(), 'dirstate')
- def invalidate(self):
+ def invalidate(self, clearfilecache=False):
unfiltered = self.unfiltered() # all file caches are stored unfiltered
- for k in self._filecache:
+ for k in self._filecache.keys():
# dirstate is invalidated separately in invalidatedirstate()
if k == 'dirstate':
continue
+ if clearfilecache:
+ del self._filecache[k]
try:
delattr(unfiltered, k)
except AttributeError:
More information about the Mercurial-devel
mailing list