D7839: nodemap: deleted older raw data file when creating a new ones

marmoute (Pierre-Yves David) phabricator at mercurial-scm.org
Sat Jan 11 12:04:52 EST 2020


marmoute created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  When we write new full files, it replace an older one with a different name. We
  add the associated cleanup for the older file to be removed after the
  transaction.
  
  We delete all file matching the expected pattern to give use extra chance to
  delete orphan files we might have failed to delete earlier.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D7839

AFFECTED FILES
  mercurial/revlogutils/nodemap.py
  tests/test-persistent-nodemap.t

CHANGE DETAILS

diff --git a/tests/test-persistent-nodemap.t b/tests/test-persistent-nodemap.t
--- a/tests/test-persistent-nodemap.t
+++ b/tests/test-persistent-nodemap.t
@@ -12,6 +12,8 @@
   $ hg debugbuilddag .+5000
   $ f --size .hg/store/00changelog.n
   .hg/store/00changelog.n: size=18
+  $ f --sha256 .hg/store/00changelog-*.nd
+  .hg/store/00changelog-????????????????.nd: sha256=5dbe62ab98a26668b544063d4d674ac4452ba903ee8895c52fd21d9bbd771e09 (glob)
   $ hg debugnodemap --dump-new | f --sha256 --size
   size=245760, sha256=5dbe62ab98a26668b544063d4d674ac4452ba903ee8895c52fd21d9bbd771e09
   $ hg debugnodemap --dump-disk | f --sha256 --bytes=256 --hexdump --size
@@ -32,3 +34,15 @@
   00d0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
   00e0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
   00f0: ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff |................|
+
+add a new commit
+
+  $ hg up
+  0 files updated, 0 files merged, 0 files removed, 0 files unresolved
+  $ echo foo > foo
+  $ hg add foo
+  $ hg ci -m 'foo'
+  $ f --size .hg/store/00changelog.n
+  .hg/store/00changelog.n: size=18
+  $ f --sha256 .hg/store/00changelog-*.nd --size
+  .hg/store/00changelog-????????????????.nd: size=245760, sha256=32d66a30ec934beecfbee8d3151d1f87b6d89ecc5713cdb123b4cdcd4e3b27a4 (glob)
diff --git a/mercurial/revlogutils/nodemap.py b/mercurial/revlogutils/nodemap.py
--- a/mercurial/revlogutils/nodemap.py
+++ b/mercurial/revlogutils/nodemap.py
@@ -9,6 +9,7 @@
 from __future__ import absolute_import
 
 import os
+import re
 import struct
 
 from .. import (
@@ -66,6 +67,16 @@
     data = persistent_data(revlog.index)
     uid = _make_uid()
     datafile = _rawdata_filepath(revlog, uid)
+    olds = _otherrawdata_filepath(revlog, uid)
+    if olds:
+        realvfs = getattr(revlog, '_realopener', revlog.opener)
+
+        def cleanup(tr):
+            for oldfile in olds:
+                realvfs.tryunlink(oldfile)
+
+        callback_id = b"revlog-cleanup-nodemap-%s" % revlog.nodemap_file
+        tr.addpostclose(callback_id, cleanup)
     # EXP-TODO: if this is a cache, this should use a cache vfs, not a
     # store vfs
     with revlog.opener(datafile, 'w') as fd:
@@ -129,6 +140,18 @@
     return b"%s-%s.nd" % (prefix, uid)
 
 
+def _otherrawdata_filepath(revlog, uid):
+    prefix = revlog.nodemap_file[:-2]
+    pattern = re.compile(b"(^|/)%s-[0-9a-f]+.nd$" % prefix)
+    newfilepath = _rawdata_filepath(revlog, uid)
+    dirpath = revlog.opener.dirname(newfilepath)
+    others = []
+    for f in revlog.opener.listdir(dirpath):
+        if pattern.match(f):
+            others.append(f)
+    return others
+
+
 ### Nodemap Trie
 #
 # This is a simple reference implementation to compute and serialise a nodemap



To: marmoute, #hg-reviewers
Cc: mercurial-devel


More information about the Mercurial-devel mailing list