D4643: filelog: custom filelog to be used with narrow repos

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Narrow repos may have file revisions whose copy/rename metadata
  references files not in the store. This can pose problems when
  consumers attempt to access a missing referenced file revision.
  
  The narrow extension hacks around this problem by implementing a
  derived filelog type that provides custom implementations of
  renamed(), size(), and cmp() which handle renames against files not
  in the narrow spec by silently removing the rename metadata.
  
  While silently dropping metadata isn't the most robust solution,
  it is the easiest to implement.
  
  This commit ports the custom narrow filelog class to core.
  
  When a narrow repo is constructed, its ifilestorage creation
  function will automatically use the new filelog type. This means
  the extra logic is 0 cost for non-narrow repos and shouldn't
  interfere with their operation.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D4643

AFFECTED FILES
  mercurial/filelog.py
  mercurial/localrepo.py

CHANGE DETAILS

diff --git a/mercurial/localrepo.py b/mercurial/localrepo.py
--- a/mercurial/localrepo.py
+++ b/mercurial/localrepo.py
@@ -743,9 +743,22 @@
 
         return filelog.filelog(self.svfs, path)
 
+ at interfaceutil.implementer(repository.ilocalrepositoryfilestorage)
+class revlognarrowfilestorage(object):
+    """File storage when using revlogs and narrow files."""
+
+    def file(self, path):
+        if path[0] == b'/':
+            path = path[1:]
+
+        return filelog.narrowfilelog(self.svfs, path, self.narrowmatch())
+
 def makefilestorage(requirements, **kwargs):
     """Produce a type conforming to ``ilocalrepositoryfilestorage``."""
-    return revlogfilestorage
+    if repository.NARROW_REQUIREMENT in requirements:
+        return revlognarrowfilestorage
+    else:
+        return revlogfilestorage
 
 # List of repository interfaces and factory functions for them. Each
 # will be called in order during ``makelocalrepository()`` to iteratively
diff --git a/mercurial/filelog.py b/mercurial/filelog.py
--- a/mercurial/filelog.py
+++ b/mercurial/filelog.py
@@ -225,3 +225,54 @@
 
     def _addrevision(self, *args, **kwargs):
         return self._revlog._addrevision(*args, **kwargs)
+
+class narrowfilelog(filelog):
+    """Filelog variation to be used with narrow stores."""
+
+    def __init__(self, opener, path, narrowmatch):
+        super(narrowfilelog, self).__init__(opener, path)
+        self._narrowmatch = narrowmatch
+
+    def renamed(self, node):
+        res = super(narrowfilelog, self).renamed(node)
+
+        # Renames that come from outside the narrowspec are problematic
+        # because we may lack the base text for the rename. This can result
+        # in code attempting to walk the ancestry or compute a diff
+        # encountering a missing revision. We address this by silently
+        # removing rename metadata if the source file is outside the
+        # narrow spec.
+        #
+        # A better solution would be to see if the base revision is available,
+        # rather than assuming it isn't.
+        #
+        # An even better solution would be to teach all consumers of rename
+        # metadata that the base revision may not be available.
+        #
+        # TODO consider better ways of doing this.
+        if res and not self._narrowmatch(res[0]):
+            return None
+
+        return res
+
+    def size(self, rev):
+        # Because we have a custom renamed() that may lie, we need to call
+        # the base renamed() to report accurate results.
+        node = self.node(rev)
+        if super(narrowfilelog, self).renamed(node):
+            return len(self.read(node))
+        else:
+            return super(narrowfilelog, self).size(rev)
+
+    def cmp(self, node, text):
+        different = super(narrowfilelog, self).cmp(node, text)
+
+        # Because renamed() may lie, we may get false positives for
+        # different content. Check for this by comparing against the original
+        # renamed() implementation.
+        if different:
+            if super(narrowfilelog, self).renamed(node):
+                t2 = self.read(node)
+                return t2 != text
+
+        return different



To: indygreg, #hg-reviewers
Cc: mercurial-devel