[PATCH 5 of 5 V2] changegroup: compute seen files as changesets are added (issue4750)

Matt Mackall mpm at selenic.com
Fri Jul 17 16:25:07 CDT 2015


On Wed, 2015-07-15 at 15:32 -0700, Gregory Szorc wrote:
> # HG changeset patch
> # User Gregory Szorc <gregory.szorc at gmail.com>
> # Date 1436998444 25200
> #      Wed Jul 15 15:14:04 2015 -0700
> # Node ID e588eefb8ae309b18d4a2e4e28134d363fd9cf8e
> # Parent  ed5d4ffdff877bfe632376a17a5a50cece817209
> changegroup: compute seen files as changesets are added (issue4750)

This all strikes me as too complex. I think we should deal with your
primary problem (the pause) for 3.5 with a straightforward patch like
the below, and then come back to the plumbing issues with flush and
extra reads and such post-3.5. It doesn't seem like the performance
impact of the additional flush is significant?

(It's also worth noting that we could be fully-populating the branch
cache here.)

diff -r de220ce26b68 mercurial/changegroup.py
--- a/mercurial/changegroup.py	Fri Jul 17 15:53:56 2015 +0200
+++ b/mercurial/changegroup.py	Fri Jul 17 16:19:59 2015 -0500
@@ -721,6 +721,8 @@
 
     changesets = files = revisions = 0
     efiles = set()
+    def clfilescb(node):
+        efiles.update(repo[node].files())
 
     tr = repo.transaction("\n".join([srctype, util.hidepassword(url)]))
     # The transaction could have been created before and already carries source
@@ -754,13 +756,11 @@
         source.callback = prog(_('changesets'), expectedtotal)
 
         source.changelogheader()
-        srccontent = cl.addgroup(source, csmap, trp)
+        srccontent = cl.addgroup(source, csmap, trp, callback=clfilescb)
         if not (srccontent or emptyok):
             raise util.Abort(_("received changelog group is empty"))
         clend = len(cl)
         changesets = clend - clstart
-        for c in xrange(clstart, clend):
-            efiles.update(repo[c].files())
         efiles = len(efiles)
         repo.ui.progress(_('changesets'), None)
 
diff -r de220ce26b68 mercurial/revlog.py
--- a/mercurial/revlog.py	Fri Jul 17 15:53:56 2015 +0200
+++ b/mercurial/revlog.py	Fri Jul 17 16:19:59 2015 -0500
@@ -1379,7 +1379,7 @@
             ifh.write(data[1])
             self.checkinlinesize(transaction, ifh)
 
-    def addgroup(self, bundle, linkmapper, transaction):
+    def addgroup(self, bundle, linkmapper, transaction, callback=None):
         """
         add a delta group
 
@@ -1459,6 +1459,11 @@
                 chain = self._addrevision(node, None, transaction, link,
                                           p1, p2, flags, (baserev, delta),
                                           ifh, dfh)
+
+                if callback:
+                    flush()
+                    callback(chain)
+
                 if not dfh and not self._inline:
                     # addrevision switched from inline to conventional
                     # reopen the index


-- 
Mathematics is the supreme nostalgia of our time.



More information about the Mercurial-devel mailing list