D1351: changegroup: use any node, not min(), in treemanifest's generatemanifests

[spectral (Kyle Lippincott)]

spectral created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This is fixing quadratic behavior, which is probably not noticeable in the
  common case, but if a very large directory gets added here, it can get pretty
  bad. This was noticed because we had some pushes that spent >25s in changegroup
  generation calling min() here, according to profiling.
  
  The original reasoning for min() being used in https://phab.mercurial-scm.org/rHG829d369fc5a89f4c290013271c6e5dff2aea63de was that, at that
  point in the series, we were adding almost everything to tmfnodes during the
  first iteration through the loop , so we needed to avoid sending child
  directories before parents. Later changes made it so that the child directories
  were added only when we visited the parent directory (not all of them on the
  first iteration), so this is no longer necessary - there won't be any child
  directories in tmfnodes before the parents have been sent.
  
  This does mean that the manifests are now exchanged unordered, whereas
  previously we would essentially do [a, b, b/c, b/c/d, e], we now can send a, b,
  and e in any order; b/c must still follow b, and b/c/d must still follow b/c.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D1351

AFFECTED FILES
  mercurial/changegroup.py

CHANGE DETAILS

diff --git a/mercurial/changegroup.py b/mercurial/changegroup.py
--- a/mercurial/changegroup.py
+++ b/mercurial/changegroup.py
@@ -733,7 +733,9 @@
 
         size = 0
         while tmfnodes:
-            dir = min(tmfnodes)
+            # Pick some element from tmfnodes, this is not necessarily the 'min'
+            # element.
+            dir = next(iter(tmfnodes))
             nodes = tmfnodes[dir]
             prunednodes = self.prune(dirlog(dir), nodes, commonrevs)
             if not dir or prunednodes:



To: spectral, #hg-reviewers
Cc: mercurial-devel