D4326: setdiscovery: precompute children revisions to avoid quadratic lookup

[indygreg (Gregory Szorc)]

indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Moving away from dagutil a few commits ago introduced quadratic
  behavior when resolving children revisions during discovery.
  
  This commit introduces a precompute step of the children revisions
  to avoid the bad behavior.
  
  I believe the new code should have near identical performance to
  what dagutil was doing before. Behavior is still slightly different
  because we take into account filtered revisions. But this change was
  made when we moved off dagutil.
  
  I added a comment about multiple invocations of this function
  redundantly calculating the children revisions. I believe this
  potentially undesirable behavior was present when we used dagutil,
  as the call to inverse() previously in this function created a new
  object and required computing children on every invocation. I thought
  we should document the potential for a performance issue rather than
  let it go undocumented.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D4326

AFFECTED FILES
  mercurial/setdiscovery.py

CHANGE DETAILS

diff --git a/mercurial/setdiscovery.py b/mercurial/setdiscovery.py
--- a/mercurial/setdiscovery.py
+++ b/mercurial/setdiscovery.py
@@ -117,13 +117,33 @@
     # update from heads
     revsheads = set(repo.revs('heads(%ld)', revs))
     _updatesample(revs, revsheads, sample, repo.changelog.parentrevs)
+
     # update from roots
     revsroots = set(repo.revs('roots(%ld)', revs))
 
-    # TODO this is quadratic
-    parentfn = lambda rev: repo.changelog.children(repo.changelog.node(rev))
+    # _updatesample() essentially does interaction over revisions to look up
+    # their children. This lookup is expensive and doing it in a loop is
+    # quadratic. We precompute the children for all relevant revisions and
+    # make the lookup in _updatesample() a simple dict lookup.
+    #
+    # Because this function can be called multiple times during discovery, we
+    # may still perform redundant work and there is room to optimize this by
+    # keeping a persistent cache of children across invocations.
+    children = {}
 
-    _updatesample(revs, revsroots, sample, parentfn)
+    parentrevs = repo.changelog.parentrevs
+    for rev in repo.changelog.revs(start=min(revsroots)):
+        # Always ensure revision has an entry so we don't need to worry about
+        # missing keys.
+        children.setdefault(rev, [])
+
+        for prev in parentrevs(rev):
+            if prev == nullrev:
+                continue
+
+            children.setdefault(prev, []).append(rev)
+
+    _updatesample(revs, revsroots, sample, children.__getitem__)
     assert sample
     sample = _limitsample(sample, size)
     if len(sample) < size:



To: indygreg, #hg-reviewers
Cc: mercurial-devel