[PATCH] discovery: be more conservative when adjusting the sample size

[Martin von Zweigbergk]

On Wed, Jun 5, 2019, 03:36 Pierre-Yves David <pierre-yves.david at ens-lyon.org>
wrote:

> # HG changeset patch
> # User Pierre-Yves David <pierre-yves.david at octobus.net>
> # Date 1559726605 -7200
> #      Wed Jun 05 11:23:25 2019 +0200
> # Node ID 2c67430451fafcdd68770436c520e2d008428986
> # Parent  12793787439538411013edffe0f9b98762d38a37
> # EXP-Topic discovery-large-undecided
> # Available At https://bitbucket.org/octobus/mercurial-devel/
> #              hg pull https://bitbucket.org/octobus/mercurial-devel/ -r
> 2c67430451fa
> discovery: be more conservative when adjusting the sample size
>
> Since 5b34972a0094, the discovery will increase the sample size when it
> detect a
> "complex" undecided set. However this detection focussed on the number of
> roots
> only, this could regress discovery performance when the undecided set has
> many
> roots that eventually get merged into a few heads.
>

Seems unlikely to happen in practice. That's why I didn't bother with it in
my patch.

Also, the performance regression you're talking about is on slow networks,
right? I.e., it's not that more round trips or more CPU would be required;
the concern is rather that it takes time to transfer the nodeids on slow
networks, right?

So if you have some CI system that merges 100k heads into one in order to
tests all proposed changes together, and it's on slow network, it would
benefit from this patch?


> To prevent such misbehavior, we adjust the logic to take in account both
> heads
> and roots. The sample size will be increased only if both are especially
> large.
>
> Performance testing on the same case as 5b34972a0094, does not show a
> significant difference.
>
> diff --git a/mercurial/setdiscovery.py b/mercurial/setdiscovery.py
> --- a/mercurial/setdiscovery.py
> +++ b/mercurial/setdiscovery.py
> @@ -236,14 +236,19 @@ class partialdiscovery(object):
>          sample = set(repo.revs('heads(%ld)', revs))
>          parentrevs = self._parentsgetter()
>
> +        ### update from both directions
> +        revsroots = set(repo.revs('roots(%ld)', revs))
> +        nbroots = len(revsroots)
> +        revsheads = sample.copy()
> +        nbheads = len(revsheads)
> +
> +        if not self._respectsize:
> +            size = max(size, min(nbroots, nbheads))
> +
>          # update from heads
> -        revsheads = sample.copy()
>          _updatesample(revs, revsheads, sample, parentrevs)
>
>          # update from roots
> -        revsroots = set(repo.revs('roots(%ld)', revs))
> -        if not self._respectsize:
> -            size = max(size, len(revsroots))
>
>          childrenrevs = self._childrengetter()
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mercurial-scm.org/pipermail/mercurial-devel/attachments/20190605/33471b4d/attachment.html>