[PATCH 2 of 6 V2] hidden: use _domainancestors to compute revs revealed by dynamic blocker

[Pierre-Yves David]

On 05/24/2017 07:08 PM, Martin von Zweigbergk via Mercurial-devel wrote:
> On Wed, May 24, 2017 at 5:42 AM, Pierre-Yves David
> <pierre-yves.david at ens-lyon.org> wrote:
>>
>>
>> On 05/24/2017 06:04 AM, Martin von Zweigbergk wrote:
>>>
>>> On Tue, May 23, 2017 at 7:10 PM, Gregory Szorc <gregory.szorc at gmail.com>
>>> wrote:
>>>>
>>>> On Tue, May 23, 2017 at 1:02 PM, Pierre-Yves David
>>>> <pierre-yves.david at ens-lyon.org> wrote:
>>>>>
>>>>>
>>>>> # HG changeset patch
>>>>> # User Pierre-Yves David <pierre-yves.david at octobus.net>
>>>>> # Date 1495373721 -7200
>>>>> #      Sun May 21 15:35:21 2017 +0200
>>>>> # Node ID e72ddd1a53c4c6321e7ecd686cd24c2a8c8914bc
>>>>> # Parent  5f964af88a0fae242ce24b0478c676d2056e0dc6
>>>>> # EXP-Topic fast-compute-hidden
>>>>> # Available At
>>>>> https://www.mercurial-scm.org/repo/users/marmoute/mercurial/
>>>>> #              hg pull
>>>>> https://www.mercurial-scm.org/repo/users/marmoute/mercurial/ -r
>>>>> e72ddd1a53c4
>>>>> hidden: use _domainancestors to compute revs revealed by dynamic blocker
>>>>>
>>>>> The complexity of computing the revealed changesets is now
>>>>> 'O(revealed)'.
>>>>> This massively speeds up the computation on large repository. Moving it
>>>>> to
>>>>> the
>>>>> millisecond range.
>>>>>
>>>>> Below are timing from two Mozilla repositories with different contents:
>>>
>>>
>>> Timing of what? How can I run the test on hg core for comparison?
>>
>>
>> Ha, good point. Using the perf extension:
>>
>>   hg perfvolatileset obsolete visible
>
> Do I just run "./hg perfvolatileset obsolete visible"? If I do that on
> the hg core repo, I get these results:
>
> Before:
> ! obsolete
> ! wall 0.004844 comb 0.010000 user 0.010000 sys 0.000000 (best of 547)
> ! visible
> ! wall 0.009090 comb 0.010000 user 0.010000 sys 0.000000 (best of 310)
>
> After:
> ! obsolete
> ! wall 0.004720 comb 0.010000 user 0.010000 sys 0.000000 (best of 554)
> ! visible
> ! wall 0.009028 comb 0.000000 user 0.000000 sys 0.000000 (best of 304)
>
>
> So there's practically no improvement at all on the hg core repo.
> Where can I find the Mozilla repos to try it on?

You need to have a least one dynamic-blocker for that code to quicks in.
(so either a bookmark or the working copy on an otherwise hidden 
changeset). You can use the following command for this purpose:

   hg up --hidden 'max(hidden())'

I use "max" because the higher the blocker is, the higher the impact of 
the old code was.

In my mercurial repository I get the following number:

Before
! obsolete
! wall 0.009808 comb 0.010000 user 0.010000 sys 0.000000 (best of 285)
! visible
! wall 0.019836 comb 0.020000 user 0.020000 sys 0.000000 (best of 149)

After
! obsolete
! wall 0.009582 comb 0.020000 user 0.020000 sys 0.000000 (best of 301)
! visible
! wall 0.012674 comb 0.010000 user 0.010000 sys 0.000000 (best of 230)

Adjusted time gives (10ms vs 3 ms) a 3x improvement
(for this specific changeset)

The running time of these algorithm is affected by multiple variable:
* how many obsolete changesets,
* how many mutable changesets in general,
* how early in the revlog is the first obsolete changeset.

So depending of the repository characteristic, timing will change a lot. 
The new algorithm is much less subject to variation on "abstract" thing 
like the early-ness of the first obsolete changeset.
(which is why I included some stats on the repository in my timing report).

For my mercurial core repository I've:
  * 38542 changeset total
  * 214 visible changesets
  * 5896 hidden changesets
  * oldest hidden changesets is rev 28845

The mozilla repository are two repos based on greg "devel" repository. 
on is build from pulling from him[1] every couple of months. The other 
one is a tarball of his local repository (so many more hidden 
changesets) that you can probably obtain from him directly.

Cheers,

-- 
Pierre-Yves David