caching pull - stable partitioning of bundle requests

Pulkit Goyal 7895pulkit at gmail.com
Sun Mar 3 16:01:08 EST 2019


On Tue, Feb 19, 2019 at 3:23 PM Boris FELD <boris.feld at octobus.net> wrote:

> On 15/02/2019 21:16, Pulkit Goyal wrote:
>
>
>
> On Thu, Oct 4, 2018 at 6:16 PM Boris FELD <lothiraldan at gmail.com> wrote:
>
>> The road for moving this in Core is clear, but not short. So far we have
>> not been able to free the necessary time to do it. Between the
>> paying-client work, we have to do to pay salaries (and keep users happy)
>> and all the time we are already investing in the community, we are already
>> fairly busy.
>>
>> In early 2017, Bitbucket gave $13, 500 to the Mercurial project to be
>> spent to help evolution to move forward. As far as we know, this money is
>> still unspent. Since stable range is a critical part of obsmarkers
>> discovery, unlocking this money to be spent on upstreaming stable range
>> would be a good idea (and fits its initial purposes). Paying for this kind
>> of work will reduce the contention with client work and help us, or others,
>> to dedicate time for it sooner than later.
>>
>
> I definitely agree that obsmarker discovery is a critical part. Pulling
> from `hg-committed` is slower sometimes as compared to pulling on a repo
> (5-7x size of hg-committed) with server having thousands of heads.
>
> Do you have any updates on the stable-range cache? In the current state
> the cache is pretty big and a lot of people faced problem with the cache
> size.  Also in case of strip or some other commands, it rebuilts the cache
> which takes more than 10 minutes on large repo which is definitely a
> bummer. Are you working on making that better and take less size? How is
> experimentation of evolve+stablerange going on?
>
>
> # Regarding the cache-size:
>
> We know that the current version caches many entries that are trivial to
> compute and does not need to be cached. In addition, the current storage
> (SQLite) does not seem very efficient.
>
> So the next iteration of the cache should be significantly smaller.
>
>
> # Regarding cache invalidation:
>
> A lot of the data in the caches are an inherent property of the changeset
> and therefore immutable. It's easy to preserve then during strip to avoid
> having to recompute things from scratch. In addition, these immutable data
> should be exchange during pull alongside the associated changesets to avoid
> client recomputing the same data over and over.
>
> The current implementation is an experimental/research implementation, all
> this should get smoothed directly in Core during the upstreaming.
>

I am bit confused when you say "things should get smoothed directly in Core
during the upstreaming". Which one of the following did you mean:

1) send a patch of the current implementation to core, once that patch gets
in, try to improve the implementation in core
2) send a series to core which contains patch of the current implementation
and other patches improving the implementation

2) is same as what Augie did for narrow, linelog, remotefilelog, Greg did
for sparse, I did for infinitepush.

Which one do you mean here?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mercurial-scm.org/pipermail/mercurial-devel/attachments/20190304/93c3787a/attachment.html>


More information about the Mercurial-devel mailing list