[PATCH 6 of 8] obsolete: exchange obsolete marker over pushkey

Patrick Mézard patrick at mezard.eu
Fri Jun 8 05:28:48 CDT 2012


Le 08/06/12 11:50, Idan Kamara a écrit :
> On Thu, Jun 7, 2012 at 8:56 PM, Patrick Mézard <patrick at mezard.eu> wrote:
>>
>> Le 07/06/12 19:24, pierre-yves.david at logilab.fr a écrit :
>>> # HG changeset patch
>>> # User Pierre-Yves.David at ens-lyon.org
>>> # Date 1339089719 -7200
>>> # Node ID e1d9043377d43a41c37f086aadfbe0552854c29a
>>> # Parent  740baee7f4950cb25ef79fa0782c1b8dfe8ffb92
>>> obsolete: exchange obsolete marker over pushkey
>>>
>>> For a version of the exchange, all markers are exchange. This won't
>>> scale and we will need a better protocol later.
>>
>> To elaborate a bit, there are 2 flavours of "won't scale":
>> - The obvious exchange everything all the time
>> - The less obvious exchange everything all the time *with pushkey*. Over
>> HTTP, pushkey sends all the data in split HTTP headers, which easily
> exceeds
>> both Apache default max header count (100). It is hard to provide numbers
>> about the store size because the current obsolete/evolve extension uses
>> another JSON base format which is much more verbose than this one, and
>> because Pierre-Yves experimental mercurial evolving repository underwent
>> several format upgrades which messed up the store, but right now it is
>> around 2.2M for 5500 markers. To summarize, it is easy to reach a
> situation
>> where the HTTP servers/proxies cannot handle the pushkey load. I do not
>> think this is a problem unless people believe "won't scale" applies to
>> Bryan-sized repositories. Note that you can work around this limitation
> with
>> SSH.
> 
> This is interesting and I feel a pretty important aspect of the feature.

I agree, some answers here:

  http://selenic.com/pipermail/mercurial-devel/2012-May/040197.html

Let me try answer with my current understanding: 

> A few questions:
>
> 1) does the 'obsstore' ever shrink or does it grow indefinitely?

It grows indefinitely. This may not be as bad as it sounds in practice, there are two extreme cases:
- Public repositories. They should accept only "killing" markers.
- Draft repositories (including local repositories). They are shared between a limited number of people, who can communicate to reset the obsolete state if it grows out of control.

Pierre-Yves suggested some kind of garbage collecting scheme, but at this point I have no concrete idea how to make that work.

The hard part is preventing draft markers to be pushed on public repositories. And this aspect is important because revisions are created in draft state by default. So if you commit locally, amend several times and push somewhere, you will push all the obsolete markers you generated during the editing steps, which are completely useless to everyone and will pollute the obsstore forever.

Markers creation/exchange policies are very important and yet to be defined. Or maybe, rediscuss an new intermediate phase between secret and draft.

> 2) when I pull from someone, am I getting his entire 'obsstore' or just
> the bits I'm interested in?

Right now, everything. The goal is to retrieve predecessors you do not have for all revisions you pulled.

This network protocol is the second important aspect still to be done. Apparently, Benoit suggested during the sprint to try something similar to rsync.

> 3) how is conflict resolution handled (or will be in the future)?

Pictures here:

  http://hg-lab.logilab.org/doc/mutable-history/html/obs-concept.html#the-obsolete-marker-concept

To summarize, when you pull revisions replacing ones you are working on or their ancestors, you are notified than X of your revisions just became "unstable". Then you have to stabilize them (all this is done by evolve extension). Most of the time, it means rebasing, but there are more complex cases. I think some of them are not yet handle automatically (that is "hg stabilize -n" cannot suggest what to do), but you can always take a long look at the revision graph and perform rebase/graft calls and finally "hg kill" commands to register the right obsolete markers.

I am not sure where the official obsolete/evolve extension repository is, but you can try this one:

  http://hg-lab.logilab.org/wip/mutable-history

Pierre-Yves updated it recently to work with hg tip.

> 4) have you considered using a revlog as the storage model?

Markers are append-only, not sure revlog would help here.

--
Patrick Mézard


More information about the Mercurial-devel mailing list