[PATCH 1 of 5 clonebundles V2] clonebundles: support for seeding clones from pre-generated bundles

[Pierre-Yves David]

On 10/09/2015 12:41 PM, Gregory Szorc wrote:
> On Fri, Oct 9, 2015 at 12:33 PM, Gregory Szorc <gregory.szorc at gmail.com
> <mailto:gregory.szorc at gmail.com>> wrote:
>
>     # HG changeset patch
>     # User Gregory Szorc <gregory.szorc at gmail.com
>     <mailto:gregory.szorc at gmail.com>>
>     # Date 1444414921 25200
>     #      Fri Oct 09 11:22:01 2015 -0700
>     # Node ID d117d532ed0b3d03c8e4fe85b819964669d1be1a
>     # Parent  6e715040c1725b5debce888c4f7d3fdbf55cc900
>     clonebundles: support for seeding clones from pre-generated bundles
>
>
> Augie and Pierre-Yves and I video conferenced about this on Tuesday. The
> big change is that client-side support is disabled by default and hidden
> behind an experimental flag. We intend to leave it this way until at
> least 3.7.
>
> Once this series is queued, I will send a patch to have the server
> extension emit a bundle2 output part informing the client about said
> config option such that servers that deploy this may nudge clients into
> testing the feature. I should probably also write a wall of text with
> instructions for server operators.
>
> Pierre-Yves isn't going to like the lack of support for bundle2. But I'm
> afraid I don't see a way to support it given the lack of a clear and
> future-proof method for checking compatibility with pre-generated
> bundle2 files. Read the commit message in patch 2. Since the feature is
> now experimental, I don't see a major problem initially landing without
> bundle2 support. We can and should add it before the feature is enabled
> by default.

The patch here is pretty paved so I do not see a good reason to delay 
the support.

We just recently settle for a format specification to be fed to
`hg bundle --type XXX`. The currently support v1 and v2:

- v1: a HG10 bundle with a cg01 (well, a HG10 bundle),
- v2: a HG20 bundle with a cg02 bundle,

Strickly speaking, v2 can expect to have anything we currently support 
in 3.6, but the changegroup part is the only purely descriptive thing we 
supports (phases and bookmark are exchanged  in pushkee "imperative" 
part right now) [now that I'm writing it, Is se obsmarkers as apossible 
other part].

As there is no used for that, we do not have any function that do the 
"type" → bundle-caps translation/checking. This should be pretty easy to 
build if we get a user for it.

Regarding extensibility, that v2 format may be extended in two ways:

1) The format is specified to be extensible. you could have v2+foobar to 
entrail the fact you also includes feature that requires support for 
thing enclosed by +"foobar" feature. Such feature name will likely be 
thing like "phases" or "cache1". We won't use a direct part → feature 
mapping because (a) we want something human friendly to types (b) we'll 
likely group multiple small related part into a single umbrella. At some 
point we'll spine a v3 to include all moderns/hard to avoid new data.

2) Bundle2 allow advisory and mandatory parts. It is possible to add 
more data to a v2 flag that what v2-reader are requested by contract to 
be able to write as long as they are advisory. For example you can add 
cache related data in your "v2" bundle as advisory. New client will add 
the changegroup and the cache, while old client will add the changegroup 
and happily ignore the cache part, recomputing it.


This last point also allow to cover another of your concern: failing 
fast on unsupported bundle. It is very easy to introduce a 
"bundld-e-manifest" part at the begining of a any bundle2 bundle. That 
part would describe the bundle content for various purpose, including 
compatibility checking for failing fast. Such part can be added at any 
point in any bundle as advisory. Client with support would fail fast. 
Other would proceed as before.

I hope this clarify thing on your end.

-- 
Pierre-Yves David