[PATCH 1 of 3] bundle: allow creation of changegroup3 bundles

Pierre-Yves David pierre-yves.david at ens-lyon.org
Sat May 6 05:50:32 EDT 2017



On 05/05/2017 12:14 AM, Gregory Szorc wrote:
> On Thu, May 4, 2017 at 12:14 PM, Pierre-Yves David
> <pierre-yves.david at ens-lyon.org <mailto:pierre-yves.david at ens-lyon.org>>
> wrote:
>
>
>
>     On 05/04/2017 07:04 PM, Martin von Zweigbergk via Mercurial-devel wrote:
>
>         # HG changeset patch
>         # User Martin von Zweigbergk <martinvonz at google.com
>         <mailto:martinvonz at google.com>>
>         # Date 1458952398 25200
>         #      Fri Mar 25 17:33:18 2016 -0700
>         # Node ID f4b255ff67d05e329cc3d80ce1552f30005f58e9
>         # Parent  06058cbb9209df54bfa0aaf9f7d10b836a1e3ee9
>         bundle: allow creation of changegroup3 bundles
>
>         With this change, one can use "hg bundle -t v3" to creat
>         changegroup3
>         bundles, which support treemanifests and revlog flags. In
>         treemanifest
>         repos, v3 is the default bundle format.
>
>
>     The bundle spec "version" cover a collection of setting for the
>     underlying content. It is not directly correlated to the changegroup
>     version.
>
>     Tree manifest is still experimental so it seems a bit early to burn
>     a new bundle spec version on this especially since we have other
>     bundle improvement coming (phase, tag cache, obsmarkers, etc).
>
>     So I don't think we should introduce a new bundlespec version yet.
>     Instead I would recommend the following:
>
>     1) introduce a parameter to control de cg-version (cgversion=03?)
>     2) automatically bump the cg version to '03' if no spec is passed
>     and tree manifest is used,
>     3) late in the 4.3 cycles consider introducing a v3 with all the new
>     improvements (if some make sense at the user-visible feature level).
>
>
> There was confusion about this in IRC.
>
> I agree with what Pierre-Yves said: while this will require a "v3" to
> support cg3, it is still a bit too early in cg3's support cycle to
> introduce a stable, user-facing bundle spec version. Until we have a
> better API for controlling bundle generation via bundle specs, I think
> this should be hacked in via an experimental config option while
> recycling "v2." Alternatively, we can name the version "expcg3" or
> something less stable and rename it (presumably to "v3") once the
> feature is stable. The latter will make it much easier to test the
> feature and won't result in accidentally producing "v2" bundles that
> legacy clients choke on.
>
> The concern with the "vN" versions is that every Mercurial client with
> knowledge of that version string has to be capable of handling any
> future bundle generated with that same "vN" version (or have code for
> filtering from other data in the clone bundles manifest, preferably from
> BUNDLESPEC). This means every time there is a new required bundle part,
> we have to invent a new "vN" or find some other way to alter
> clonebundles entries so old clients know they can't consume them.

TL;DR; We needs the flexibility of params based spec. We should start 
there and care about the "vN" version later in the cycle.

About bundle parameters
-----------------------

(putting compatibility question aside for now)

There are two dimensions in play here:

1) data format (how we encode the information),
2) data content (what data do we encore in the bundle).

Formats makes it "easy" to think in terms of linear increasing version 
number, but it is less clear for content. The mix of both makes it more 
complex.

The above make me think we should aim for a parameter centric solution here.

There are multiple kinds of data one -may- want includes into a bundles.

* changesets,
* phases,
* bookmarks,
* obsolescence,
* (and various caches)

I do not think there will be any definitive answer here. For example 
people might want to generate a bundle with or without bookmarks. So we 
needs a reasonable "interface" for users to select the bundle content. 
bundlespec parameters seems like the way to go. (eg: 
"v3;phases=1;bookmarks=0;obsolescence=0")

In addition to that, there are the question of -how- these data are 
encoded (eg: changegroup version). Since there are multiple types of 
data, it seems useful to be able to control version used for each part 
(eg: v3;cg.version=04;obsolescence.version=05).


Of course, we should have a reasonable interface for our simple user.
So, the way I see the bundlespec's "vN" is more collection of default 
for the various value described above. picking a "v3" will select value 
for the known parameters, but parameters will override them. For example:

* v2 → cg.version=02;phases=no;bookmarks=no;obsolescence=no
* v3 → cg.version=03;phases=yes;bookmarks=no;obs…=yes;obs….version=02

And the following would be valid

* v3;bookmark=yes
* v3;cg.version=02

Depending of the features in use in a repository, a default version is 
picked:

For example:
* old format -> v1
* general delta -> default v2+
* tree manifest -> default v3+

About compatibility
-------------------

Gregory is raising a valid concern regarding bundle parameters. Right 
now they are ignored by the code and all existing (clonebundle) client 
do not process them at all.

However, this concerns is clonebundle specific. As long as we do not 
exposes `v2;pa=rams` bundles in clonebundle manifest we are in the 
clear. other usages will be fine with using parameters with v2.

In all cases, old client will always mis-behave when it comes to the 
behavior of `hg bundle v2;any=params;values=foo;` I think we'll have to 
lives with that :-/.

Way forward
-----------

Here is a proposal of how to move forward on this topic

1) focus on defining the parameters first,

2) add safety to ensure we do not generate clone bundle manifest using 
"v2;pa=rams"

3) decide on the value of "v3" around the end of the 4.3 cycle.

What do you think?

-- 
Pierre-Yves David


More information about the Mercurial-devel mailing list