Wireprotocol changes for parentdelta

Pradeepkumar Gayam in3xes at gmail.com
Sun Aug 1 17:05:50 CDT 2010

-- Pradeep

On Sun, Aug 1, 2010 at 7:19 PM, Matt Mackall <mpm at selenic.com> wrote:

> On Sun, 2010-08-01 at 12:35 +0530, Pradeepkumar Gayam wrote:
> >
> > On Fri, Jul 30, 2010 at 11:04 PM, Matt Mackall <mpm at selenic.com>
> > wrote:
> >
> >         So we're going to need a completely new changegroup command.
> >         One that
> >         takes some option flags. The unified protocol supports varargs
> >         commands,
> >         so handling multiple new flags should be easy.
> >
> > Meaning, a completely new command like 'newchangegroup()' which takes
> > varargs and does the same job as changegroup() with additional
> > functionality??
> Yes. Except using names like new, modern, NG, extended, etc., is just
> silly because all things cease to be new very quickly and it'll
> eventually be time for a newnewchangegroup. Even changegroup2 is better,
> though something descriptive like changegroup{flags,opts} might also be
> good. If we call our new bundle format bundle2, then changegroup2 might
> be appropriate.
> >         Also, what's going on with the bundle format for parentdelta?
> >         It
> >         currently only supports linear deltas. So we need to extend
> >         it,
> >         otherwise the big manifests we were trying to shrink will
> >         bloat back up
> >         10x over the wire when we do a clone or pull, which makes the
> >         whole
> >         feature a little pointless. And that means we're going to need
> >         a new
> >         bundle version number.
> >
> > I am not sure about the new bundle format. I am not expert in other
> > areas. What ever I decide, it would be parentdelta centric.(I am not
> > sure if it matters for others)
> Yes, it absolutely does matter for others. This is the ONE thing I've
> been trying to get through to you and vsh about your projects the whole
> time: fixing up the wire protocol for your projects is going to be hard
> and is going to require lots of interaction with other people. The wire
> protocol (and the bundle format, which is obviously part of it) are
> painful to change because we have to allow for backwards compatibility.
> We do not want to create a new bundle format every time we add a
> feature. We need to think ahead.
> Fact is, we've already done a large amount of that thinking ahead at the
> 1.5 sprint. But no one's going to tell you about it unless you ask.
> Here's a really obvious feature that the new bundle format also needs to
> support: shallow clone. Another is long hashes.
> The bundle chunk format looks like:
> 4  length
> 20 node
> 20 p1
> 20 p2
> 20 changeset hash (aka linkrev)
> *  data
> It's going to need to be extended to something like:
> 4 flags
> 4 length
> n node
> n p1
> n p2
> n delta parent (either p1, p2, n-1, or possibly null?)
> n changeset hash
> * data
> http://mercurial.selenic.com/wiki/BundleFormat
> >  So, I didn't think anything concrete.
> Ok, but you know what's required for your piece, so start telling us
> about it.

New bundle format is useful only when both server and client are new.
Because, if one of them is old we can't use the new protocol. If new client
is interacting with new server both of them are not aware of other's repo.
Then a flag for each chunk in the group should be enough to ease the
problem. 'delta parent' is particularly not required for the this, but can
be used for cross checking.

> > I am still not clear about some definitions.
> >
> > 1) What is new repo and old repo.
> >              As far as experts cleared to me in IRC if we initiate or
> > clone with new hg then we call it a new repo. If it is right, how do
> > mark them as new repo. Is it something goes into requires or a flag
> > into revlog header or both?
> Traditionally, we do this with a requires flag. Newly-created
> repositories add the flag to say "to read and write from this repo, you
> must understand feature X".
> > 2) Shall we add parent deltas on the top of tip deltas in old repo.
> No. An old repo must continue to be readable by both old and new
> clients. A new client can never 'upgrade' the requires file of an
> existing repo.

So, I can do something like this:

        if self._parentdelta:
            chain = self.deltachain(rev)
            chain = [r for r in xrange(base + 1, rev + 1)]

Which does not affect the performance on the old repos.

> --
> Mathematics is the supreme nostalgia of our time.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://selenic.com/pipermail/mercurial-devel/attachments/20100802/e813755d/attachment.htm>

More information about the Mercurial-devel mailing list