New bundle format; code structure

[Steve Borho]

On Mon, May 23, 2011 at 9:16 AM, Sune Foldager <cryo at cyanite.org> wrote:
> I'm currently looking at and thinking about the new bundle format. Not just
> the
> format, but also how we'll implement it in the code. The current state of
> affairs is a slight mess, as I wrote a mail about recently: sometimes full
> bundles are used, often headerless ones. Sometimes there is compression,
> sometimes not etc. I have tried to sum it up in the tables below:
>
> Push:
>
> proto    full bundle?    compression
> http     yes             any (capability), unchunked
> ssh      no              none, (streamed in 4k chunks)
>
> Pull:
>
> proto    full bundle?    compression
> http     no              ext. zlib (4k chunks, 64k for decomp.)
> ssh      no              none
>
>
> Here "ext. zlib" means external zlib, as in: not being done by the bundler
> (in general, not much bundle-related IS done by the two bundler classes..
> they are a bit strange, really :p).
>
> Since we're introducing a new argument for getbundle anyway, wouldn't it be
> nicer if we could work towards something like this for pull:
>
> Pull ng:
>
> proto    full bundle?    compression
> http     yes             any (capability/server's choice, chunked)
> ssh      yes             any (capability/server's choice, chunked)
>
> Default could then be zlib for http, none for ssh. Of course this will
> require
> shuffling some code around, possibly enhancing the capabilities of the
> bundling
> code and such, to ensure we can still do efficient chunked compressions and
> decompressions. I have a vague idea in mind I could make more concrete.
>
> For push, we need a new unbundle command (like getbundle; sendbundle?), and
> we
> could perhaps "fix" ssh in that case as well, so we can unify it with http.
>
> Thoughts on that?
>
> Also, a draft of a new bundle format is here:
> http://mercurial.selenic.com/wiki/BundleFormat2
>
> It's rough, and Benoit had some comments (on irc), which I forgot. I tried
> to
> include the following features:
>
> - support for capabilities for future enhancements
> - support for progress by including number of chunks for each changegroup
> - support for deltaparent (for generaldelta)
>
> Thoughts on that?

+1 on any path that adds optional compression for ssh, given the
sucktastic ssh clients on Windows.

-- 
Steve Borho