RFC: A mercurial pack extension

[Peter Arrenbrecht]

John, I appreciate your efforts. I have some experience working with
this bundle-sharing approach in a small team (3 guys). We used a
common ftp drop. What we needed most were clear naming conventions. In
the end, we had

  myproject-base.hg
  myproject-joe.hg
  myproject-jim.hg
  ...

for the base repo and the individual developers' latest contributions.
The contributions were bundles against the base repo, so they were
usually quite small. I was in charge of periodically updating
myproject-base.hg to keep the uploads of developer contributions
small. For review and collaboration on versioned patch queues, we
would upload

  myproject-`hg id -i -r qparent`-feature.hgq

This ensured we would know exactly on top of which revision the queue
was currently meant to apply cleanly. (Within the queue, we had
another regime for keeping collaboration under control, but that is
another topic.)

So in addition to simplifying the handling of bundle-sharing repos, I
think we should try to establish a common way to work with these
beasts, not least some naming conventions. If my approach sounds
reasonable, I'd be glad to contribute a wiki page on it.

Re: pack/unpack

It took me a while to hunt down `--base null` too, at the time.
However, since `hg help bundle` at last tells you to use `--base null`
to get a full bundle, maybe adding another command is going too far
here? Especially as, at least in the way we did it, doing a full
bundle is rarely needed by a newcomer to a project. They rather
contribute partial bundles against the existing base. I do like `hg
bundle --full` as suggested by Mark, though.

In clone/unpack, couldn't you assume that a reference to a file always
means a bundle so we can do away with the bundle: prefix? I cannot
imagine we'll have a better candidate for what a file repo would be by
default in the foreseeable future. Or at least when the file extension
is .hg (or whatever common convention we can agree on). Then both pack
and unpack could be nicely handled by the existing commands without a
lot of arcane clutter. Maybe having to pass -u to unbundle is still
cluttering things up a bit.

I very much like the idea of supporting automatic downloads of
bundles. I guess automating the download and unbundling of xy-base.hg
when you clone a _new_ repo from xy-zz.hg is going too far in casting
the above conventions in stone.

Maybe an option to upload to some standard protocols could be added to
bundle, too? Like

  hg bundle ftp://my.host.com//hgbundle/myproject-peo.hg ../base

where ../base is my local clone of myproject-base.hg. Of course, one
could also use a tag for this. Might be especially handy if the
[paths] from .hgrc would work here.

If any help is needed, I'll gladly give what I can. I'm fairly new to
Python, though.

-peo


On Feb 18, 2008 4:39 AM, Mark Williamson <mark.williamson at cl.cam.ac.uk> wrote:
> Hi John,
>
> > I recently wrote an extension and was hoping to get some feedback on it.
> > This extension is called 'pack' and creates whole-history bundles, and can
> > clone from them. It doesn't do anything that couldn't be done before with
> > some manual work, but I felt the urge to automate the process.
>
> Thank you!  It's nice that it's possible to package an entire repository
> history in a bundle but a bit frustrating not to have a more brief way of
> doing this (and of unpacking / cloning it later).
>
> I wonder if the developers would consider integrating this functionality into
> the hg core, rather than as an extension.  The extension is a useful
> addition, of course, but I'd think it might be good to have this stuff all
> work by default.  Had you considered taking that approach?
>
> > It creates two new sub-commands 'pack' and 'unpack' which create the bundle
> > and clone from it respectively. The clone sub-command is also extended to
> > support bundle sources in the form 'bundle:<filename>' and bundle+<url>.
> > This clone operates the same as unpack with the bundle prefix removed. I
> > would especially appreciate feedback on that syntax, I'm not sure if it's
> > very mercurial-esque.
>
> I think that having the two commands isn't strictly necessary, although it
> makes life nicer when working in an extension and it makes for more succinct
> commandlines.
>
> As a user, I'd be mostly happy if I could just do something like "hg
> bundle --full <bundlename>" to create a bundle with all history in, then "hg
> clone bundle://<bundlename>" to clone from it.
>
> > This extension is intended to be useful to users of static web hosting,
> > ftp, IM, etc. or other situations where ssh or http are not
> > appropriate/available, and distributing an entire directory tree would be
> > inconvenient. There is a feature request/suggestion on the Mercurial wiki
> > of allowing clone from a bundle. Also, I remember some issues on the
> > mailing list with zip files and bundles were suggested but I had to hunt on
> > the wiki to figure out how to get one that had all the changesets in it.
>
> I see your extension can automatically download bundles for you, which is a
> nice improvement for enabling bundles as a "clone" source.  Is that just over
> HTTP or can it use any other protocols?
>
> Cheers,
> Mark
>
> --
> Push Me Pull You - Distributed SCM tool (http://www.cl.cam.ac.uk/~maw48/pmpu/)
> _______________________________________________
> Mercurial mailing list
> Mercurial at selenic.com
> http://selenic.com/mailman/listinfo/mercurial
>