[RFC] kbfiles: an extension to track binary files with less wasted bandwidth

Na'Tosha Bard natosha at unity3d.com
Thu Sep 22 09:19:03 CDT 2011


So, to pick this topic up again, can we get an open punchlist of things that
the mercurial community (and project leader) believes is "missing" for the
largefiles extension? E.g, what is missing for it to be accepted into
mercurial?

The main repository is living here:
https://developers.kilnhg.com/Repo/Kiln/largefiles/largefiles

(there's also a branch with some compatibility stuff that's useful for Kiln
users, but that is not so relevant here).

Cheers,
Na'Tosha

2011/8/15 Greg Ward <greg-hg at gerg.ca>

> On Sun, Aug 14, 2011 at 6:24 PM, Andrew Pritchard <awpritchard at gmail.com>
> wrote:
> >> Idea: how hard would it be to add the standin prefix dir to the wire
> >> protocol? It should only have to be sent once per conversation.
> >
> > That would actually not be particularly difficult, and it could easily
> > be stored in '.hg/largefiles/standins' or the like.  The
> > largefiles-kiln branch already has code for dealing with per-repo
> > prefixes, to make kbfiles repositories continue to work, but it
> > currently just checks for the existence of ''.hg/store/data/.hglf/*.i"
> > or ".kbf/*.i".
>
> So users of kbfiles have the same problem as users of bfiles? I.e.
> switching to largefiles would be easy if largefiles transparently
> recognized the old standin dir, but disruptive if we force them to
> convert with a filemap?
>
> > This would still make hosting providers' jobs a bit
> > harder, since they would have to keep track of the prefix in order to
> > handle largefiles separately.
>
> If they choose to. IMHO if a hosting provider chooses to say "we
> support largefiles, but *only* with the canonical default .hglf [or
> whatever we settle on] prefix", that's entirely reasonable. bfiles and
> kbfiles should be viewed as prototypes.
>
> > Just brainstorming here, but currently the value of the 'lfilestore'
> > capability carries no meaning: I set it arbitrarily to 'serve' to
> > leave it open to future versions of the protocol or alternative ways
> > of providing a store, but it could be used as the prefix, as in
> > 'lfilestore=.hglf'.  This would be a bit weird in that the
> > capabilities reported by the server could vary per repository, and it
> > also doesn't solve the problem of identifying the prefix when pushing
> > largefiles changesets to a repository that currently has none.
>
> That does seem weird.
>
> > Abandoning that line of thought almost entirely, it could use the
> > pushkey mechanism, adding no new wireproto commands and even taking
> > advantage of the warn-on-already-existing key behavior in the case of
> > a repository that already has largefiles.
>
> Yeah, that sounds sensible.
>
> > There could be some weirdness if two developers simultaneously add
> > largefiles to a previously-vanilla repository and try to push them,
> > but as long as new largefiles repositories always use a specific
> > prefix, that shouldn't cause any problems.
>
> I don't think we should expose any way to use a non-default standin
> prefix. .hgbfiles/ and .hgkbf/ are legacy prefixes that we should
> support for the convenience of users, but there is no reason to create
> a new largefiles repo using those prefixes. One ring to rule them all,
> and all that.
>
> Greg
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel at selenic.com
> http://selenic.com/mailman/listinfo/mercurial-devel
>



-- 
*Na'Tosha Bard*
Build & Infrastructure Developer | Unity Technologies

*E-Mail:* natosha at unity3d.com
*Skype:* natosha.bard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://selenic.com/pipermail/mercurial-devel/attachments/20110922/d7d95f57/attachment.html>


More information about the Mercurial-devel mailing list