bfiles: Rethinking the .hgbfiles directory (WAS: Re: bfiles filename encoding)

[Benjamin Pollack]

On Jun 7, 2010, at 11:50 AM, Benoit Boissinot wrote:

> I'm not sure you need the same perf characteristics as hg, so why
> can't you just hash the filename (or is that even necessary, can't you
> have a git like approach with just blobs indexed by their sha? does
> the server need to know the filenames?)

That's basically what I've decided is the right solution.

I've been doing a lot of trying to actually use bfiles on various systems, and
I've become convinced that the existing .hgbfiles system cannot meaningfully
be made cross-platform. The best solution I can come up with is hybridencode
or a variant. Unfortunately, hybridencode degrades into a non-reversible hash
format, which breaks bfiles' expectation that the .hgbfiles path is
reversible. After some spelunking through "real" repositories, I reluctantly
have to say that this comes up way more often than you might think--i.e.,
bfiles can expect to hit the non-reversible version of hybridencode often. The
only suitable solution to this I can come up with is using a manifest of some
type to restore reversibility.

Provided that you agree with everything I just said, I don't see a value in
keeping .hgbfiles as a directory. Instead, it'd make sense to me to bring it
in line with Mercurial's manifests and simply make it be a list of tuples in
the form (filename, SHA1 checksum). bfput/bfupdate would be updated
accordingly to merely look for a file named after the SHA-1 checksum,
regardless of path. Everything else in bfiles (i.e., the UI, the structure of
.hg/bfiles, etc.) would basically continue to work as-is.

The only downside I see to this system is that the standins can no longer be
edited by hand, nor can the diff be seen in the history view. The first is
trivially solved by adding an "hg debugbfsetstandin" or similar. I'm not
convinced that losing the latter is a big deal compared to never having bfiles
work correctly on Windows, but if that really bothers you, it'd be easy enough
to go with a (rather inscrutable) text format instead.

I'd love to hear what you think of this proposal. I've begun working on a
patch to make bfiles to work this way so we can talk in terms of code; my goal
is to have an early draft ready late tomorrow or Friday. If this idea is
sufficiently repulsive that you wouldn't accept the patch ;-) then let me know
so that I'm not wasting my time, but I'd really like to know what alternative
system could work on all platforms.

Cheers,

--Benjamin