Making large file handling a user-transparent, final solution

[Matt Mackall]

On Sun, 2014-11-30 at 20:39 -0500, dukeofgaming wrote:
> Hi,
> 
> For several years I have pondered about using the largefiles extension or
> not, and now that I' forced to look at the problem again by a practical
> problem (large binary assets) I see two fundamental problems for its usage:
> 
> 
> 1) It is not transparent to the user: It requires a static limit, that in
> my opinion should be dynamic (i.e. large files could be cached)... file
> revisions not handled by binary diffing, and requires manual file extension
> configuration (when IMHO, this could be done by detecting binary content as
> a sensible default, then let the user specify other files).
> 
> 2) It is not enabled by default: This leads to services like Bitbucket not
> supporting it, and being disregarded as a real problem. Yes, it is said it
> breaks the D in DVCS, but I think this would not be the case if the default
> behavior was to download all binary assets. if binary files revisions were
> handled by binary diffing, this would probably be less of a problem, and if
> when cloning Mercurial just suggested to download the latest file and point
> to a cached resource instead of reconstructing history from a separate
> store.
> 
> Large file handling is a fundamental problem in DVCS technology, and I
> hereby propose that Mercurial exploits the solution it already has, and
> stops seeing large files as a last resort and more as a feature.

There's a fundamental friction between the largefiles approach and how
the rest of Mercurial works that means it will never be possible to
fully reconcile the two. Much like you're never going to have a vehicle
that's both a good sports car and a good aircraft carrier. So I am (to
put it mildly) not optimistic that your vision is achievable. But I'm
happy to be proven wrong and am looking forward to your patches.

> Sure, there are major technical obstacles to this... but imagine... if
> Mercurial *just worked* for binary/large files.

When the technical obstacles are solved, we'll be happy to mark it a
first-class feature. Until then, advertising to people that largefiles
are anything but a last resort solution is actively doing them a
disservice in their deployment planning.

-- 
Mathematics is the supreme nostalgia of our time.