Bundlerepo and at-rest bundle2 handling

[Eric Sumner]

On 12/30/14, 3:10 PM, "Pierre-Yves David" <pierre-yves.david at ens-lyon.org>
wrote:
>On 12/21/2014 03:44 PM, Matt Mackall wrote:
>> On Fri, 2014-12-19 at 23:21 +0000, Eric Sumner wrote:
>>>
>>>I¹d like to propose adding
>>> a temprepo class that is backed by a copy-on-write vfs; this should
>>> allow the ­B flag to run a standard unbundle operation in a temprepo
>>> which is then used for the rest of the command.  It also would enable
>>> the possibility of a ‹test global option, which would perform the
>>> listed command inside a temprepo instead of directly to the localrepo.
>>>
>>> Does this seem like a reasonable approach to take?
>>
>> Not really. If we have a file with small deltas but large history, the
>> size of this temprepo will potentially be huge relative to the bundle.
>> In Mercurial itself, our .po files are a good example of this.
>
>I think Eric meant some kind of Frankenstein file object that only wrote
>the appended section in /tmp/.
>
>If it worked, it would fix your concern about pto-like files.

I¹ve spent some time poking around with this idea and have produced a
mostly-working prototype: by using block-level copy-on-write, the amount
of temporary space used can be kept to somewhere near the number of bytes
that would be written by the operation on the real repository.

Unfortunately, there¹s still some code around that uses OS services
directly instead of using the VFS abstraction. Since this VFS is only
superficially backed by the filesystem, there are a significant number of
test failures due to ³No such file or directory.² The other bugs are
likely caused by poor emulation of a filesystem¹s actual behavior.
Uncertainty regarding the amount of cleanup required in the rest of the
codebase makes a cost-benefit analysis tricky at best.

Overall, this could be a viable project supporting ³dry runs² of various
kinds in Mercurial, including bundlerepo. The performance will be similar
to (but slightly worse than) the corresponding live operation, which may
be unacceptable for some common operations.   In particular, write-free
inspection of bundles would still require a similar mechanism to the
current bundlerepo implementation.

It appears that I won't have time to pursue this any farther, but I'd be
happy to go into more depth if somebody wants to pick this up and run with
it.


  ‹ Eric