Design document for allowing alternative client side storage
sean at farley.io
Thu Mar 10 22:02:43 EST 2016
Durham Goode <durham at fb.com> writes:
> On 3/10/16 6:13 PM, Sean Farley wrote:
>> Sean Farley <sean at farley.io> writes:
>>> Durham Goode <durham at fb.com> writes:
>>>> We're going to be investigating alternative client side storage
>>>> strategies for Mercurial at Facebook over the next few months. We've
>>>> already moved off revlogs for our filelog storage (via remotefilelog),
>>>> and will likely need to avoid revlogs when we move to tree manifests as
>>>> As part of this, I've put together a design doc describing a high level
>>>> idea that would let us experiment with different storage backends, and
>>>> provides a path for migrating existing users over. It's currently
>>>> focused on situations like ours, where you have parts of the repository
>>>> on a central server and parts on the client, but the overall design may
>>>> be of interest to the community.
>>>> It's a bit light on concrete format details, since the main goal is to
>>>> put abstractions in place that would let us break away from the existing
>>>> formats and experiment.
>>>> Feel free to comment on the doc (you have to sign in to Quip to
>>>> comment), or respond by email.
>>> It'd be interesting if this was modular enough so that the server could
>>> offload it's storage to a CDN (or S3 or whatever).
> One of the goals is to enable look-aside revision fetching (like
> remotefilelog already has, where we fetch file contents from memcache)
> for all contents of the repository. So yea, from a client side this is
> a property we'll be aiming for. But it's more a side effect of going
> with a purely key/value based storage (instead of globally order revs).
> Not so much a side effect of the modular-ness.
Ah, I see.
>> For more context: trying to replace the need for largefiles.
> Yea, it could help. But there's still the problem of mercurial thinking
> it can load every file completely into memory, so it doesn't help the
> multi-GB large file problem so much.
Yeah, that's true.
More information about the Mercurial-devel