Design document for allowing alternative client side storage

Sean Farley sean at farley.io
Thu Mar 10 22:02:43 EST 2016


Durham Goode <durham at fb.com> writes:

> On 3/10/16 6:13 PM, Sean Farley wrote:
>> Sean Farley <sean at farley.io> writes:
>>
>>> Durham Goode <durham at fb.com> writes:
>>>
>>>> We're going to be investigating alternative client side storage
>>>> strategies for Mercurial at Facebook over the next few months. We've
>>>> already moved off revlogs for our filelog storage (via remotefilelog),
>>>> and will likely need to avoid revlogs when we move to tree manifests as
>>>> well.
>>>>
>>>> As part of this, I've put together a design doc describing a high level
>>>> idea that would let us experiment with different storage backends, and
>>>> provides a path for migrating existing users over. It's currently
>>>> focused on situations like ours, where you have parts of the repository
>>>> on a central server and parts on the client, but the overall design may
>>>> be of interest to the community.
>>>>
>>>> https://quip.com/TFR2Aw0lu0LB
>>>>
>>>> It's a bit light on concrete format details, since the main goal is to
>>>> put abstractions in place that would let us break away from the existing
>>>> formats and experiment.
>>>>
>>>> Feel free to comment on the doc (you have to sign in to Quip to
>>>> comment), or respond by email.
>>> It'd be interesting if this was modular enough so that the server could
>>> offload it's storage to a CDN (or S3 or whatever).
> One of the goals is to enable look-aside revision fetching (like 
> remotefilelog already has, where we fetch file contents from memcache) 
> for all contents of the repository.  So yea, from a client side this is 
> a property we'll be aiming for. But it's more a side effect of going 
> with a purely key/value based storage (instead of globally order revs). 
> Not so much a side effect of the modular-ness.

Ah, I see.

>> For more context: trying to replace the need for largefiles.
> Yea, it could help.  But there's still the problem of mercurial thinking 
> it can load every file completely into memory, so it doesn't help the 
> multi-GB large file problem so much.

Yeah, that's true.


More information about the Mercurial-devel mailing list