Design document for allowing alternative client side storage

Sean Farley sean at
Thu Mar 10 22:02:43 EST 2016

Durham Goode <durham at> writes:

> On 3/10/16 6:13 PM, Sean Farley wrote:
>> Sean Farley <sean at> writes:
>>> Durham Goode <durham at> writes:
>>>> We're going to be investigating alternative client side storage
>>>> strategies for Mercurial at Facebook over the next few months. We've
>>>> already moved off revlogs for our filelog storage (via remotefilelog),
>>>> and will likely need to avoid revlogs when we move to tree manifests as
>>>> well.
>>>> As part of this, I've put together a design doc describing a high level
>>>> idea that would let us experiment with different storage backends, and
>>>> provides a path for migrating existing users over. It's currently
>>>> focused on situations like ours, where you have parts of the repository
>>>> on a central server and parts on the client, but the overall design may
>>>> be of interest to the community.
>>>> It's a bit light on concrete format details, since the main goal is to
>>>> put abstractions in place that would let us break away from the existing
>>>> formats and experiment.
>>>> Feel free to comment on the doc (you have to sign in to Quip to
>>>> comment), or respond by email.
>>> It'd be interesting if this was modular enough so that the server could
>>> offload it's storage to a CDN (or S3 or whatever).
> One of the goals is to enable look-aside revision fetching (like 
> remotefilelog already has, where we fetch file contents from memcache) 
> for all contents of the repository.  So yea, from a client side this is 
> a property we'll be aiming for. But it's more a side effect of going 
> with a purely key/value based storage (instead of globally order revs). 
> Not so much a side effect of the modular-ness.

Ah, I see.

>> For more context: trying to replace the need for largefiles.
> Yea, it could help.  But there's still the problem of mercurial thinking 
> it can load every file completely into memory, so it doesn't help the 
> multi-GB large file problem so much.

Yeah, that's true.

More information about the Mercurial-devel mailing list