largefiles: server storage?

Justin Holewinski justin.holewinski at
Fri Oct 14 10:33:26 CDT 2011

On Fri, Oct 14, 2011 at 11:06 AM, Na'Tosha Bard <natosha at> wrote:

> 2011/10/14 Justin Holewinski <justin.holewinski at>
>> On Wed, Oct 12, 2011 at 1:05 PM, Justin Holewinski <
>> justin.holewinski at> wrote:
>>> I'm excited for the new largefiles extension, and I have been trying it
>>> out on some local test repositories.  I realize the extension (in its
>>> "official" form) is very new and subject to change, but I have a question on
>>> how the large files are (or are going to be) stored on the server.
>>> Let's say I have two local repositories S and C.  S represents the
>>> "server" repository, and C represents a client clone.  If I add large files
>>> to C and push to S, the large files appear to be stored in
>>> $HOME/.largefiles, not in the .hg directory for S.
> I believe this makes sense, based on the current implementation.
>>   It looks like S just contains hashes for the files, which makes sense.
> This is correct -- the hashes are sitting in S/.hglf -- correct?

They are stored in .hg/store/data/~2ehglf.  There appear to be .i files that
store all past revision hashes, and the .i files are stored in a structure
that mirrors the repository structure.  The client repo has the .hglf
directory.  If I run "hg update" on the server repo, then I get the .hglf
directory, and "hg update null" removes it.

Also, this may be a bug: in my test repository, I have all of the largefiles
in an assets/ directory.  If I run "hg update" on the server, this directory
is created.  But if I run "hg update null", then the contents of assets/ are
deleted, but the directory still remains, unlike other directories that
contain only normally-versioned files.

>>   Incidentally, will there be a config option for this, for users that
>>> wish to sandbox all hg-related files in a separate directory?
> Every large-files enabled repo will have it's own set of standins
> maintained in repo/.hglf -- I don't see any reason why this should be able
> to be moved out of the repository because it is repo-specific.  Also the
> standins are very small text files, so why do they need to be elsewhere?

I was actually referring to the opposite: will I be able to configure the
server to store all largefiles blobs in the .hg directory, or some other
user-configurable directory?

>> Now, let's say I create a new repository accessible over SSH, called S2.
>>>  If I push C to S2, the largefiles seem to be stored in *both*
>>> $HOME/.largefiles (in the SSH account) and the .hg directory for S2.  Things
>>> are getting a bit inconsistent.
> That does sound inconsistent -- to me, anyway.  There shouldn't be any
> largefiles in S2/.hg -- there should only be the textfiles with the SHA1
> sums in S2/.hglf -- is the S2/.hglf directory there?

There is no .hglf directory, the largefiles appear to be stored in
.hg/largefiles (in the server repo):

$ ls -l .hg/largefiles
total 71584
-rw------- 2 hg hg    71576 2011-10-12 12:49
-rw------- 2 hg hg 24376104 2011-10-12 12:49
-rw------- 2 hg hg     7821 2011-10-12 12:49
-rw------- 2 hg hg 24376128 2011-10-12 12:49
-rw------- 2 hg hg     7813 2011-10-12 12:49
-rw------- 2 hg hg 24376116 2011-10-12 12:49
-rw------- 2 hg hg    71567 2011-10-12 12:49

>> I have not tested HTTP/HTTPS, but what is the expected behavior in this
>>> case?  There may not be a writable home directory in this case.
>>> More specifically, what are the planned methods for storing large files
>>> on mercurial servers?
>> Ping?  Any comment from the largefiles devs on the planned server storage
>> model?
> I'm not really sure we have a concrete plan yet.  This extension (at least
> in this form) is very new.

Is it still going to be released with Mercurial 2.0?

> Some of us are expecting to use largefiles with Kiln, which just implements
> the server-side stuff already.  Some people will be migrating from the old
> bfiles extension, which means they already have a central share set up
> somewhere (but I assume some conversion will be necessary).  Greg is
> preparing a way for users to migrate from bfiles to largefiles, so he might
> have some idea on this.
> The built-in serving ability of largefiles was developed by the team at
> FogCreek, so hopefully one of them can reply with what their vision was.
> My initial thought is that:
> The $HOME/.largefiles cache should be configurable server-side, if it is
> not already
> Each repo should only contain the hashes in repo/.hglf -- when largefiles
> are uploaded, they should probably all go directly to the cache.

That makes sense to me, as long as the cache path is configurable. :)

> Cheers,
> Na'Tosha
> --
> *Na'Tosha Bard*
> Build & Infrastructure Developer | Unity Technologies
> *E-Mail:* natosha at
> *Skype:* natosha.bard



Justin Holewinski
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <>

More information about the Mercurial-devel mailing list