largefiles: server storage?
justin.holewinski at gmail.com
Fri Oct 14 10:33:26 CDT 2011
On Fri, Oct 14, 2011 at 11:06 AM, Na'Tosha Bard <natosha at unity3d.com> wrote:
> 2011/10/14 Justin Holewinski <justin.holewinski at gmail.com>
>> On Wed, Oct 12, 2011 at 1:05 PM, Justin Holewinski <
>> justin.holewinski at gmail.com> wrote:
>>> I'm excited for the new largefiles extension, and I have been trying it
>>> out on some local test repositories. I realize the extension (in its
>>> "official" form) is very new and subject to change, but I have a question on
>>> how the large files are (or are going to be) stored on the server.
>>> Let's say I have two local repositories S and C. S represents the
>>> "server" repository, and C represents a client clone. If I add large files
>>> to C and push to S, the large files appear to be stored in
>>> $HOME/.largefiles, not in the .hg directory for S.
> I believe this makes sense, based on the current implementation.
>> It looks like S just contains hashes for the files, which makes sense.
> This is correct -- the hashes are sitting in S/.hglf -- correct?
They are stored in .hg/store/data/~2ehglf. There appear to be .i files that
store all past revision hashes, and the .i files are stored in a structure
that mirrors the repository structure. The client repo has the .hglf
directory. If I run "hg update" on the server repo, then I get the .hglf
directory, and "hg update null" removes it.
Also, this may be a bug: in my test repository, I have all of the largefiles
in an assets/ directory. If I run "hg update" on the server, this directory
is created. But if I run "hg update null", then the contents of assets/ are
deleted, but the directory still remains, unlike other directories that
contain only normally-versioned files.
>> Incidentally, will there be a config option for this, for users that
>>> wish to sandbox all hg-related files in a separate directory?
> Every large-files enabled repo will have it's own set of standins
> maintained in repo/.hglf -- I don't see any reason why this should be able
> to be moved out of the repository because it is repo-specific. Also the
> standins are very small text files, so why do they need to be elsewhere?
I was actually referring to the opposite: will I be able to configure the
server to store all largefiles blobs in the .hg directory, or some other
>> Now, let's say I create a new repository accessible over SSH, called S2.
>>> If I push C to S2, the largefiles seem to be stored in *both*
>>> $HOME/.largefiles (in the SSH account) and the .hg directory for S2. Things
>>> are getting a bit inconsistent.
> That does sound inconsistent -- to me, anyway. There shouldn't be any
> largefiles in S2/.hg -- there should only be the textfiles with the SHA1
> sums in S2/.hglf -- is the S2/.hglf directory there?
There is no .hglf directory, the largefiles appear to be stored in
.hg/largefiles (in the server repo):
$ ls -l .hg/largefiles
-rw------- 2 hg hg 71576 2011-10-12 12:49
-rw------- 2 hg hg 24376104 2011-10-12 12:49
-rw------- 2 hg hg 7821 2011-10-12 12:49
-rw------- 2 hg hg 24376128 2011-10-12 12:49
-rw------- 2 hg hg 7813 2011-10-12 12:49
-rw------- 2 hg hg 24376116 2011-10-12 12:49
-rw------- 2 hg hg 71567 2011-10-12 12:49
>> I have not tested HTTP/HTTPS, but what is the expected behavior in this
>>> case? There may not be a writable home directory in this case.
>>> More specifically, what are the planned methods for storing large files
>>> on mercurial servers?
>> Ping? Any comment from the largefiles devs on the planned server storage
> I'm not really sure we have a concrete plan yet. This extension (at least
> in this form) is very new.
Is it still going to be released with Mercurial 2.0?
> Some of us are expecting to use largefiles with Kiln, which just implements
> the server-side stuff already. Some people will be migrating from the old
> bfiles extension, which means they already have a central share set up
> somewhere (but I assume some conversion will be necessary). Greg is
> preparing a way for users to migrate from bfiles to largefiles, so he might
> have some idea on this.
> The built-in serving ability of largefiles was developed by the team at
> FogCreek, so hopefully one of them can reply with what their vision was.
> My initial thought is that:
> The $HOME/.largefiles cache should be configurable server-side, if it is
> not already
> Each repo should only contain the hashes in repo/.hglf -- when largefiles
> are uploaded, they should probably all go directly to the cache.
That makes sense to me, as long as the cache path is configurable. :)
> *Na'Tosha Bard*
> Build & Infrastructure Developer | Unity Technologies
> *E-Mail:* natosha at unity3d.com
> *Skype:* natosha.bard
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the Mercurial-devel