largefiles: server storage?

[Na'Tosha Bard]

2011/10/14 Justin Holewinski <justin.holewinski at gmail.com>

>
>
> On Wed, Oct 12, 2011 at 1:05 PM, Justin Holewinski <
> justin.holewinski at gmail.com> wrote:
>
>> I'm excited for the new largefiles extension, and I have been trying it
>> out on some local test repositories.  I realize the extension (in its
>> "official" form) is very new and subject to change, but I have a question on
>> how the large files are (or are going to be) stored on the server.
>>
>> Let's say I have two local repositories S and C.  S represents the
>> "server" repository, and C represents a client clone.  If I add large files
>> to C and push to S, the large files appear to be stored in
>> $HOME/.largefiles, not in the .hg directory for S.
>>
>
I believe this makes sense, based on the current implementation.


>  It looks like S just contains hashes for the files, which makes sense.
>>
>
This is correct -- the hashes are sitting in S/.hglf -- correct?


>  Incidentally, will there be a config option for this, for users that wish
>> to sandbox all hg-related files in a separate directory?
>>
>
Every large-files enabled repo will have it's own set of standins maintained
in repo/.hglf -- I don't see any reason why this should be able to be moved
out of the repository because it is repo-specific.  Also the standins are
very small text files, so why do they need to be elsewhere?


> Now, let's say I create a new repository accessible over SSH, called S2.
>>  If I push C to S2, the largefiles seem to be stored in *both*
>> $HOME/.largefiles (in the SSH account) and the .hg directory for S2.  Things
>> are getting a bit inconsistent.
>>
>
That does sound inconsistent -- to me, anyway.  There shouldn't be any
largefiles in S2/.hg -- there should only be the textfiles with the SHA1
sums in S2/.hglf -- is the S2/.hglf directory there?


> I have not tested HTTP/HTTPS, but what is the expected behavior in this
>> case?  There may not be a writable home directory in this case.
>>
>>
>> More specifically, what are the planned methods for storing large files on
>> mercurial servers?
>>
>
> Ping?  Any comment from the largefiles devs on the planned server storage
> model?
>

I'm not really sure we have a concrete plan yet.  This extension (at least
in this form) is very new.

Some of us are expecting to use largefiles with Kiln, which just implements
the server-side stuff already.  Some people will be migrating from the old
bfiles extension, which means they already have a central share set up
somewhere (but I assume some conversion will be necessary).  Greg is
preparing a way for users to migrate from bfiles to largefiles, so he might
have some idea on this.

The built-in serving ability of largefiles was developed by the team at
FogCreek, so hopefully one of them can reply with what their vision was.

My initial thought is that:

The $HOME/.largefiles cache should be configurable server-side, if it is not
already
Each repo should only contain the hashes in repo/.hglf -- when largefiles
are uploaded, they should probably all go directly to the cache.

Cheers,
Na'Tosha


-- 
*Na'Tosha Bard*
Build & Infrastructure Developer | Unity Technologies

*E-Mail:* natosha at unity3d.com
*Skype:* natosha.bard
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://selenic.com/pipermail/mercurial-devel/attachments/20111014/0e417c5c/attachment.html>