Allow largefiles to be at a different location - Implementation details

[Matt Harbison]

On Tue, 24 May 2016 07:03:45 -0400, Piotr Listkiewicz  
<piotr.listkiewicz at gmail.com> wrote:

>>
>> Unfortunately, I'm not familiar at all with the wire protocol.  It will
>> take me a few days to catch up.
>
>
> It would be awesome if you made an effort. BTW whats the best hours to  
> meet
> You in irc, what is Your nick?

I haven't made it onto IRC yet.  I would be around between 00:00 and 04:00  
GMT, give or take, depending on the day.

Is there a wiki or thumbnail sketch of what you are trying to do  
somewhere?  It might help me when looking at the protocol code.  I vaguely  
remember this being discussed at one point, but don't recall the issues or  
rejected ideas, aside from what you outlined here.

> That would rely on the other side closing the socket to signal EOF.  I'm
>> guessing there's some SSH or HTTP server or proxy that doesn't do that
>> reliably?
>
>
> I found the answer in proto.getlfile function:
>
> # Since we can't set an HTTP content-length header here, and
> # Mercurial core provides no way to give the length of a streamres
> # (and reading the entire file into RAM would be ill-advised), we
> # just send the length on the first line of the response, like the
> # ssh proto does for string responses.
>
> So i guess length must stay. So probably we must ditinguish between  
> getfile
> from server where it gets length at the beggining of file stream and
> getting file from different location where it reads it to the end of  
> stream.
>
>
> 2016-05-24 4:29 GMT+02:00 Matt Harbison <mharbison72 at gmail.com>:
>
>> On Mon, 23 May 2016 16:52:43 -0400, Piotr Listkiewicz <
>> piotr.listkiewicz at gmail.com> wrote:
>>
>> There are several ways to implement getting large files from different
>>> location and no matter which solution is chosen there is a problem when
>>> client uses hg that doesn't support new feature to connect to  
>>> repository
>>> with this feature enabled. I would like to know which of the following
>>> solutions you find best or do you have any comments etc:
>>>
>>
>> Unfortunately, I'm not familiar at all with the wire protocol.  It will
>> take me a few days to catch up.
>>
>> [snip]
>>
>>
>>> Any of those ideas have problems with knowing size of largefile, are  
>>> there
>>> any reason that proto.getlfile must include size at the beggining of  
>>> the
>>> stream - wouldn't it be easier if client just download all data from
>>> stream?
>>>
>>
>> That would rely on the other side closing the socket to signal EOF.  I'm
>> guessing there's some SSH or HTTP server or proxy that doesn't do that
>> reliably?