[PATCH 09 of 13] largefiles: getlfile must hit end of HTTP chunked streams to reuse connections

[Martin Geisler]

Mads Kiilerich <mads at kiilerich.com> writes:

> # HG changeset patch
> # User Mads Kiilerich <madski at unity3d.com>
> # Date 1366079710 -7200
> #      Tue Apr 16 04:35:10 2013 +0200
> # Node ID 22879b41a70b922fed215dcec5b8886c968658d7
> # Parent  0aaf7db2201975bcf9f5d82da481167b4c4c2ad6
> largefiles: getlfile must hit end of HTTP chunked streams to reuse connections
>
> We did read the exactly the right number of bytes from the response body. But
> if the response came in chunked encoding then that meant that the HTTP layer
> still hadn't read the last 0-sized chunk and expected the app layer to read
> more data from the stream. The app layer was however happy and sent another
> request which had to be sent on another HTTP connection while the old one was
> lingering until some other event closed the connection.
>
> Adding an extra read where we expect to hit the end of file makes the HTTP
> connection ready for reuse. This thus plugs a real socket leak.
>
> To distinguish HTTP from SSH we look at self's class, just like it is
> done in putlfile.

Would

  if isinstance(self, httppeer.httppeer)

not be equivalent and the usual way to write this? Or did you write it
like this to mimic putlfile?

> diff --git a/hgext/largefiles/proto.py b/hgext/largefiles/proto.py
> --- a/hgext/largefiles/proto.py
> +++ b/hgext/largefiles/proto.py
> @@ -126,6 +126,13 @@
>              # SSH streams will block if reading more than length
>              for chunk in util.filechunkiter(stream, 128 * 1024, length):
>                  yield chunk
> +            # HTTP streams must hit the end to process the last empty
> +            # chunk of Chunked-Encoding so the connection can be reused.
> +            if issubclass(self.__class__, httppeer.httppeer):
> +                chunk = stream.read(1)
> +                if chunk:
> +                    self._abort(error.ResponseError(_("unexpected response:"),
> +                                                    chunk))
>  
>          @batchable
>          def statlfile(self, sha):
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel at selenic.com
> http://selenic.com/mailman/listinfo/mercurial-devel

-- 
Martin Geisler