[PATCH 07 of 11] py3: byteify the decoded JSON responses upon receipt in the LFS blobstore

[Matt Harbison]

On Mon, 28 Jan 2019 06:59:44 -0500, Yuya Nishihara <yuya at tcha.org> wrote:

> On Mon, 28 Jan 2019 00:20:53 -0500, Matt Harbison wrote:
>> # HG changeset patch
>> # User Matt Harbison <matt_harbison at yahoo.com>
>> # Date 1548629295 18000
>> #      Sun Jan 27 17:48:15 2019 -0500
>> # Node ID b98988169d4a9c7890b93091683fa4ec38d61a47
>> # Parent  1d6f4c32abc28ea54e3d1d8487a1d773033aedf0
>> py3: byteify the decoded JSON responses upon receipt in the LFS  
>> blobstore
>
>> -        return response
>> +        def encodestr(x):
>> +            if isinstance(x, unicode):
>
> Fixed s/unicode/pycompat.unicode/ in flight.
>
>> +                return x.encode(u'utf-8')
>> +            return x
>> +
>> +        return pycompat.rapply(encodestr, response)
>
> I assume here JSON strings are encoding agnostic (i.e. ASCII.) If the  
> JSON
> had a filename for example, it wouldn't be always correct to encode data  
> as
> UTF-8.

Somewhere along the line, I got it in my head that the spec explicitly  
said the JSON payload was utf-8 encoded.  Of course, I can't find that  
now, and the test-lfs-test-server.t#git-server output doesn't have a  
charset in the header for the JSON exchange.  That said, some  
clients/servers do[1]:

   > POST /systec_bin4.git/objects/batch HTTP/1.1
   > Host: 172.17.31.252:9999
   > Accept: application/vnd.git-lfs+json; charset=utf-8
   > Authorization: RemoteAuth  ........secret key...................
   > Content-Length: 216
   > Content-Type: application/vnd.git-lfs+json; charset=utf-8
   > User-Agent: git-lfs/2.2.1 (GitHub; windows amd64; go 1.8.3; git  
621d1f82)
   >
   {... JSON ...}trace git-lfs: HTTP: 401


   < HTTP/1.1 401 Unauthorized
   < Content-Length: 26
   < Content-Type: text/plain; charset=utf-8
   < Date: Wed, 27 Sep 2017 06:55:39 GMT
   < Www-Authenticate: Basic realm=git-lfs-server

Is it worth trying to use that header info, if it exists?

There wouldn't be any real file names in the payload- just URLs containing  
'.hg/lfs/objects' and the oid.  The schemas are here:

https://github.com/git-lfs/git-lfs/blob/master/tq/schemas/http-batch-request-schema.json
https://github.com/git-lfs/git-lfs/blob/master/tq/schemas/http-batch-response-schema.json

[1] https://github.com/git-lfs/git-lfs/issues/2618