[PATCH 3 of 3 STABLE] py3: define and use json.loads polyfill

[Gregory Szorc]

On Sat, Nov 2, 2019 at 7:14 PM Yuya Nishihara <yuya at tcha.org> wrote:

> On Sat, 02 Nov 2019 12:19:58 -0700, Gregory Szorc wrote:
> > # HG changeset patch
> > # User Gregory Szorc <gregory.szorc at gmail.com>
> > # Date 1572721775 25200
> > #      Sat Nov 02 12:09:35 2019 -0700
> > # Branch stable
> > # Node ID b4a9220a4eb5a61891027d59cd0bded6dc5f7578
> > # Parent  0d0cd63ca1702b901df2cc021d1f51c77bc0bf61
> > py3: define and use json.loads polyfill
>
> Queued for stable, thanks.
>
> Just wondered if we can drop support for Python 3.5.
>

I would entertain dropping Python 3.5. Are there any popular distros
shipping it? We're good on RHEL and Debian/Ubuntu, I think...


>
> > +    # Python 3.5's json.load and json.loads require str. We polyfill its
> > +    # code for detecting encoding from bytes.
> > +    if sys.version_info[0:2] < (3, 6):
> > +
> > +        def _detect_encoding(b):
> > +            bstartswith = b.startswith
> > +            if bstartswith((codecs.BOM_UTF32_BE, codecs.BOM_UTF32_LE)):
> > +                return 'utf-32'
> > +            if bstartswith((codecs.BOM_UTF16_BE, codecs.BOM_UTF16_LE)):
> > +                return 'utf-16'
> > +            if bstartswith(codecs.BOM_UTF8):
> > +                return 'utf-8-sig'
> > +
> > +            if len(b) >= 4:
> > +                if not b[0]:
> > +                    # 00 00 -- -- - utf-32-be
> > +                    # 00 XX -- -- - utf-16-be
> > +                    return 'utf-16-be' if b[1] else 'utf-32-be'
> > +                if not b[1]:
> > +                    # XX 00 00 00 - utf-32-le
> > +                    # XX 00 00 XX - utf-16-le
> > +                    # XX 00 XX -- - utf-16-le
> > +                    return 'utf-16-le' if b[2] or b[3] else 'utf-32-le'
> > +            elif len(b) == 2:
> > +                if not b[0]:
> > +                    # 00 XX - utf-16-be
> > +                    return 'utf-16-be'
> > +                if not b[1]:
> > +                    # XX 00 - utf-16-le
> > +                    return 'utf-16-le'
> > +            # default
> > +            return 'utf-8'
>
> There should be no need to support encodings other than utf-8 (as Python
> 2.7
> wouldn't), but it's copy-pasta so I don't care.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.mercurial-scm.org/pipermail/mercurial-devel/attachments/20191102/a8138316/attachment.html>