D3303: cborutil: implement support for indefinite length CBOR types

[indygreg (Gregory Szorc)]

indygreg added inline comments.

INLINE COMMENTS

> yuja wrote in cborutil.py:73
> I don't think yielding `encoder.encode` would make much sense
> because an array item can also be a nested indefinite array, in
> which case, we can't use `writeitem()`.

Indeed.

Proper support for nesting will likely require a whole new high-level encoder API. Because state of the nesting needs to be tracked somewhere.

FWIW, the more I'm looking at the CBOR code, the more I'm thinking we will end up having to reinvent the full wheel. Not-yet-submitted commits to add wire protocol commands to do CBOR things are spending a *ton* of time in cbor2. The reason appears to be primarily driven by cbor2's insistence on using `write()`. There are a few places where we need to emit a generator of chunks. And the overhead from instantiating `io.BytesIO` instances to handle the `write()` from cbor2 only to call `getvalue()` to retrieve the data is non-trivial.

The next version of this may just invent a whole new CBOR encoder with only limited support for types. Or at least I'll change the API so a streaming array doesn't require an encoder be passed in.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D3303

To: indygreg, #hg-reviewers
Cc: yuja, mercurial-devel