[PATCH] py3: write out hgextindex as bytes in setup.py

Gregory Szorc gregory.szorc at gmail.com
Fri Apr 5 18:59:42 EDT 2019

> On Apr 5, 2019, at 15:44, Matt Harbison <mharbison72 at gmail.com> wrote:
>> On Fri, 05 Apr 2019 18:38:54 -0400, Matt Harbison <mharbison72 at gmail.com> wrote:
>> # HG changeset patch
>> # User Matt Harbison <matt_harbison at yahoo.com>
>> # Date 1554503803 14400
>> #      Fri Apr 05 18:36:43 2019 -0400
>> # Node ID d8aef987cc88a5abead0c9cecf5e14066f161968
>> # Parent  a975821d0938615b8cb235f12cc6461a42dcfbd8
>> py3: write out hgextindex as bytes in setup.py
> There are subsequent problems that need to be resolved with py2exe.  First, it's
> complaining about py2 syntax in the vendored `concurrent` package.  Obviously we
> don't want that picked up in py3, and there's logic in setup.py to only add it
> under py2.  But if added explicitly to the exclude list, the build fails
> complaining there's no such package.
> Commenting that py2 syntax out, it then dies inside the custom module loader at
> `spec = finder.find_spec(..)`, complaining:
>    'VendorImporter' object has no attribute 'find_spec'
> Wrapping that in an exception handler that does nothing allows the build to
> complete, but hg.exe can't load dispatch.py from the zip file.  There's
> a TODO in the custom loader to support loading from zip files, but it seems
> to bail out before that point because `finder.find_spec()` returns None.  The
> loader can't be removed, because there's still something very early that needs
> to be treated as bytes.
> Presumably there's already a loader that knows how to read zip files that we
> could just delegate to somehow, but I really don't have a good understanding of
> how all these pieces fit together.  `sys.meta_path` at this point is:
>    [mercurial.hgpathentryfinder, _frozen_importlib.BuiltinImporter,
>     _frozen_importlib.FrozenImporter, _frozen_importlib_external.PathFinder]

The source transforming module importer is going to be “fun” to get working for py2exe. That’s because I believe py2exe is performing its own pyc compilation so the zip has only pyc files. I have doubts that the module compilation includes our custom source transformer. But I could be wrong.

We have a few paths forward...

a) coerce py2exe to work with our custom module importer. This entails either compiling transformed source into bytecode and teaching our importer to load it from the zip file. Or putting .py files in the zip and making our module importer work with that.

b) get rid of the source transforming module importer. (I was kinda hoping we’d be going down this path already. But doing so entails b’’ everywhere and we were ratholing in mass rewriting the repo.)

c) give up on py2exe and use PyOxidizer. (I’m still hacking on this, albeit slowly.) PyOxidizer is superior because it is faster and will be a self-contained binary. Mercurial is very much my main test case for PyOxidizer and I’ll add features to PyOxidizer to make it work with Mercurial.

More information about the Mercurial-devel mailing list