Shipping 3.7 wheels

[Gregory Szorc]

On Thu, Jan 21, 2016 at 8:47 PM, Gregory Szorc <gregory.szorc at gmail.com>
wrote:

> On Thu, Jan 21, 2016 at 8:40 PM, Gregory Szorc <gregory.szorc at gmail.com>
> wrote:
>
>> On Wed, Jan 20, 2016 at 4:41 PM, Matt Mackall <mpm at selenic.com> wrote:
>>
>>> On Wed, 2016-01-20 at 15:53 -0800, Gregory Szorc wrote:
>>> > On Wed, Jan 20, 2016 at 3:35 PM, Matt Mackall <mpm at selenic.com> wrote:
>>> >
>>> > > On Wed, 2016-01-20 at 13:46 -0800, Gregory Szorc wrote:
>>> > > > On Wed, Jan 20, 2016 at 1:02 PM, Matt Mackall <mpm at selenic.com>
>>> wrote:
>>> > > >
>>> > > > > On Mon, 2016-01-18 at 11:42 -0800, Gregory Szorc wrote:
>>> > > > > > On Fri, Jan 15, 2016 at 11:25 AM, Matt Mackall <
>>> mpm at selenic.com>
>>> > > wrote:
>>> > > > > >
>>> > > > > > > On Thu, 2016-01-14 at 13:57 -0800, Gregory Szorc wrote:
>>> > > > > > > > I /think/ packaging improvements in this cycle finally put
>>> us in
>>> > > > > position
>>> > > > > > > > to offer binary wheels for Windows and OS X starting with
>>> the 3.7
>>> > > > > > > release.
>>> > > > > > > > (We can't offer binary wheels for Linux because, well,
>>> there
>>> > > still
>>> > > > > isn't
>>> > > > > > > a
>>> > > > > > > > good answer for binary wheel compatibility on Linux.) The
>>> > > advantage
>>> > > > > of
>>> > > > > > > > binary wheels is we can upload them to PyPI and people can
>>> `pip
>>> > > > > install
>>> > > > > > > > Mercurial` and get the C extensions without needing to
>>> have a
>>> > > working
>>> > > > > > > > compile environment.
>>> > > > > > > >
>>> > > > > > > > There /might/ be an issue with the hg version string. I
>>> think
>>> > > wheels
>>> > > > > are
>>> > > > > > > > more strict about the version format and may not like our
>>> "+1"
>>> > > > > > > annotation.
>>> > > > > > > > This should be easily correctable with some setup.py
>>> muckery.
>>> > > > > > > >
>>> > > > > > > > The bigger issue is how to produce and distribute them.
>>> I'm not
>>> > > sure
>>> > > > > what
>>> > > > > > > > changes to the release process need to be made to support
>>> > > generating
>>> > > > > > > wheels.
>>> > > > > > > >
>>> > > > > > > > What needs to be done for us to produce wheels and upload
>>> them to
>>> > > > > PyPI as
>>> > > > > > > > part of the release process?
>>> > > > > > >
>>> > > > > > > I haven't the foggiest. I have a hacked[1] tool named twine
>>> in my
>>> > > > > release
>>> > > > > > > script
>>> > > > > > > and that's just about the sum total of my interaction with
>>> PyPI
>>> > > over
>>> > > > > the
>>> > > > > > > past
>>> > > > > > > decade.
>>> > > > > > >
>>> > > > > > > Probably the best way forward is for someone (not it!) to
>>> make a
>>> > > test
>>> > > > > > > package on PyPI to experiment with wheel uploads and report
>>> back.
>>> > > > > > >
>>> > > > > > > [1] The capitalization of the package on PyPI and setup.py
>>> disagree
>>> > > > > > > causing stock twine to puke.
>>> > > > > > >
>>> > > > > > >
>>> > > > > > I created an account on the test pypi service and uploaded
>>> wheels for
>>> > > > > > Windows. https://testpypi.python.org/pypi/mercurial
>>> > > > > >
>>> > > > > > In theory you can point pip at this PyPI instance, but I
>>> haven't
>>> > > figured
>>> > > > > > out how to do that yet. (`pip -i
>>> https://testpypi.python.org/pypi/
>>> > > > > install
>>> > > > > > mercurial` is complaining with "Could not find a version that
>>> > > satisfies
>>> > > > > the
>>> > > > > > requirement install (from versions: ) No matching distribution
>>> found
>>> > > for
>>> > > > > > install"). But e.g. `pip install
>>> > > > > >
>>> > > > >
>>> > >
>>> https://testpypi.python.org/packages/cp27/m/mercurial/mercurial-3.7rc1-cp27-
>>> > > > > no
>>> > > > > > ne-win_amd64.whl`
>>> > > > > > should work.
>>> > > > > >
>>> > > > > > Ping me with your test PyPI username and I'll add you as a
>>> > > maintainer for
>>> > > > > > the "mercurial" project there (they do wipe the database every
>>> so
>>> > > often).
>>> > > > >
>>> > > > > Umm.. why? I'm not able to/going to build wheels?
>>> > > > >
>>> > > > >
>>> > > > The whole point of my original e-mail was to figure out how we
>>> would
>>> > > start
>>> > > > creating and distributing wheels to end users.
>>> > >
>>> > > I understood that. And my response was: I have no idea about PyPI at
>>> all,
>>> > > someone needs to figure it out.
>>> > >
>>> >
>>> > Oh, I assumed you were uploading source archives to PyPI. I see from
>>> > https://pypi.python.org/pypi/Mercurial/ that a few others have
>>> access. So I
>>> > guess it is one of them.
>>>
>>> No, it is me. But here is the full extent of my knowledge of and
>>> interaction
>>> with PyPI, as enshrined in my release script:
>>>
>>>  cd release-build-$1
>>>  twine upload -s dist/mercurial-$1.tar.gz
>>>
>>> Previous Augie did it with a more manual process.
>>>
>>> > I wasn't suggesting you open a test account. I was just posting the
>>> wheels
>>> > so others could install them and report issues.
>>>
>>> Got it.
>>>
>>> > Would you feel comfortable giving me access rights to PyPI so I can
>>> upload
>>> > Windows wheels?
>>>
>>> Done. There should be a 3.7-rc tagged in an hour or so. You should add
>>> yourself
>>> to mercurial-packagers so you get the reminder email when releases
>>> happen.
>>>
>>
>> Thanks.
>>
>> I uploaded Windows 32 and 64 bit wheels to PyPI. The packaging process
>> munged "3.7-rc" to "3.7rc0." I'm not sure where this is happening. But I
>> recall seeing a PEP establishing a versioning convention and I believe I
>> heard something about pip/setuptools/distutils starting to adhere to this
>> PEP. So that's probably it.
>>
>> I also got a HTTP 413 when using twine to upload the .whl files. So I
>> used the web interface to upload.
>>
>> Weirdly, `pip.exe install Mercurial` will grab the .tar.gz instead of the
>> wheels I uploaded. If I throw some -v -v -v flags to `pip install`, it
>> appears this is due to the "3.7-rc" vs "3.7rc0" version conflict. It
>> appears to favor "3.7-rc" over "3.7rc0" and ignores the wheels because they
>> are an old version. This should "just work" for release builds. But we
>> should figure out how to get the RC versions to align.
>>
>> If anyone knows about Python package versioning, it would be helpful...
>>
>>
> PEP-0440 is what we're looking for.
> https://www.python.org/dev/peps/pep-0440/#pre-release-separators is
> interesting. Essentially "3.7-rc" is accepted, but it is normalized to
> "3.7rc0" because the pre-release separator isn't used in the normal form
> and there is an "implicit pre-release number" of 0. Now to figure out how
> to stop this normalization during packaging. Or, we could change
> Mercurial's versioning format to match the PEP-0440 normalized form...
>

It appears setuptools.dist.Distribution.__init__() is normalizing the
version and isn't providing us the opportunity to override it. Annotate
says it is all PEP 0440 related.

We may need to change our versioning scheme to be in compliance with
PEP-0440 :/

Matt: how do you feel about rolling a "3.7rc1" tag?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://selenic.com/pipermail/mercurial-devel/attachments/20160121/504fec2d/attachment.html>