Shipping 3.7 wheels

[Gregory Szorc]

On Thu, Jan 21, 2016 at 8:40 PM, Gregory Szorc <gregory.szorc at gmail.com>
wrote:

> On Wed, Jan 20, 2016 at 4:41 PM, Matt Mackall <mpm at selenic.com> wrote:
>
>> On Wed, 2016-01-20 at 15:53 -0800, Gregory Szorc wrote:
>> > On Wed, Jan 20, 2016 at 3:35 PM, Matt Mackall <mpm at selenic.com> wrote:
>> >
>> > > On Wed, 2016-01-20 at 13:46 -0800, Gregory Szorc wrote:
>> > > > On Wed, Jan 20, 2016 at 1:02 PM, Matt Mackall <mpm at selenic.com>
>> wrote:
>> > > >
>> > > > > On Mon, 2016-01-18 at 11:42 -0800, Gregory Szorc wrote:
>> > > > > > On Fri, Jan 15, 2016 at 11:25 AM, Matt Mackall <mpm at selenic.com
>> >
>> > > wrote:
>> > > > > >
>> > > > > > > On Thu, 2016-01-14 at 13:57 -0800, Gregory Szorc wrote:
>> > > > > > > > I /think/ packaging improvements in this cycle finally put
>> us in
>> > > > > position
>> > > > > > > > to offer binary wheels for Windows and OS X starting with
>> the 3.7
>> > > > > > > release.
>> > > > > > > > (We can't offer binary wheels for Linux because, well, there
>> > > still
>> > > > > isn't
>> > > > > > > a
>> > > > > > > > good answer for binary wheel compatibility on Linux.) The
>> > > advantage
>> > > > > of
>> > > > > > > > binary wheels is we can upload them to PyPI and people can
>> `pip
>> > > > > install
>> > > > > > > > Mercurial` and get the C extensions without needing to have
>> a
>> > > working
>> > > > > > > > compile environment.
>> > > > > > > >
>> > > > > > > > There /might/ be an issue with the hg version string. I
>> think
>> > > wheels
>> > > > > are
>> > > > > > > > more strict about the version format and may not like our
>> "+1"
>> > > > > > > annotation.
>> > > > > > > > This should be easily correctable with some setup.py
>> muckery.
>> > > > > > > >
>> > > > > > > > The bigger issue is how to produce and distribute them. I'm
>> not
>> > > sure
>> > > > > what
>> > > > > > > > changes to the release process need to be made to support
>> > > generating
>> > > > > > > wheels.
>> > > > > > > >
>> > > > > > > > What needs to be done for us to produce wheels and upload
>> them to
>> > > > > PyPI as
>> > > > > > > > part of the release process?
>> > > > > > >
>> > > > > > > I haven't the foggiest. I have a hacked[1] tool named twine
>> in my
>> > > > > release
>> > > > > > > script
>> > > > > > > and that's just about the sum total of my interaction with
>> PyPI
>> > > over
>> > > > > the
>> > > > > > > past
>> > > > > > > decade.
>> > > > > > >
>> > > > > > > Probably the best way forward is for someone (not it!) to
>> make a
>> > > test
>> > > > > > > package on PyPI to experiment with wheel uploads and report
>> back.
>> > > > > > >
>> > > > > > > [1] The capitalization of the package on PyPI and setup.py
>> disagree
>> > > > > > > causing stock twine to puke.
>> > > > > > >
>> > > > > > >
>> > > > > > I created an account on the test pypi service and uploaded
>> wheels for
>> > > > > > Windows. https://testpypi.python.org/pypi/mercurial
>> > > > > >
>> > > > > > In theory you can point pip at this PyPI instance, but I haven't
>> > > figured
>> > > > > > out how to do that yet. (`pip -i
>> https://testpypi.python.org/pypi/
>> > > > > install
>> > > > > > mercurial` is complaining with "Could not find a version that
>> > > satisfies
>> > > > > the
>> > > > > > requirement install (from versions: ) No matching distribution
>> found
>> > > for
>> > > > > > install"). But e.g. `pip install
>> > > > > >
>> > > > >
>> > >
>> https://testpypi.python.org/packages/cp27/m/mercurial/mercurial-3.7rc1-cp27-
>> > > > > no
>> > > > > > ne-win_amd64.whl`
>> > > > > > should work.
>> > > > > >
>> > > > > > Ping me with your test PyPI username and I'll add you as a
>> > > maintainer for
>> > > > > > the "mercurial" project there (they do wipe the database every
>> so
>> > > often).
>> > > > >
>> > > > > Umm.. why? I'm not able to/going to build wheels?
>> > > > >
>> > > > >
>> > > > The whole point of my original e-mail was to figure out how we would
>> > > start
>> > > > creating and distributing wheels to end users.
>> > >
>> > > I understood that. And my response was: I have no idea about PyPI at
>> all,
>> > > someone needs to figure it out.
>> > >
>> >
>> > Oh, I assumed you were uploading source archives to PyPI. I see from
>> > https://pypi.python.org/pypi/Mercurial/ that a few others have access.
>> So I
>> > guess it is one of them.
>>
>> No, it is me. But here is the full extent of my knowledge of and
>> interaction
>> with PyPI, as enshrined in my release script:
>>
>>  cd release-build-$1
>>  twine upload -s dist/mercurial-$1.tar.gz
>>
>> Previous Augie did it with a more manual process.
>>
>> > I wasn't suggesting you open a test account. I was just posting the
>> wheels
>> > so others could install them and report issues.
>>
>> Got it.
>>
>> > Would you feel comfortable giving me access rights to PyPI so I can
>> upload
>> > Windows wheels?
>>
>> Done. There should be a 3.7-rc tagged in an hour or so. You should add
>> yourself
>> to mercurial-packagers so you get the reminder email when releases happen.
>>
>
> Thanks.
>
> I uploaded Windows 32 and 64 bit wheels to PyPI. The packaging process
> munged "3.7-rc" to "3.7rc0." I'm not sure where this is happening. But I
> recall seeing a PEP establishing a versioning convention and I believe I
> heard something about pip/setuptools/distutils starting to adhere to this
> PEP. So that's probably it.
>
> I also got a HTTP 413 when using twine to upload the .whl files. So I used
> the web interface to upload.
>
> Weirdly, `pip.exe install Mercurial` will grab the .tar.gz instead of the
> wheels I uploaded. If I throw some -v -v -v flags to `pip install`, it
> appears this is due to the "3.7-rc" vs "3.7rc0" version conflict. It
> appears to favor "3.7-rc" over "3.7rc0" and ignores the wheels because they
> are an old version. This should "just work" for release builds. But we
> should figure out how to get the RC versions to align.
>
> If anyone knows about Python package versioning, it would be helpful...
>
>
PEP-0440 is what we're looking for.
https://www.python.org/dev/peps/pep-0440/#pre-release-separators is
interesting. Essentially "3.7-rc" is accepted, but it is normalized to
"3.7rc0" because the pre-release separator isn't used in the normal form
and there is an "implicit pre-release number" of 0. Now to figure out how
to stop this normalization during packaging. Or, we could change
Mercurial's versioning format to match the PEP-0440 normalized form...
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://selenic.com/pipermail/mercurial-devel/attachments/20160121/9b6a69f5/attachment.html>