[PATCH STABLE V2] packaging: add make target for linux wheels
Boris Feld
boris.feld at octobus.net
Thu Apr 27 16:49:50 UTC 2017
# HG changeset patch
# User Boris Feld <boris.feld at octobus.net>
# Date 1493131801 -7200
# Tue Apr 25 16:50:01 2017 +0200
# Branch stable
# Node ID d49b79b313989b62482da8e560fafa3373ae90b5
# Parent 52cca17ac523b490acdab259e794e7c2898efeda
packaging: add make target for linux wheels
Having linux wheels is going to helps system without compiler or python-dev
plus speed up the installation for everyone.
I followed the manylinux example repository
https://github.com/pypa/python-manylinux-demo
to add a make target (build-linux-wheels) using
official docker image to build python 2 linux wheels
for mercurial. It generates Python 2.6 and Python 2.7 for both
32 and 64 bits architectures.
I had to blacklist several test cases for various reasons:
* test-convert-git.t and test-subrepo-git.t because of the git version
* test-patchbomb-tls.t because of warning using tls 1.0
It's likely because the docker image is based on centos 5.0 and
openssl is outdated.
diff -r 52cca17ac523 -r d49b79b31398 .hgignore
--- a/.hgignore Mon Apr 24 14:38:30 2017 -0400
+++ b/.hgignore Tue Apr 25 16:50:01 2017 +0200
@@ -62,6 +62,9 @@
mercurial/osutil.py
mercurial/parsers.py
+# Generated wheels
+wheelhouse/
+
syntax: regexp
^\.pc/
^\.(pydev)?project
diff -r 52cca17ac523 -r d49b79b31398 Makefile
--- a/Makefile Mon Apr 24 14:38:30 2017 -0400
+++ b/Makefile Tue Apr 25 16:50:01 2017 +0200
@@ -270,6 +270,14 @@
mkdir -p packages/centos7
contrib/dockerrpm centos7
+linux-wheels: linux-wheels-x86_64 linux-wheels-i686
+
+linux-wheels-x86_64:
+ docker run -e "HGTEST_JOBS=$(shell nproc)" --rm -ti -v `pwd`:/src quay.io/pypa/manylinux1_x86_64 /src/contrib/build-linux-wheels.sh
+
+linux-wheels-i686:
+ docker run -e "HGTEST_JOBS=$(shell nproc)" --rm -ti -v `pwd`:/src quay.io/pypa/manylinux1_i686 linux32 /src/contrib/build-linux-wheels.sh
+
.PHONY: help all local build doc cleanbutpackages clean install install-bin \
install-doc install-home install-home-bin install-home-doc \
dist dist-notests check tests check-code update-pot \
@@ -278,4 +286,5 @@
docker-ubuntu-xenial docker-ubuntu-xenial-ppa \
docker-ubuntu-yakkety docker-ubuntu-yakkety-ppa \
fedora20 docker-fedora20 fedora21 docker-fedora21 \
- centos5 docker-centos5 centos6 docker-centos6 centos7 docker-centos7
+ centos5 docker-centos5 centos6 docker-centos6 centos7 docker-centos7 \
+ linux-wheels
diff -r 52cca17ac523 -r d49b79b31398 contrib/build-linux-wheels.sh
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/contrib/build-linux-wheels.sh Tue Apr 25 16:50:01 2017 +0200
@@ -0,0 +1,34 @@
+#!/bin/bash
+# This file is directly inspired by
+# https://github.com/pypa/python-manylinux-demo/blob/master/travis/build-wheels.sh
+set -e -x
+
+PYTHON_TARGETS=$(ls -d /opt/python/cp2*/bin)
+
+# Create an user for the tests
+useradd hgbuilder
+
+# Bypass uid/gid problems
+cp -R /src /io && chown -R hgbuilder:hgbuilder /io
+
+# Compile wheels for Python 2.X
+for PYBIN in $PYTHON_TARGETS; do
+ "${PYBIN}/pip" wheel /io/ -w wheelhouse/
+done
+
+# Bundle external shared libraries into the wheels with
+# auditwheel (https://github.com/pypa/auditwheel) repair.
+# It also fix the ABI tag on the wheel making it pip installable.
+for whl in wheelhouse/*.whl; do
+ auditwheel repair "$whl" -w /src/wheelhouse/
+done
+
+# Install packages and run the tests for all Python versions
+cd /io/tests/
+
+for PYBIN in $PYTHON_TARGETS; do
+ # Install mercurial wheel as root
+ "${PYBIN}/pip" install mercurial --no-index -f /src/wheelhouse
+ # But run tests as hgbuilder user (non-root)
+ su hgbuilder -c "\"${PYBIN}/python\" /io/tests/run-tests.py --with-hg=\"${PYBIN}/hg\" --blacklist=/io/contrib/linux-wheel-centos5-blacklist"
+done
diff -r 52cca17ac523 -r d49b79b31398 contrib/linux-wheel-centos5-blacklist
--- /dev/null Thu Jan 01 00:00:00 1970 +0000
+++ b/contrib/linux-wheel-centos5-blacklist Tue Apr 25 16:50:01 2017 +0200
@@ -0,0 +1,3 @@
+test-convert-git.t
+test-subrepo-git.t
+test-patchbomb-tls.t
diff -r 52cca17ac523 -r d49b79b31398 tests/test-clonebundles.t
--- a/tests/test-clonebundles.t Mon Apr 24 14:38:30 2017 -0400
+++ b/tests/test-clonebundles.t Tue Apr 25 16:50:01 2017 +0200
@@ -61,7 +61,7 @@
$ echo "http://localhost:$HGPORT1/bundle.hg" > server/.hg/clonebundles.manifest
$ hg clone http://localhost:$HGPORT server-not-runner
applying clone bundle from http://localhost:$HGPORT1/bundle.hg
- error fetching bundle: (.* refused.*|Protocol not supported) (re)
+ error fetching bundle: (.* refused.*|Protocol not supported|Cannot assign requested address) (re)
abort: error applying bundle
(if this error persists, consider contacting the server operator or disable clone bundles via "--config ui.clonebundles=false")
[255]
diff -r 52cca17ac523 -r d49b79b31398 tests/test-http-bad-server.t
--- a/tests/test-http-bad-server.t Mon Apr 24 14:38:30 2017 -0400
+++ b/tests/test-http-bad-server.t Tue Apr 25 16:50:01 2017 +0200
@@ -36,6 +36,7 @@
$ hg clone http://localhost:$HGPORT/ clone
abort: error: Connection reset by peer (no-windows !)
abort: error: An existing connection was forcibly closed by the remote host (windows !)
+ (?)
[255]
(The server exits on its own, but there is a race between that and starting a new server.
@@ -223,7 +224,7 @@
readline(92 from -1) -> (41) content-type: application/mercurial-0.1\r\n
readline(51 from -1) -> (19) vary: X-HgProto-1\r\n
readline(32 from -1) -> (19) x-hgargs-post: 28\r\n
- readline(13 from -1) -> (13) x-hgproto-1:
+ readline(13 from -1) -> (13) x-hgproto-1:
read limit reached; closing socket
readline\(315 from (-1|65537)\) -> \(27\) POST /\?cmd=batch HTTP/1.1\\r\\n (re)
readline(288 from -1) -> (27) Accept-Encoding: identity\r\n
diff -r 52cca17ac523 -r d49b79b31398 tests/test-http-proxy.t
--- a/tests/test-http-proxy.t Mon Apr 24 14:38:30 2017 -0400
+++ b/tests/test-http-proxy.t Tue Apr 25 16:50:01 2017 +0200
@@ -87,7 +87,7 @@
misconfigured hosts)
$ http_proxy=localhost:$HGPORT2 hg clone --config http_proxy.always=True http://localhost:$HGPORT/ f
- abort: error: (Connection refused|Protocol not supported|.* actively refused it) (re)
+ abort: error: (Connection refused|Protocol not supported|.* actively refused it|Cannot assign requested address) (re)
[255]
do not use the proxy if it is in the no list
diff -r 52cca17ac523 -r d49b79b31398 tests/test-worker.t
--- a/tests/test-worker.t Mon Apr 24 14:38:30 2017 -0400
+++ b/tests/test-worker.t Tue Apr 25 16:50:01 2017 +0200
@@ -78,6 +78,7 @@
> test 100000.0 abort
start
abort: known exception
+ (?)
[255]
$ hg --config "extensions.t=$abspath" --config 'worker.numcpus=2' \
More information about the Mercurial-devel
mailing list