[PATCH STABLE] packaging: add make target for linux wheels

Boris Feld boris.feld at octobus.net
Wed Apr 26 17:01:37 UTC 2017


# HG changeset patch
# User Boris Feld <boris.feld at octobus.net>
# Date 1493131801 -7200
#      Tue Apr 25 16:50:01 2017 +0200
# Branch stable
# Node ID 9605a99df098d4e6e3b0943982a21fea6a9421f9
# Parent  6e0368b6e0bb2aa5210daec091c0200583553a78
packaging: add make target for linux wheels

Having linux wheels is going to helps system without compiler or python-dev
plus speed up the installation for everyone.

I followed the manylinux example repository
https://github.com/pypa/python-manylinux-demo
to add a make target (build-linux-wheels) using
official docker image to build python 2 linux wheels
for mercurial. It generates Python 2.6 and Python 2.7 for both
32 and 64 bits architectures.

I had to blacklist several test cases for various reasons:
* test-http-bad-server.t, readline(65537) was transformed in readline(-1)
* test-convert-git.t and test-subrepo-git.t because of the git version
* test-patchbomb-tls.t because of warning using tls 1.0
  It's likely because the docker image is based on centos 5.0 and
  openssl is outdated.
* test-clonebundles.t and test-http-proxy.t because of the following
  output line that changed:
  -  error fetching bundle: (.* refused.*|Protocol not supported) (re)
  +  error fetching bundle: Cannot assign requested address

diff -r 6e0368b6e0bb -r 9605a99df098 .hgignore
--- a/.hgignore	Thu Apr 20 22:51:28 2017 +0900
+++ b/.hgignore	Tue Apr 25 16:50:01 2017 +0200
@@ -62,6 +62,9 @@
 mercurial/osutil.py
 mercurial/parsers.py
 
+# Generated wheels
+wheelhouse/
+
 syntax: regexp
 ^\.pc/
 ^\.(pydev)?project
diff -r 6e0368b6e0bb -r 9605a99df098 Makefile
--- a/Makefile	Thu Apr 20 22:51:28 2017 +0900
+++ b/Makefile	Tue Apr 25 16:50:01 2017 +0200
@@ -270,6 +270,14 @@
 	mkdir -p packages/centos7
 	contrib/dockerrpm centos7
 
+linux-wheels: linux-wheels-x86_64 linux-wheels-i686
+	
+linux-wheels-x86_64:
+	docker run -e "HGTEST_JOBS=$(shell nproc)" --rm -ti -v `pwd`:/src quay.io/pypa/manylinux1_x86_64 /src/contrib/build-linux-wheels.sh
+	
+linux-wheels-i686:
+	docker run -e "HGTEST_JOBS=$(shell nproc)" --rm -ti -v `pwd`:/src quay.io/pypa/manylinux1_i686 linux32 /src/contrib/build-linux-wheels.sh
+
 .PHONY: help all local build doc cleanbutpackages clean install install-bin \
 	install-doc install-home install-home-bin install-home-doc \
 	dist dist-notests check tests check-code update-pot \
@@ -278,4 +286,5 @@
 	docker-ubuntu-xenial docker-ubuntu-xenial-ppa \
 	docker-ubuntu-yakkety docker-ubuntu-yakkety-ppa \
 	fedora20 docker-fedora20 fedora21 docker-fedora21 \
-	centos5 docker-centos5 centos6 docker-centos6 centos7 docker-centos7
+	centos5 docker-centos5 centos6 docker-centos6 centos7 docker-centos7 \
+	linux-wheels
diff -r 6e0368b6e0bb -r 9605a99df098 contrib/build-linux-wheels.sh
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/contrib/build-linux-wheels.sh	Tue Apr 25 16:50:01 2017 +0200
@@ -0,0 +1,34 @@
+#!/bin/bash
+# This file is directly inspired by
+# https://github.com/pypa/python-manylinux-demo/blob/master/travis/build-wheels.sh
+set -e -x
+
+PYTHON_TARGETS=$(ls -d /opt/python/cp2*/bin)
+
+# Create an user for the tests
+useradd hgbuilder
+
+# Bypass uid/gid problems
+cp -R /src /io && chown -R hgbuilder:hgbuilder /io
+
+# Compile wheels for Python 2.X
+for PYBIN in $PYTHON_TARGETS; do
+    "${PYBIN}/pip" wheel /io/ -w wheelhouse/
+done
+
+# Bundle external shared libraries into the wheels with
+# auditwheel (https://github.com/pypa/auditwheel) repair.
+# It also fix the ABI tag on the wheel making it pip installable.
+for whl in wheelhouse/*.whl; do
+    auditwheel repair "$whl" -w /src/wheelhouse/
+done
+
+# Install packages and run the tests for all Python versions
+cd /io/tests/
+
+for PYBIN in $PYTHON_TARGETS; do
+    # Install mercurial wheel as root
+    "${PYBIN}/pip" install mercurial --no-index -f /src/wheelhouse
+    # But run tests as hgbuilder user (non-root)
+    su hgbuilder -c "\"${PYBIN}/python\" /io/tests/run-tests.py --with-hg=\"${PYBIN}/hg\" --blacklist=/io/contrib/linux-wheel-centos5-blacklist"
+done
diff -r 6e0368b6e0bb -r 9605a99df098 contrib/linux-wheel-centos5-blacklist
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/contrib/linux-wheel-centos5-blacklist	Tue Apr 25 16:50:01 2017 +0200
@@ -0,0 +1,6 @@
+test-http-bad-server.t
+test-convert-git.t
+test-subrepo-git.t
+test-patchbomb-tls.t
+test-clonebundles.t
+test-http-proxy.t


More information about the Mercurial-devel mailing list