D6754: contrib: proof of concept script to build Mac packages without system python

[mharbison72 (Matt Harbison)]

mharbison72 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  This isn't close to being ready- I'm just looking for ideas before I get too far
  off into the weeds.  It will build a *.pkg that does install, and both
  `hg version` and `hg debuginstall` look healthy.  As noted in the script though,
  there are still references to /Library/Frameworks/Python.framework (which is the
  python.org installed stuff), so it may not work on a fresh system.
  
  At this point, I'm wondering if we should just curl the source for OpenSSL and
  Python, and build the non-framework version with the destination set to the
  proper directory.  The packaging for thg does this (though it probably doesn't
  have to), and it takes about 10 minutes on a Mac Mini.  5 Minutes of that is
  running the tests on the python build, so maybe there's a way to shut that off.
  
  As far as alternatives, I started down the path of using virtualenv here, and
  shipping that.  I'm skeptical of the experimental nature of --relocatable, and
  grepping on the tree showed the build path in the *.pyc files.  I have no idea
  if that matters.  I left pip in place so that things like evolve could be
  installed, but it errored out when specifying `--user`.  I didn't look into why.
  
  I also looked at pyinstaller.  I couldn't figure out how to convince it to
  bundle the C extensions, and it appears to append its own output when the hg
  script exits with a non zero value.  I briefly looked at bbfreeze and cx_freeze,
  but didn't get as far.  I've had enough frustration with py2exe and py2app that
  I'm not wanting to struggle with yet another packager.
  
  Ultimately I think we should keep the target for building against system python
  (there was some special functionality unlocked with a specifically crafted dummy
  cert), and this should possibly be a different target.  (I don't see a way to
  switch between the two by simply overriding $PYTHON the way this is going.)  I
  basically copy/pasted the makefile as a starting point, but moved it into a
  shell script so the environment could be activated.  This either needs to be
  moved to contrib/, or folded back into the makefile.  I kinda lean towards the
  former, and have that called by make.  The readme in the installer will also
  have to be conditionalize/updated, since it mentions relying on system python.
  
  It would be nice to open up the range of supported platforms back to say, 10.9.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6754

AFFECTED FILES
  make_osx.sh

CHANGE DETAILS

diff --git a/make_osx.sh b/make_osx.sh
new file mode 100755
--- /dev/null
+++ b/make_osx.sh
@@ -0,0 +1,87 @@
+#!/bin/sh
+
+set -e
+
+# Extra args passed in, not version
+OSXVERSIONFLAGS=
+
+# TODO: default MACOSX_DEPLOYMENT_TARGET=10.9
+# TODO: adjust ${OSXVER} accordingly
+
+rm -rf build/mercurial
+python -m virtualenv build/mercurial/usr/local/mercurial
+source build/mercurial/usr/local/mercurial/bin/activate
+
+# TODO: Don't install this into the virtual environment. Also, use docutils-python3
+python -m pip install docutils
+
+python setup.py install --optimize=1  \
+	  --root=build/mercurial/ --prefix=/usr/local/mercurial
+
+make -C doc all install DESTDIR="`pwd`/build/mercurial/"
+
+# Place a bogon .DS_Store file in the target dir so we can be
+# sure it doesn't get included in the final package.
+touch build/mercurial/.DS_Store
+
+# install zsh completions - this location appears to be
+# searched by default as of macOS Sierra.
+install -d build/mercurial/usr/local/share/zsh/site-functions/
+install -m 0644 contrib/zsh_completion build/mercurial/usr/local/share/zsh/site-functions/_hg
+
+# install bash completions - there doesn't appear to be a
+# place that's searched by default for bash, so we'll follow
+# the lead of Apple's git install and just put it in a
+# location of our own.
+install -d build/mercurial/usr/local/hg/contrib/
+install -m 0644 contrib/bash_completion build/mercurial/usr/local/hg/contrib/hg-completion.bash
+
+make -C contrib/chg \
+	  HGPATH=/usr/local/bin/hg \
+	  PYTHON=python \
+	  HGEXTDIR=/usr/local/mercurial/lib/python2.7/site-packages/hgext \
+	  DESTDIR=../../build/mercurial \
+	  PREFIX=/usr/local \
+	  clean install
+
+# Keep pip around so that things like evolve can be installed.
+rm -f build/mercurial/usr/local/mercurial/bin/rst* \
+      build/mercurial/usr/local/mercurial/bin/activate* \
+      build/mercurial/usr/local/mercurial/bin/wheel \
+      build/mercurial/usr/local/mercurial/bin/easy_install
+
+# LANG=C' to avoid "sed: RE error: illegal byte sequence"
+LANG=C sed -i '' -e "s|`pwd`/build/mercurial||g" build/mercurial/usr/local/mercurial/bin/*
+
+# The virtualenv contains a symlink back to the real python install.  We can
+# either slim this down somehow (to avoid distributing whatever random packages
+# are already installed), or build python from source with a custom --dest.
+mkdir -p build/mercurial/usr/local/mercurial/Library/Frameworks/Python.framework/Versions
+cp -R /Library/Frameworks/Python.framework/Versions/2.7 build/mercurial/usr/local/mercurial/Library/Frameworks/Python.framework/Versions
+ln -s -f /usr/local/mercurial/Python.framework/Versions/2.7/Python build/mercurial/usr/local/mercurial/.Python
+
+# Running /usr/local/mercurial/Python.framework/Versions/2.7/bin/python2.7 and
+# printing sys.paths lists out stuff in /Library/Frameworks/Python.framework,
+# so this may not be totally detached from a standard python.org installation.
+# sys.exec_prefix and sys.prefix also point there.  That stuff appears to live
+# in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/_sysconfigdata.py.
+# The various *.pyc files that are copied over also have /Library/Frameworks strings
+# in them.  Then again, so do *.pyc files in a virtualenv.
+ln -s /usr/local/mercurial/bin/hg build/mercurial/usr/local/bin/hg
+
+mkdir -p ${OUTPUTDIR:-dist}
+
+HGVER=$(python2 contrib/genosxversion.py ${OSXVERSIONFLAGS} \
+       build/mercurial/usr/local/mercurial/lib/python2.7/site-packages/mercurial/__version__.py) && \
+OSXVER=$(sw_vers -productVersion | cut -d. -f1,2) && \
+PYVER=$(python -V 2>&1 | cut -d' ' -f2 | cut -d. -f1,2) && \
+pkgbuild --filter .DS_Store --root build/mercurial/ \
+  --identifier org.mercurial-scm.mercurial \
+  --version "${HGVER}" \
+  build/mercurial.pkg && \
+
+productbuild --distribution contrib/packaging/macosx/distribution.xml \
+  --package-path build/ \
+  --version "${HGVER}" \
+  --resources contrib/packaging/macosx/ \
+  "${OUTPUTDIR:-dist/}"/Mercurial-"${HGVER}"-py"${PYVER}"-macosx"${OSXVER}".pkg



To: mharbison72, #hg-reviewers
Cc: mercurial-devel