D6097: wix: functionality to automate building WiX installers

indygreg (Gregory Szorc) phabricator at mercurial-scm.org
Fri Mar 8 13:50:38 EST 2019


indygreg created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  Like we did for Inno Setup, we want to make it easier to
  produce WiX installers. This commit does that.
  
  We introduce a new hgpackaging.wix module for performing
  all the high-level tasks required to produce WiX installers.
  This required miscellaneous enhancements to existing code in
  hgpackaging, including support for signing binaries.
  
  A new build.py script for calling into the module APIs has been
  created. It behaves very similarly to the Inno Setup build.py
  script.
  
  Unlike Inno Setup, we didn't have code in the repo previously
  to generate WiX installers. It appears that all existing
  automation for building WiX installers lives in the
  https://bitbucket.org/tortoisehg/thg-winbuild repository - most
  notably in its setup.py file. My strategy for inventing the
  code in this commit was to step through the code in that repo's
  setup.py and observe what it was doing. Despite the length of
  setup.py in that repository, the actual amount of steps required
  to produce a WiX installer is actually quite low. It consists
  of a basic py2exe build plus invocations of candle.exe and
  light.exe to produce the MSI.
  
  One rabbit hole that gave me fits was locating the Visual Studio
  9 C Runtime merge modules. These merge modules are only present
  on your system if you have a full Visual Studio 2008 installation.
  Fortunately, I have a copy of Visual Studio 2008 and was able
  to install all the required updates. I then uploaded these merge
  modules to a personal repository on GitHub. That is where the
  added code references them from. We probably don't need to
  ship the merge modules. But that is for another day.
  
  The installs from the MSIs produced with the new automation
  differ from the last official MSI in the following ways:
  
  - Our HTML manual pages have UNIX line endings instead of Windows.
  - We ship modules in the mercurial.pure package. It appears the upstream packaging code is not including this package due to omission (they supply an explicit list of packages that has drifted out of sync with our setup.py).
  - We do not ship various distutils.* modules. This is because virtualenvs have a custom distutils/__init__.py that automagically imports distutils from its original location and py2exe gets confused by this. We don't use distutils in core Mercurial and don't provide a usable python.exe, so this omission should be acceptable.
  - The version of the enum package is different and we ship an enum.pyc instead of an enum/__init__.py.
  - The version of the docutils package is different and we ship a different set of files.
  - The version of Sphinx is drastically newer and we ship a number of files the old version did not. (I'm not sure why we ship Sphinx - I think it is a side-effect of the way the THG code was installing dependencies.)
  - We ship the idna package (dependent of requests which is a dependency of newer versions of Sphinx).
  - The version of imagesize is different and we ship an imagesize.pyc instead of an imagesize/__init__.pyc.
  - The version of the jinja2 package is different and the sets of files differs.
  - We ship the packaging package, which is a dependency for Sphinx.
  - The version of the pygments package is different and the sets of files differs.
  - We ship the requests package, which is a dependency for Sphinx.
  - We ship the snowballstemmer package, which is a dependency for Sphinx.
  - We ship the urllib3 package, which is a dependency for requests, which is a dependency for Sphinx.
  - We ship a newer version of the futures package, which includes a handful of extra modules that match Python 3 module names.
  
  1. no-check-commit because foo_bar naming

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6097

AFFECTED FILES
  contrib/packaging/hgpackaging/downloads.py
  contrib/packaging/hgpackaging/util.py
  contrib/packaging/hgpackaging/wix.py
  contrib/packaging/wix/README.txt
  contrib/packaging/wix/build.py
  contrib/packaging/wix/readme.rst
  contrib/packaging/wix/requirements.txt
  contrib/packaging/wix/requirements.txt.in
  tests/test-check-code.t
  tests/test-check-py3-compat.t

CHANGE DETAILS

diff --git a/tests/test-check-py3-compat.t b/tests/test-check-py3-compat.t
--- a/tests/test-check-py3-compat.t
+++ b/tests/test-check-py3-compat.t
@@ -7,6 +7,7 @@
   $ testrepohg files 'set:(**.py)' \
   > -X contrib/packaging/hgpackaging/ \
   > -X contrib/packaging/inno/ \
+  > -X contrib/packaging/wix/ \
   > -X hgdemandimport/demandimportpy2.py \
   > -X mercurial/thirdparty/cbor \
   > | sed 's|\\|/|g' | xargs "$PYTHON" contrib/check-py3-compat.py
diff --git a/tests/test-check-code.t b/tests/test-check-code.t
--- a/tests/test-check-code.t
+++ b/tests/test-check-code.t
@@ -16,7 +16,9 @@
   Skipping contrib/packaging/hgpackaging/inno.py it has no-che?k-code (glob)
   Skipping contrib/packaging/hgpackaging/py2exe.py it has no-che?k-code (glob)
   Skipping contrib/packaging/hgpackaging/util.py it has no-che?k-code (glob)
+  Skipping contrib/packaging/hgpackaging/wix.py it has no-che?k-code (glob)
   Skipping contrib/packaging/inno/build.py it has no-che?k-code (glob)
+  Skipping contrib/packaging/wix/build.py it has no-che?k-code (glob)
   Skipping i18n/polib.py it has no-che?k-code (glob)
   Skipping mercurial/statprof.py it has no-che?k-code (glob)
   Skipping tests/badserverext.py it has no-che?k-code (glob)
diff --git a/contrib/packaging/wix/requirements.txt.in b/contrib/packaging/wix/requirements.txt.in
new file mode 100644
--- /dev/null
+++ b/contrib/packaging/wix/requirements.txt.in
@@ -0,0 +1,6 @@
+docutils
+enum
+future
+pygments
+pypiwin32
+sphinx
diff --git a/contrib/packaging/wix/requirements.txt b/contrib/packaging/wix/requirements.txt
new file mode 100644
--- /dev/null
+++ b/contrib/packaging/wix/requirements.txt
@@ -0,0 +1,132 @@
+#
+# This file is autogenerated by pip-compile
+# To update, run:
+#
+#    pip-compile --generate-hashes contrib/packaging/wix/requirements.txt.in -o contrib/packaging/wix/requirements.txt -U
+#
+alabaster==0.7.12 \
+    --hash=sha256:446438bdcca0e05bd45ea2de1668c1d9b032e1a9154c2c259092d77031ddd359 \
+    --hash=sha256:a661d72d58e6ea8a57f7a86e37d86716863ee5e92788398526d58b26a4e4dc02 \
+    # via sphinx
+babel==2.6.0 \
+    --hash=sha256:6778d85147d5d85345c14a26aada5e478ab04e39b078b0745ee6870c2b5cf669 \
+    --hash=sha256:8cba50f48c529ca3fa18cf81fa9403be176d374ac4d60738b839122dfaaa3d23 \
+    # via sphinx
+certifi==2018.11.29 \
+    --hash=sha256:47f9c83ef4c0c621eaef743f133f09fa8a74a9b75f037e8624f83bd1b6626cb7 \
+    --hash=sha256:993f830721089fef441cdfeb4b2c8c9df86f0c63239f06bd025a76a7daddb033 \
+    # via requests
+chardet==3.0.4 \
+    --hash=sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae \
+    --hash=sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691 \
+    # via requests
+colorama==0.4.1 \
+    --hash=sha256:05eed71e2e327246ad6b38c540c4a3117230b19679b875190486ddd2d721422d \
+    --hash=sha256:f8ac84de7840f5b9c4e3347b3c1eaa50f7e49c2b07596221daec5edaabbd7c48 \
+    # via sphinx
+docutils==0.14 \
+    --hash=sha256:02aec4bd92ab067f6ff27a38a38a41173bf01bed8f89157768c1573f53e474a6 \
+    --hash=sha256:51e64ef2ebfb29cae1faa133b3710143496eca21c530f3f71424d77687764274 \
+    --hash=sha256:7a4bd47eaf6596e1295ecb11361139febe29b084a87bf005bf899f9a42edc3c6
+enum==0.4.7 \
+    --hash=sha256:8c7cf3587eda51008bcc1eed99ea2c331ccd265c231dbaa95ec5258d3dc03100
+future==0.17.1 \
+    --hash=sha256:67045236dcfd6816dc439556d009594abf643e5eb48992e36beac09c2ca659b8
+idna==2.8 \
+    --hash=sha256:c357b3f628cf53ae2c4c05627ecc484553142ca23264e593d327bcde5e9c3407 \
+    --hash=sha256:ea8b7f6188e6fa117537c3df7da9fc686d485087abf6ac197f9c46432f7e4a3c \
+    # via requests
+imagesize==1.1.0 \
+    --hash=sha256:3f349de3eb99145973fefb7dbe38554414e5c30abd0c8e4b970a7c9d09f3a1d8 \
+    --hash=sha256:f3832918bc3c66617f92e35f5d70729187676313caa60c187eb0f28b8fe5e3b5 \
+    # via sphinx
+jinja2==2.10 \
+    --hash=sha256:74c935a1b8bb9a3947c50a54766a969d4846290e1e788ea44c1392163723c3bd \
+    --hash=sha256:f84be1bb0040caca4cea721fcbbbbd61f9be9464ca236387158b0feea01914a4 \
+    # via sphinx
+markupsafe==1.1.1 \
+    --hash=sha256:00bc623926325b26bb9605ae9eae8a215691f33cae5df11ca5424f06f2d1f473 \
+    --hash=sha256:09027a7803a62ca78792ad89403b1b7a73a01c8cb65909cd876f7fcebd79b161 \
+    --hash=sha256:09c4b7f37d6c648cb13f9230d847adf22f8171b1ccc4d5682398e77f40309235 \
+    --hash=sha256:1027c282dad077d0bae18be6794e6b6b8c91d58ed8a8d89a89d59693b9131db5 \
+    --hash=sha256:24982cc2533820871eba85ba648cd53d8623687ff11cbb805be4ff7b4c971aff \
+    --hash=sha256:29872e92839765e546828bb7754a68c418d927cd064fd4708fab9fe9c8bb116b \
+    --hash=sha256:43a55c2930bbc139570ac2452adf3d70cdbb3cfe5912c71cdce1c2c6bbd9c5d1 \
+    --hash=sha256:46c99d2de99945ec5cb54f23c8cd5689f6d7177305ebff350a58ce5f8de1669e \
+    --hash=sha256:500d4957e52ddc3351cabf489e79c91c17f6e0899158447047588650b5e69183 \
+    --hash=sha256:535f6fc4d397c1563d08b88e485c3496cf5784e927af890fb3c3aac7f933ec66 \
+    --hash=sha256:62fe6c95e3ec8a7fad637b7f3d372c15ec1caa01ab47926cfdf7a75b40e0eac1 \
+    --hash=sha256:6dd73240d2af64df90aa7c4e7481e23825ea70af4b4922f8ede5b9e35f78a3b1 \
+    --hash=sha256:717ba8fe3ae9cc0006d7c451f0bb265ee07739daf76355d06366154ee68d221e \
+    --hash=sha256:79855e1c5b8da654cf486b830bd42c06e8780cea587384cf6545b7d9ac013a0b \
+    --hash=sha256:7c1699dfe0cf8ff607dbdcc1e9b9af1755371f92a68f706051cc8c37d447c905 \
+    --hash=sha256:88e5fcfb52ee7b911e8bb6d6aa2fd21fbecc674eadd44118a9cc3863f938e735 \
+    --hash=sha256:8defac2f2ccd6805ebf65f5eeb132adcf2ab57aa11fdf4c0dd5169a004710e7d \
+    --hash=sha256:98c7086708b163d425c67c7a91bad6e466bb99d797aa64f965e9d25c12111a5e \
+    --hash=sha256:9add70b36c5666a2ed02b43b335fe19002ee5235efd4b8a89bfcf9005bebac0d \
+    --hash=sha256:9bf40443012702a1d2070043cb6291650a0841ece432556f784f004937f0f32c \
+    --hash=sha256:ade5e387d2ad0d7ebf59146cc00c8044acbd863725f887353a10df825fc8ae21 \
+    --hash=sha256:b00c1de48212e4cc9603895652c5c410df699856a2853135b3967591e4beebc2 \
+    --hash=sha256:b1282f8c00509d99fef04d8ba936b156d419be841854fe901d8ae224c59f0be5 \
+    --hash=sha256:b2051432115498d3562c084a49bba65d97cf251f5a331c64a12ee7e04dacc51b \
+    --hash=sha256:ba59edeaa2fc6114428f1637ffff42da1e311e29382d81b339c1817d37ec93c6 \
+    --hash=sha256:c8716a48d94b06bb3b2524c2b77e055fb313aeb4ea620c8dd03a105574ba704f \
+    --hash=sha256:cd5df75523866410809ca100dc9681e301e3c27567cf498077e8551b6d20e42f \
+    --hash=sha256:e249096428b3ae81b08327a63a485ad0878de3fb939049038579ac0ef61e17e7 \
+    # via jinja2
+packaging==19.0 \
+    --hash=sha256:0c98a5d0be38ed775798ece1b9727178c4469d9c3b4ada66e8e6b7849f8732af \
+    --hash=sha256:9e1cbf8c12b1f1ce0bb5344b8d7ecf66a6f8a6e91bcb0c84593ed6d3ab5c4ab3 \
+    # via sphinx
+pygments==2.3.1 \
+    --hash=sha256:5ffada19f6203563680669ee7f53b64dabbeb100eb51b61996085e99c03b284a \
+    --hash=sha256:e8218dd399a61674745138520d0d4cf2621d7e032439341bc3f647bff125818d
+pyparsing==2.3.1 \
+    --hash=sha256:66c9268862641abcac4a96ba74506e594c884e3f57690a696d21ad8210ed667a \
+    --hash=sha256:f6c5ef0d7480ad048c054c37632c67fca55299990fff127850181659eea33fc3 \
+    # via packaging
+pypiwin32==223 \
+    --hash=sha256:67adf399debc1d5d14dffc1ab5acacb800da569754fafdc576b2a039485aa775 \
+    --hash=sha256:71be40c1fbd28594214ecaecb58e7aa8b708eabfa0125c8a109ebd51edbd776a
+pytz==2018.9 \
+    --hash=sha256:32b0891edff07e28efe91284ed9c31e123d84bea3fd98e1f72be2508f43ef8d9 \
+    --hash=sha256:d5f05e487007e29e03409f9398d074e158d920d36eb82eaf66fb1136b0c5374c \
+    # via babel
+pywin32==224 \
+    --hash=sha256:22e218832a54ed206452c8f3ca9eff07ef327f8e597569a4c2828be5eaa09a77 \
+    --hash=sha256:32b37abafbfeddb0fe718008d6aada5a71efa2874f068bee1f9e703983dcc49a \
+    --hash=sha256:35451edb44162d2f603b5b18bd427bc88fcbc74849eaa7a7e7cfe0f507e5c0c8 \
+    --hash=sha256:4eda2e1e50faa706ff8226195b84fbcbd542b08c842a9b15e303589f85bfb41c \
+    --hash=sha256:5f265d72588806e134c8e1ede8561739071626ea4cc25c12d526aa7b82416ae5 \
+    --hash=sha256:6852ceac5fdd7a146b570655c37d9eacd520ed1eaeec051ff41c6fc94243d8bf \
+    --hash=sha256:6dbc4219fe45ece6a0cc6baafe0105604fdee551b5e876dc475d3955b77190ec \
+    --hash=sha256:9bd07746ce7f2198021a9fa187fa80df7b221ec5e4c234ab6f00ea355a3baf99 \
+    # via pypiwin32
+requests==2.21.0 \
+    --hash=sha256:502a824f31acdacb3a35b6690b5fbf0bc41d63a24a45c4004352b0242707598e \
+    --hash=sha256:7bf2a778576d825600030a110f3c0e3e8edc51dfaafe1c146e39a2027784957b \
+    # via sphinx
+six==1.12.0 \
+    --hash=sha256:3350809f0555b11f552448330d0b52d5f24c91a322ea4a15ef22629740f3761c \
+    --hash=sha256:d16a0141ec1a18405cd4ce8b4613101da75da0e9a7aec5bdd4fa804d0e0eba73 \
+    # via packaging, sphinx
+snowballstemmer==1.2.1 \
+    --hash=sha256:919f26a68b2c17a7634da993d91339e288964f93c274f1343e3bbbe2096e1128 \
+    --hash=sha256:9f3bcd3c401c3e862ec0ebe6d2c069ebc012ce142cce209c098ccb5b09136e89 \
+    # via sphinx
+sphinx==1.8.4 \
+    --hash=sha256:b53904fa7cb4b06a39409a492b949193a1b68cc7241a1a8ce9974f86f0d24287 \
+    --hash=sha256:c1c00fc4f6e8b101a0d037065043460dffc2d507257f2f11acaed71fd2b0c83c
+sphinxcontrib-websupport==1.1.0 \
+    --hash=sha256:68ca7ff70785cbe1e7bccc71a48b5b6d965d79ca50629606c7861a21b206d9dd \
+    --hash=sha256:9de47f375baf1ea07cdb3436ff39d7a9c76042c10a769c52353ec46e4e8fc3b9 \
+    # via sphinx
+typing==3.6.6 \
+    --hash=sha256:4027c5f6127a6267a435201981ba156de91ad0d1d98e9ddc2aa173453453492d \
+    --hash=sha256:57dcf675a99b74d64dacf6fba08fb17cf7e3d5fdff53d4a30ea2a5e7e52543d4 \
+    --hash=sha256:a4c8473ce11a65999c8f59cb093e70686b6c84c98df58c1dae9b3b196089858a \
+    # via sphinx
+urllib3==1.24.1 \
+    --hash=sha256:61bf29cada3fc2fbefad4fdf059ea4bd1b4a86d2b6d15e1c7c0b582b9752fe39 \
+    --hash=sha256:de9529817c93f27c8ccbfead6985011db27bd0ddfcdb2d86f3f663385c6a9c22 \
+    # via requests
diff --git a/contrib/packaging/wix/README.txt b/contrib/packaging/wix/readme.rst
rename from contrib/packaging/wix/README.txt
rename to contrib/packaging/wix/readme.rst
--- a/contrib/packaging/wix/README.txt
+++ b/contrib/packaging/wix/readme.rst
@@ -1,29 +1,71 @@
-WiX installer source files
+WiX Installer
+=============
+
+The files in this directory are used to produce an MSI installer using
+the WiX Toolset (http://wixtoolset.org/).
+
+The MSI installers require elevated (admin) privileges due to the
+installation of MSVC CRT libraries into the Windows system store. See
+the Inno Setup installers in the ``inno`` sibling directory for installers
+that do not have this requirement.
+
+Requirements
+============
+
+Building the WiX installers requires a Windows machine. The following
+dependencies must be installed:
+
+* Python 2.7 (download from https://www.python.org/downloads/)
+* Microsoft Visual C++ Compiler for Python 2.7
+  (https://www.microsoft.com/en-us/download/details.aspx?id=44266)
+* Python 3.5+ (to run the ``build.py`` script)
+
+Building
+========
+
+The ``build.py`` script automates the process of producing an MSI
+installer. It manages fetching and configuring non-system dependencies
+(such as py2exe, gettext, and various Python packages).
+
+The script requires an activated ``Visual C++ 2008`` command prompt.
+A shortcut to such a prompt was installed with ``Microsoft Visual
+C++ Compiler for Python 2.7``. From your Start Menu, look for
+``Microsoft Visual C++ Compiler Package for Python 2.7`` then
+launch either ``Visual C++ 2008 32-bit Command Prompt`` or
+``Visual C++ 2008 64-bit Command Prompt``.
+
+From the prompt, change to the Mercurial source directory. e.g.
+``cd c:\src\hg``.
+
+Next, invoke ``build.py`` to produce an MSI installer. You will need
+to supply the path to the Python interpreter to use.::
+
+   $ python3 contrib\packaging\wix\build.py \
+      --python c:\python27\python.exe
+
+.. note::
+
+   The script validates that the Visual C++ environment is active and
+   that the architecture of the specified Python interpreter matches the
+   Visual C++ environment. An error is raised otherwise.
+
+If everything runs as intended, dependencies will be fetched and
+configured into the ``build`` sub-directory, Mercurial will be built,
+and an installer placed in the ``dist`` sub-directory. The final line
+of output should print the name of the generated installer.
+
+Additional options may be configured. Run ``build.py --help`` to see
+a list of program flags.
+
+Relationship to TortoiseHG
 ==========================
 
-The files in this folder are used by the thg-winbuild [1] package
-building architecture to create a Mercurial MSI installer.   These files
-are versioned within the Mercurial source tree because the WXS files
-must kept up to date with distribution changes within their branch.  In
-other words, the default branch WXS files are expected to diverge from
-the stable branch WXS files.  Storing them within the same repository is
-the only sane way to keep the source tree and the installer in sync.
-
-The MSI installer builder uses only the mercurial.ini file from the
-contrib/win32 folder.
+TortoiseHG uses the WiX files in this directory.
 
-The MSI packages built by thg-winbuild require elevated (admin)
-privileges to be installed due to the installation of MSVC CRT libraries
-under the C:\WINDOWS\WinSxS folder.  Thus the InnoSetup installers may
-still be useful to some users.
+The code for building TortoiseHG installers lives at
+https://bitbucket.org/tortoisehg/thg-winbuild and is maintained by
+Steve Borho (steve at borho.org).
 
-To build your own MSI packages, clone the thg-winbuild [1] repository
-and follow the README.txt [2] instructions closely.  There are fewer
-prerequisites for a WiX [3] installer than an InnoSetup installer, but
-they are more specific.
-
-Direct questions or comments to Steve Borho <steve at borho.org>
-
-[1] http://bitbucket.org/tortoisehg/thg-winbuild
-[2] http://bitbucket.org/tortoisehg/thg-winbuild/src/tip/README.txt
-[3] http://wix.sourceforge.net/
+When changing behavior of the WiX installer, be sure to notify
+the TortoiseHG Project of the changes so they have ample time
+provide feedback and react to those changes.
diff --git a/contrib/packaging/wix/build.py b/contrib/packaging/wix/build.py
new file mode 100755
--- /dev/null
+++ b/contrib/packaging/wix/build.py
@@ -0,0 +1,65 @@
+#!/usr/bin/env python3
+# Copyright 2019 Gregory Szorc <gregory.szorc at gmail.com>
+#
+# This software may be used and distributed according to the terms of the
+# GNU General Public License version 2 or any later version.
+
+# no-check-code because Python 3 native.
+
+"""Code to build Mercurial WiX installer."""
+
+import argparse
+import os
+import pathlib
+import sys
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser()
+
+    parser.add_argument('--name',
+                        help='Application name',
+                        default='Mercurial')
+    parser.add_argument('--python',
+                        help='Path to Python executable to use',
+                        required=True)
+    parser.add_argument('--sign-sn',
+                        help='Subject name (or fragment thereof) of certificate '
+                             'to use for signing')
+    parser.add_argument('--sign-cert',
+                        help='Path to certificate to use for signing')
+    parser.add_argument('--sign-password',
+                        help='Password for signing certificate')
+    parser.add_argument('--sign-timestamp-url',
+                        help='URL of timestamp server to use for signing')
+    parser.add_argument('--version',
+                        help='Version string to use')
+
+    args = parser.parse_args()
+
+    here = pathlib.Path(os.path.abspath(os.path.dirname(__file__)))
+    source_dir = here.parent.parent.parent
+
+    sys.path.insert(0, str(source_dir / 'contrib' / 'packaging'))
+
+    from hgpackaging.wix import (
+        build_installer,
+        build_signed_installer,
+    )
+
+    fn = build_installer
+    kwargs = {
+        'source_dir': source_dir,
+        'python_exe': pathlib.Path(args.python),
+        'version': args.version,
+    }
+
+    if args.sign_sn or args.sign_cert:
+        fn = build_signed_installer
+        kwargs['name'] = args.name
+        kwargs['subject_name'] = args.sign_sn
+        kwargs['cert_path'] = args.sign_cert
+        kwargs['cert_password'] = args.sign_password
+        kwargs['timestamp_url'] = args.sign_timestamp_url
+
+    fn(**kwargs)
diff --git a/contrib/packaging/hgpackaging/wix.py b/contrib/packaging/hgpackaging/wix.py
new file mode 100644
--- /dev/null
+++ b/contrib/packaging/hgpackaging/wix.py
@@ -0,0 +1,248 @@
+# wix.py - WiX installer functionality
+#
+# Copyright 2019 Gregory Szorc <gregory.szorc at gmail.com>
+#
+# This software may be used and distributed according to the terms of the
+# GNU General Public License version 2 or any later version.
+
+# no-check-code because Python 3 native.
+
+import os
+import pathlib
+import re
+import subprocess
+
+from .downloads import (
+    download_entry,
+)
+from .py2exe import (
+    build_py2exe,
+)
+from .util import (
+    extract_zip_to_directory,
+    sign_with_signtool,
+)
+
+
+SUPPORT_WXS = [
+    ('contrib.wxs', r'contrib'),
+    ('dist.wxs', r'dist'),
+    ('doc.wxs', r'doc'),
+    ('help.wxs', r'mercurial\help'),
+    ('i18n.wxs', r'i18n'),
+    ('locale.wxs', r'mercurial\locale'),
+    ('templates.wxs', r'mercurial\templates'),
+]
+
+
+EXTRA_PACKAGES = {
+    'distutils',
+    'enum',
+    'imagesize',
+    'pygments',
+    'sphinx',
+}
+
+
+EXCLUDES = {
+    # Python 3 only.
+    'jinja2.asyncsupport',
+}
+
+
+def find_version(source_dir: pathlib.Path):
+    version_py = source_dir / 'mercurial' / '__version__.py'
+
+    with version_py.open('r', encoding='utf-8') as fh:
+        source = fh.read().strip()
+
+    m = re.search('version = b"(.*)"', source)
+    return m.group(1)
+
+
+def normalize_version(version):
+    """Normalize Mercurial version string so WiX accepts it.
+
+    Version strings have to be numeric X.Y.Z.
+    """
+
+    if '+' in version:
+        version, extra = version.split('+', 1)
+    else:
+        extra = None
+
+    # 4.9rc0
+    if version[:-1].endswith('rc'):
+        version = version[:-3]
+
+    versions = [int(v) for v in version.split('.')]
+    while len(versions) < 3:
+        versions.append(0)
+
+    major, minor, build = versions[:3]
+
+    if extra:
+        # <commit count>-<hash>+<date>
+        build = int(extra.split('-')[0])
+
+    return '.'.join('%d' % x for x in (major, minor, build))
+
+
+def ensure_vc90_merge_modules(build_dir):
+    x86 = (
+        download_entry('vc9-crt-x86-msm', build_dir,
+                       local_name='microsoft.vcxx.crt.x86_msm.msm')[0],
+        download_entry('vc9-crt-x86-msm-policy', build_dir,
+                       local_name='policy.x.xx.microsoft.vcxx.crt.x86_msm.msm')[0]
+    )
+
+    x64 = (
+        download_entry('vc9-crt-x64-msm', build_dir,
+                       local_name='microsoft.vcxx.crt.x64_msm.msm')[0],
+        download_entry('vc9-crt-x64-msm-policy', build_dir,
+                       local_name='policy.x.xx.microsoft.vcxx.crt.x64_msm.msm')[0]
+    )
+    return {
+        'x86': x86,
+        'x64': x64,
+    }
+
+
+def run_candle(wix, cwd, wxs, source_dir, defines=None):
+    args = [
+        str(wix / 'candle.exe'),
+        '-nologo',
+        str(wxs),
+        '-dSourceDir=%s' % source_dir,
+    ]
+
+    if defines:
+        args.extend('-d%s=%s' % define for define in sorted(defines.items()))
+
+    subprocess.run(args, cwd=str(cwd), check=True)
+
+
+def make_post_build_signing_fn(name, subject_name=None, cert_path=None,
+                               cert_password=None, timestamp_url=None):
+    """Create a callable that will use signtool to sign hg.exe."""
+
+    def post_build_sign(source_dir, build_dir, dist_dir, version):
+        description = '%s %s' % (name, version)
+
+        sign_with_signtool(dist_dir / 'hg.exe', description,
+                           subject_name=subject_name, cert_path=cert_path,
+                           cert_password=cert_password,
+                           timestamp_url=timestamp_url)
+
+    return post_build_sign
+
+
+def build_installer(source_dir: pathlib.Path, python_exe: pathlib.Path,
+                    msi_name='mercurial', version=None, post_build_fn=None):
+    """Build a WiX MSI installer.
+
+    ``source_dir`` is the path to the Mercurial source tree to use.
+    ``arch`` is the target architecture. either ``x86`` or ``x64``.
+    ``python_exe`` is the path to the Python executable to use/bundle.
+    ``version`` is the Mercurial version string. If not defined,
+    ``mercurial/__version__.py`` will be consulted.
+    ``post_build_fn`` is a callable that will be called after building
+    Mercurial but before invoking WiX. It can be used to e.g. facilitate
+    signing. It is passed the paths to the Mercurial source, build, and
+    dist directories and the resolved Mercurial version.
+    """
+    arch = 'x64' if r'\x64' in os.environ.get('LIB', '') else 'x86'
+
+    hg_build_dir = source_dir / 'build'
+    dist_dir = source_dir / 'dist'
+
+    requirements_txt = (source_dir / 'contrib' / 'packaging' /
+                        'wix' / 'requirements.txt')
+
+    build_py2exe(source_dir, hg_build_dir,
+                 python_exe, 'wix', requirements_txt,
+                 extra_packages=EXTRA_PACKAGES, extra_excludes=EXCLUDES)
+
+    version = version or normalize_version(find_version(source_dir))
+    print('using version string: %s' % version)
+
+    if post_build_fn:
+        post_build_fn(source_dir, hg_build_dir, dist_dir, version)
+
+    build_dir = hg_build_dir / ('wix-%s' % arch)
+
+    build_dir.mkdir(exist_ok=True)
+
+    wix_pkg, wix_entry = download_entry('wix', hg_build_dir)
+    wix_path = hg_build_dir / ('wix-%s' % wix_entry['version'])
+
+    if not wix_path.exists():
+        extract_zip_to_directory(wix_pkg, wix_path)
+
+    ensure_vc90_merge_modules(hg_build_dir)
+
+    source_build_rel = pathlib.Path(os.path.relpath(source_dir, build_dir))
+
+    defines = {'Platform': arch}
+
+    for wxs, rel_path in SUPPORT_WXS:
+        wxs = source_dir / 'contrib' / 'packaging' / 'wix' / wxs
+        wxs_source_dir = source_dir / rel_path
+        run_candle(wix_path, build_dir, wxs, wxs_source_dir, defines=defines)
+
+    source = source_dir / 'contrib' / 'packaging' / 'wix' / 'mercurial.wxs'
+    defines['Version'] = version
+    defines['Comments'] = 'Installs Mercurial version %s' % version
+    defines['VCRedistSrcDir'] = str(hg_build_dir)
+
+    run_candle(wix_path, build_dir, source, source_build_rel, defines=defines)
+
+    msi_path = source_dir / 'dist' / (
+        '%s-%s-%s.msi' % (msi_name, version, arch))
+
+    args = [
+        str(wix_path / 'light.exe'),
+        '-nologo',
+        '-ext', 'WixUIExtension',
+        '-sw1076',
+        '-spdb',
+        '-o', str(msi_path),
+    ]
+
+    for source, rel_path in SUPPORT_WXS:
+        assert source.endswith('.wxs')
+        args.append(str(build_dir / ('%s.wixobj' % source[:-4])))
+
+    args.append(str(build_dir / 'mercurial.wixobj'))
+
+    subprocess.run(args, cwd=str(source_dir), check=True)
+
+    print('%s created' % msi_path)
+
+    return {
+        'msi_path': msi_path,
+    }
+
+
+def build_signed_installer(source_dir: pathlib.Path, python_exe: pathlib.Path,
+                           name: str, version=None, subject_name=None,
+                           cert_path=None, cert_password=None,
+                           timestamp_url=None):
+    """Build an installer with signed executables."""
+
+    post_build_fn = make_post_build_signing_fn(
+        name,
+        subject_name=subject_name,
+        cert_path=cert_path,
+        cert_password=cert_password,
+        timestamp_url=timestamp_url)
+
+    info = build_installer(source_dir, python_exe=python_exe,
+                           msi_name=name.lower(), version=version,
+                           post_build_fn=post_build_fn)
+
+    description = '%s %s' % (name, version)
+
+    sign_with_signtool(info['msi_path'], description,
+                       subject_name=subject_name, cert_path=cert_path,
+                       cert_password=cert_password, timestamp_url=timestamp_url)
diff --git a/contrib/packaging/hgpackaging/util.py b/contrib/packaging/hgpackaging/util.py
--- a/contrib/packaging/hgpackaging/util.py
+++ b/contrib/packaging/hgpackaging/util.py
@@ -8,6 +8,7 @@
 # no-check-code because Python 3 native.
 
 import distutils.version
+import getpass
 import os
 import pathlib
 import subprocess
@@ -50,6 +51,89 @@
     ]
 
 
+def windows_10_sdk_info():
+    """Resolves information about the Windows 10 SDK."""
+
+    base = pathlib.Path(os.environ['ProgramFiles(x86)']) / 'Windows Kits' / '10'
+
+    if not base.is_dir():
+        raise Exception('unable to find Windows 10 SDK at %s' % base)
+
+    # Find the latest version.
+    bin_base = base / 'bin'
+
+    versions = [v for v in os.listdir(bin_base) if v.startswith('10.')]
+    version = sorted(versions, reverse=True)[0]
+
+    bin_version = bin_base / version
+
+    return {
+        'root': base,
+        'version': version,
+        'bin_root': bin_version,
+        'bin_x86': bin_version / 'x86',
+        'bin_x64': bin_version / 'x64'
+    }
+
+
+def find_signtool():
+    """Find signtool.exe from the Windows SDK."""
+    sdk = windows_10_sdk_info()
+
+    for key in ('bin_x64', 'bin_x86'):
+        p = sdk[key] / 'signtool.exe'
+
+        if p.exists():
+            return p
+
+    raise Exception('could not find signtool.exe in Windows 10 SDK')
+
+
+def sign_with_signtool(file_path, description, subject_name=None,
+                       cert_path=None, cert_password=None,
+                       timestamp_url=None):
+    """Digitally sign a file with signtool.exe.
+
+    ``file_path`` is file to sign.
+    ``description`` is text that goes in the signature.
+
+    The signing certificate can be specified by ``cert_path`` or
+    ``subject_name``. These correspond to the ``/f`` and ``/n`` arguments
+    to signtool.exe, respectively.
+
+    The certificate password can be specified via ``cert_password``. If
+    not provided, you will be prompted for the password.
+
+    ``timestamp_url`` is the URL of a RFC 3161 timestamp server (``/tr``
+    argument to signtool.exe).
+    """
+    if cert_path and subject_name:
+        raise ValueError('cannot specify both cert_path and subject_name')
+
+    while cert_path and not cert_password:
+        cert_password = getpass.getpass('password for %s: ' % cert_path)
+
+    args = [
+        str(find_signtool()), 'sign',
+        '/v',
+        '/fd', 'sha256',
+        '/d', description,
+    ]
+
+    if cert_path:
+        args.extend(['/f', str(cert_path), '/p', cert_password])
+    elif subject_name:
+        args.extend(['/n', subject_name])
+
+    if timestamp_url:
+        args.extend(['/tr', timestamp_url, '/td', 'sha256'])
+
+    args.append(str(file_path))
+
+    print('signing %s' % file_path)
+    subprocess.run(args, check=True)
+
+
 PRINT_PYTHON_INFO = '''
 import platform; print("%s:%s" % (platform.architecture()[0], platform.python_version()))
 '''.strip()
diff --git a/contrib/packaging/hgpackaging/downloads.py b/contrib/packaging/hgpackaging/downloads.py
--- a/contrib/packaging/hgpackaging/downloads.py
+++ b/contrib/packaging/hgpackaging/downloads.py
@@ -31,12 +31,42 @@
         'sha256': '6bd383312e7d33eef2e43a5f236f9445e4f3e0f6b16333c6f183ed445c44ddbd',
         'version': '0.6.9',
     },
+    # The VC9 CRT merge modules aren't readily available on most systems because
+    # they are only installed as part of a full Visual Studio 2008 install.
+    # While we could potentially extract them from a Visual Studio 2008
+    # installer, it is easier to just fetch them from a known URL.
+    'vc9-crt-x86-msm': {
+        'url': 'https://github.com/indygreg/vc90-merge-modules/raw/9232f8f0b2135df619bf7946eaa176b4ac35ccff/Microsoft_VC90_CRT_x86.msm',
+        'size': 615424,
+        'sha256': '837e887ef31b332feb58156f429389de345cb94504228bb9a523c25a9dd3d75e',
+    },
+    'vc9-crt-x86-msm-policy': {
+        'url': 'https://github.com/indygreg/vc90-merge-modules/raw/9232f8f0b2135df619bf7946eaa176b4ac35ccff/policy_9_0_Microsoft_VC90_CRT_x86.msm',
+        'size': 71168,
+        'sha256': '3fbcf92e3801a0757f36c5e8d304e134a68d5cafd197a6df7734ae3e8825c940',
+    },
+    'vc9-crt-x64-msm': {
+        'url': 'https://github.com/indygreg/vc90-merge-modules/raw/9232f8f0b2135df619bf7946eaa176b4ac35ccff/Microsoft_VC90_CRT_x86_x64.msm',
+        'size': 662528,
+        'sha256': '50d9639b5ad4844a2285269c7551bf5157ec636e32396ddcc6f7ec5bce487a7c',
+    },
+    'vc9-crt-x64-msm-policy': {
+        'url': 'https://github.com/indygreg/vc90-merge-modules/raw/9232f8f0b2135df619bf7946eaa176b4ac35ccff/policy_9_0_Microsoft_VC90_CRT_x86_x64.msm',
+        'size': 71168,
+        'sha256': '0550ea1929b21239134ad3a678c944ba0f05f11087117b6cf0833e7110686486',
+    },
     'virtualenv': {
         'url': 'https://files.pythonhosted.org/packages/37/db/89d6b043b22052109da35416abc3c397655e4bd3cff031446ba02b9654fa/virtualenv-16.4.3.tar.gz',
         'size': 3713208,
         'sha256': '984d7e607b0a5d1329425dd8845bd971b957424b5ba664729fab51ab8c11bc39',
         'version': '16.4.3',
     },
+    'wix': {
+        'url': 'https://github.com/wixtoolset/wix3/releases/download/wix3111rtm/wix311-binaries.zip',
+        'size': 34358269,
+        'sha256': '37f0a533b0978a454efb5dc3bd3598becf9660aaf4287e55bf68ca6b527d051d',
+        'version': '3.11.1',
+    },
 }
 
 



To: indygreg, #hg-reviewers
Cc: mercurial-devel


More information about the Mercurial-devel mailing list