[PATCH] docker: add Docker files for running an Apache mod_wsgi server

Wed Nov 12 12:51:44 CST 2014

On 11/12/14 9:58 AM, Brendan Cully wrote:
> I like it.
>
> Can I ask why you need to apt-get install mercurial, when you appear
> to be either using the user-supplied source tree or cloning from
> selenic?

We need a Mercurial to clone from selenic. I suppose we could defer 
installing Mercurial unless we actually need it. Or, we could download a 
tarball and "bootstrap" a Mercurial install. I think these are follow-up 
fodder.

> It might be nice to arrange to have the apache logs written into the
> host, perhaps with another VOLUME argument.

Great idea. I think it should be a follow-up.

> I also worry vaguely that the default pushable repo is a bit of a
> security risk -- it might make sense to leave those settings out so
> that you get hg serve defaults (I assume that per-repo settings still
> work to override this).

As stated in the README, this isn't meant to be used in production. The 
default repo is all about developer convenience so you can spin up a 
container and have somewhere to push changes to without requiring you to 
manually init a repo.

I think a feature to make the creation of the default repo optional or 
better scripting to create repos would be excellent feature additions.

> On Tuesday, 11 November 2014 at 20:37, Gregory Szorc wrote:
>> # HG changeset patch
>> # User Gregory Szorc <gregory.szorc at gmail.com>
>> # Date 1415766730 28800
>> #      Tue Nov 11 20:32:10 2014 -0800
>> # Node ID 09160f191fc370d779fea0c627dfe01240cefc83
>> # Parent  18cc87e4375afaeb5986ef9e941854cefa893759
>> docker: add Docker files for running an Apache mod_wsgi server
>>
>> I frequently find myself wanting to run hgweb in a production-like
>> environment, with a real HTTP server and multiple WSGI workers.
>>
>> This patch introduces a Docker environment for running Mercurial
>> under Apache + mod_wsgi. With just a few command executions, it is
>> possible to spin up a Docker container running hgweb.
>>
>> The Docker environment is designed with customization in mind. With no
>> options, we clone Mercurial from upstream and run @ with a multi-repo
>> hgweb setup that allows pushes. An empty repository is created. You can
>> push your favorite repository to it and test things.
>>
>> It is possible to customize the hgweb WSGI and config files if you wish
>> to stray from the defaults.
>>
>> For Mercurial developers, it is possible to mount your local Mercurial
>> checkout directory in the container. This means that you can make local
>> changes and fire up an Apache WSGI environment within seconds to test
>> those changes. (This is the primary use case I developed this patch
>> for.)
>>
>> You can also employ data volume magic to have persistent repositories in
>> the container.
>>
>> All of this is documented in the README.rst.
>>
>> Is the container and environment perfect? No. But you have to start
>> somewhere. I'm already using this environment to examine memory and
>> performance behavior of Mercurial when concurrently serving multiple
>> clones. I find this environment extremely useful and I feel Mercurial
>> developers will enjoy this new tool in their toolbelt.
>>
>> diff --git a/contrib/docker-apache-server/Dockerfile b/contrib/docker-apache-server/Dockerfile
>> new file mode 100644
>> --- /dev/null
>> +++ b/contrib/docker-apache-server/Dockerfile
>> @@ -0,0 +1,24 @@
>> +FROM debian:wheezy
>> +
>> +ENV DEBIAN_FRONTEND noninteractive
>> +ENV WSGI_PROCESSES 4
>> +ENV WSGI_THREADS 8
>> +ENV WSGI_MAX_REQUESTS 100000
>> +
>> +EXPOSE 80
>> +VOLUME ["/var/hg/htdocs", "/var/hg/repos"]
>> +
>> +RUN apt-get update && apt-get -y install mercurial libapache2-mod-wsgi \
>> +    python-dev
>> +
>> +# Install our own Apache site.
>> +RUN a2dissite 000-default
>> +ADD vhost.conf /etc/apache2/sites-available/hg
>> +RUN a2ensite hg
>> +
>> +ADD hgwebconfig /defaulthgwebconfig
>> +
>> +ADD entrypoint.sh /entrypoint.sh
>> +ENTRYPOINT ["/entrypoint.sh"]
>> +
>> +CMD ["/usr/sbin/apache2", "-DFOREGROUND"]
>> diff --git a/contrib/docker-apache-server/README.rst b/contrib/docker-apache-server/README.rst
>> new file mode 100644
>> --- /dev/null
>> +++ b/contrib/docker-apache-server/README.rst
>> @@ -0,0 +1,134 @@
>> +====================
>> +Apache Docker Server
>> +====================
>> +
>> +This directory contains code for running a Mercurial hgweb server via
>> +mod_wsgi with the Apache HTTP Server inside a Docker container.
>> +
>> +.. important::
>> +
>> +   This container is intended for testing purposes only: it is
>> +   **not** meant to be suitable for production use.
>> +
>> +Building Image
>> +==============
>> +
>> +The first step is to build a Docker image containing Apache and mod_wsgi::
>> +
>> +  $ docker build -t hg-apache .
>> +
>> +Running the Server
>> +==================
>> +
>> +You can run the server::
>> +
>> +  $ docker run -rm -it hg-apache
>> +
>> +You should see some start-up actions (including a Mercurial install)
>> +and some output saying Apache has started.
>> +
>> +The HTTP server is bound to port 80, but it isn't accessible outside
>> +the Docker container. That's not very exciting.
>> +
>> +To make the HTTP server accessible outside the container, you'll need
>> +to publish the port on the host machine::
>> +
>> +  $ docker run -rm -it -p 8000:80 hg-apache
>> +
>> +Now if you load ``http://localhost:8000/`` (or whatever interface Docker
>> +is using), you should see hgweb running!
>> +
>> +For your convenience, we've created an empty repository available at
>> +``/repo``. Feel free to populate it with ``hg push``.
>> +
>> +Customizing the Server
>> +======================
>> +
>> +By default, the Docker container runs an up-to-date build of the @
>> +bookmark from upstream Mercurial. It installs its own hgweb config and
>> +set of repositories. It uses some reasonable defaults for mod_wsgi.
>> +
>> +Customizing the WSGI Dispatcher And Mercurial Config
>> +----------------------------------------------------
>> +
>> +By default, the Docker environment installs a custom ``hgweb.wsgi``
>> +file (based on the example in ``contrib/hgweb.wsgi``). The file
>> +is installed into ``/var/hg/htdocs//hgweb.wsgi``.
>> +
>> +A default hgweb configuration file is also installed. The ``hgwebconfig``
>> +file from this directory is installed into ``/var/hg/htdocs/config``.
>> +
>> +You have a few options for customizing these files.
>> +
>> +The simplest is to hack up ``hgwebconfig`` and ``entrypoint.sh`` in
>> +this directory and to rebuild the Docker image. This has the downside
>> +that the Mercurial working copy is modified and you may accidentally
>> +commit unwanted changes.
>> +
>> +The next simplest is to copy this directory somewhere, make your changes,
>> +then rebuild the image. No working copy changes involved.
>> +
>> +The preferred solution is to mount a host file into the container and
>> +overwrite the built-in defaults.
>> +
>> +For example, say we create a custom hgweb config file in ``~/hgweb``. We
>> +can start the container like so to install our custom config file::
>> +
>> +  $ docker run --rm -it -v ~/hgweb:/var/hg/htdocs/config -p 8000:80 hg-apache
>> +
>> +You can do something similar to install a custom WSGI dispatcher::
>> +
>> +  $ docker run --rm --it -v ~/hgweb.wsgi:/var/hg/htdocs/hgweb.wsgi -p 8000:80 hg-apache
>> +
>> +Managing Repositories
>> +---------------------
>> +
>> +Repositories are served from ``/var/hg/repos`` by default. This directory
>> +is configured as a Docker volume. This means you can mount an existing
>> +data volume container in the container so repository data is persisted
>> +across container invocations. See
>> +https://docs.docker.com/userguide/dockervolumes/ for more.
>> +
>> +Alternatively, if you just want to perform lightweight repository
>> +manipulation, open a shell in the container::
>> +
>> +  $ docker exec -it <container> /bin/bash
>> +
>> +Then run ``hg init``, etc to manipulate the repositories in ``/var/hg/repos``.
>> +
>> +Running Mercurial from Local Source
>> +-----------------------------------
>> +
>> +By default, Mercurial is cloned from upstream and the ``@`` bookmark
>> +is installed.
>> +
>> +Mercurial developers may wish to run Mercurial from locally modified
>> +source code rather than the upstream source. To do this, mount your
>> +local Mercurial checkout under ``/var/hg/source`` in the container.
>> +For example::
>> +
>> +  $ docker run --rm -it -v ~/src/hg:/var/hg/source -p 8000:80 hg-apache
>> +
>> +.. warning::
>> +
>> +   If the architecture of the host machine doesn't match that of the
>> +   Docker host (e.g. when running Boot2Docker under OS X), Mercurial's
>> +   Python C extensions will fail to run. Be sure to ``make clean`` your
>> +   local source tree before mounting it in the container.
>> +
>> +mod_wsgi Configuration Settings
>> +-------------------------------
>> +
>> +mod_wsgi settings can be controlled with the following environment
>> +variables.
>> +
>> +WSGI_PROCESSES
>> +   Number of WSGI processes to run.
>> +WSGI_THREADS
>> +   Number of threads to run in each WSGI process
>> +WSGI_MAX_REQUESTS
>> +   Maximum number of requests each WSGI process may serve before it is
>> +   reaped.
>> +
>> +See https://code.google.com/p/modwsgi/wiki/ConfigurationDirectives#WSGIDaemonProcess
>> +for more on these settings.
>> diff --git a/contrib/docker-apache-server/entrypoint.sh b/contrib/docker-apache-server/entrypoint.sh
>> new file mode 100755
>> --- /dev/null
>> +++ b/contrib/docker-apache-server/entrypoint.sh
>> @@ -0,0 +1,58 @@
>> +#!/bin/sh
>> +
>> +# This script gets executed on container start. Its job is to set up
>> +# the Mercurial environment and invoke the server.
>> +
>> +# Mercurial can be started in two modes.
>> +# If the MERCURIAL_SOURCE environment variable is set and it points to a
>> +# Mercurial source directory, we will install Mercurial from that directory.
>> +# Otherwise, we download the Mercurial source and install it manually.
>> +
>> +set -e
>> +
>> +SOURCE_DIR=/var/hg/source
>> +INSTALL_DIR=/var/hg/install
>> +REPOS_DIR=/var/hg/repos
>> +HTDOCS_DIR=/var/hg/htdocs
>> +
>> +if [ ! -d ${SOURCE_DIR} ]; then
>> +  hg clone http://selenic.com/repo/hg ${SOURCE_DIR}
>> +fi
>> +
>> +cd ${SOURCE_DIR}
>> +/usr/bin/python2.7 setup.py install --root=/ --prefix=${INSTALL_DIR} --force
>> +cd ..
>> +
>> +mkdir -p ${HTDOCS_DIR}
>> +
>> +if [ ! -f ${HTDOCS_DIR}/config ]; then
>> +  cp /defaulthgwebconfig ${HTDOCS_DIR}/config
>> +fi
>> +
>> +if [ ! -f ${HTDOCS_DIR}/hgweb.wsgi ]; then
>> +  cat >> ${HTDOCS_DIR}/hgweb.wsgi << EOF
>> +config = '${HTDOCS_DIR}/config'
>> +
>> +import sys
>> +sys.path.insert(0, '${INSTALL_DIR}/lib/python2.7/site-packages')
>> +
>> +from mercurial import demandimport
>> +demandimport.enable()
>> +
>> +from mercurial.hgweb import hgweb
>> +application = hgweb(config)
>> +EOF
>> +fi
>> +
>> +mkdir -p ${REPOS_DIR}
>> +
>> +if [ ! -d ${REPOS_DIR}/repo ]; then
>> +  echo "Creating empty repo"
>> +  ${INSTALL_DIR}/bin/hg init ${REPOS_DIR}/repo
>> +  chown -R www-data:www-data ${REPOS_DIR}/repo
>> +fi
>> +
>> +. /etc/apache2/envvars
>> +
>> +echo "Starting Apache HTTP Server"
>> +exec "$@"
>> diff --git a/contrib/docker-apache-server/hgwebconfig b/contrib/docker-apache-server/hgwebconfig
>> new file mode 100644
>> --- /dev/null
>> +++ b/contrib/docker-apache-server/hgwebconfig
>> @@ -0,0 +1,6 @@
>> +[paths]
>> +/ = /var/hg/repos/**
>> +
>> +[web]
>> +allow_push = *
>> +push_ssl = False
>> diff --git a/contrib/docker-apache-server/vhost.conf b/contrib/docker-apache-server/vhost.conf
>> new file mode 100644
>> --- /dev/null
>> +++ b/contrib/docker-apache-server/vhost.conf
>> @@ -0,0 +1,24 @@
>> +# Apache won't be able to resolve its own hostname, so we sneak this
>> +# into the global context to silence a confusing-to-user warning on
>> +# server start.
>> +ServerName hg
>> +
>> +<VirtualHost *:80>
>> +  DocumentRoot /var/hg/htdocs
>> +  <Directory />
>> +    Options FollowSymLinks
>> +    AllowOverride None
>> +  </Directory>
>> +
>> +  SetEnv HGENCODING UTF-8
>> +  SetEnv LC_TYPE UTF-8
>> +
>> +  WSGIDaemonProcess hg processes=${WSGI_PROCESSES} threads=${WSGI_THREADS} maximum-requests=${WSGI_MAX_REQUESTS} user=www-data group=www-data display-name=hg-wsgi
>> +  WSGIProcessGroup hg
>> +
>> +  WSGIScriptAliasMatch ^(.*) /var/hg/htdocs/hgweb.wsgi$1
>> +
>> +  ErrorLog ${APACHE_LOG_DIR}/error.log
>> +  LogLevel warn
>> +  CustomLog ${APACHE_LOG_DIR}/access.log combined
>> +</VirtualHost>
>> _______________________________________________
>> Mercurial-devel mailing list
>> Mercurial-devel at selenic.com
>> http://selenic.com/mailman/listinfo/mercurial-devel
>>