Differences between revisions 32 and 33
Revision 32 as of 2018-06-09 07:27:10
Size: 11763
Comment: about memory
Revision 33 as of 2018-06-09 07:35:07
Size: 11759
Comment:
Deletions are marked like this. Additions are marked like this.
Line 265: Line 265:
That should be enough to make hgweb up-and-running, but if you see your hgweb processes use too much memory after a while, then please [[BugTracker|file a bug]], meanwhile you can alleviate it with these lines in gunicorn.conf:

{{{
max_requests = 100  # restart workers after serving 100 ± 5 requests
That should be enough to get it up and running, but if you see your hgweb processes use too much memory after a while, then please [[BugTracker|file a bug]], meanwhile you can alleviate it with these lines in gunicorn.conf:

{{{
# restart workers after serving 100 ± 5 requests
max_requests = 100

{i} This page does not meet our wiki style guidelines. Please help improve this page by cleaning up its formatting.

Configure Nginx

This page explains Nginx web server specific configuration for publishing repositories with the hgweb and reverse-proxy methods.

1. hgweb (with uwsgi)

This example shows how to serve a set of Mercurial repositories hosted on the subdirectory /hg of a server named mygoodserver.org

Repositories are located at /var/www/hg/repos

See also PublishingRepositories

This setup has been tested on a Debian Wheezy system.

1.1. uwsgi configuration

For this example, uwsgi configuration directory is /etc/uwsgi/. You'll have to add this sample configuration file in apps-available subdirectory and symlink it from apps-enable subdirectory. Make sure you start uwgsi service later.

/etc/uwsgi/apps-available/hgweb.ini

[uwsgi]
processes = 2
socket = /run/uwsgi/app/hgweb/socket
chdir = /var/www/hg
wsgi-file = hgweb.wsgi
uid = www-data
gid = www-data

The above configuration file instructs wsgi to launch 2 processes handling the hgweb.wsgi script, located at /var/www/hg directory. The processes communicates is through the socket located at /run/uwsgi/app/hgweb/socket

1.2. hgweb configuration

This is a subtly modified example of the script usually deployed with Mercurial (see PublishingRepositories#Getting_the_hgweb_script) hgweb.wsgi

config = "/var/www/hg/hgweb.config"
import os
os.environ["HGENCODING"] = "UTF-8"
import cgitb; cgitb.enable()
from mercurial import demandimport; demandimport.enable()
from mercurial.hgweb import hgweb
application = hgweb(config)

As this script is intended to run from the uwsgi daemon, notice it should be placed at /var/www/hg

The hgweb.wsgi reads a configuration file which can specify a number of things, most importantly repositories local path and some general information. See PublishingRepositories#Configuration_of_hgweb for more details. hgweb.config

[paths]
/ = /var/www/hg/repos/*
[web]
style = gitweb
baseurl = http://mygoodserver.org/hg/
contact = Who knows!
staticurl = /hg/static

1.3. nginx configuration (repositories at /hg subdirectory)

As we intend to server repositories from a subdirectory of the server, we need to modify default site configuration. Provided nginx configuration is located at /etc/nginx/ the default site configuration is: /etc/nginx/sites-available/default

server {
        listen 80 default_server;
        listen [::]:80 default_server ipv6only=on;

        root /var/www;
        index index.html index.htm;

        # Make site accessible from http://localhost/
        server_name localhost mygoodserver mygoodserver.org;

        location /hg/ {
                uwsgi_pass unix:/run/uwsgi/app/hgweb/socket;
                include uwsgi_params;
                uwsgi_param SERVER_ADDR $server_addr;
#Note 1
                uwsgi_param REMOTE_USER $remote_user;
#Note 2
                uwsgi_modifier1 30;
                uwsgi_param SCRIPT_NAME /hg;

#Note 3
                limit_except GET HEAD {
                        allow 192.168.10.0/24;
                        allow 127.0.0.1;
                        allow ::1;
                        deny all;
                        auth_basic "Mercurial repository";
                        auth_basic_user_file /var/www/hg/.htpasswd;
                }
        }

        location /hg/static {
                alias /usr/share/mercurial/templates/static/;
                expires 30d;
        }
}

This configuration only includes Mercurial repository serving, you may want to add additional web locations or services.

Notes:

  1. /!\ REMOTE_USER passes authenticate username to hgweb script. This is required for proper authentication.

  2. /!\ The uwsgi_modifier1 30 option sets the uWSGI modifier UWSGI_MODIFIER_MANAGE_PATH_INFO. This per-request modifier instructs the uWSGI server to rewrite the PATH_INFO value removing the SCRIPT_NAME from it. (See also http://uwsgi-docs.readthedocs.org/en/latest/Nginx.html#dynamic-apps)

  3. This section enables push basic authorization for specified ips. See also HgWebDirStepByStep#Configuring_nginx

1.4. Repository configuration

You can (and probably should) add per-repositories configuration. This configuration is specified in <repository root>/.hg/hgrc. Where <repository root> is /var/www/hg/repos in this example. Relevant section in this configuration file is [web]. You can specify there allowed pushers and/or wether ssl is required:

[web]
allow_push = granteduser
push_ssl = false

See also PublishingRepositories#Allowing_push and PublishingRepositories#Putting_useful_information_in_the_index_page

{i} You can create a template preconfigured repository which you can clone or copy when you need a new repository. After the copy, just rename it.

2. hgweb (with gunicorn)

This is a different example that shows hgweb set up on an different wsgi server — gunicorn. It also uses more moving parts and optional tweaks, so each one is described only briefly if at all — it mostly is left as an exercise for the reader. It also exists as an Ansible role

This setup was deployed on Debian Stretch (and is likely to work in similar distros with minimal changes), but there are bits that may not be fit for production (by design of this recipe).

Some conventions used for this example:

  • hgweb is run by user hguser
  • repositories are in /home/hguser/repos/

  • recent versions of Mercurial and EvolveExtension are installed locally

  • gunicorn is installed into a virtualenv, not as a system package
  • supervisor is used to launch the whole thing

2.1. Prerequisites

First of all, do apt-get install build-essential mercurial python-dev virtualenv

Then make sure you have user named hguser (or however you decide to call it) and do as that user:

hg clone https://www.mercurial-scm.org/repo/hg -r stable ~/hg
hg clone https://www.mercurial-scm.org/repo/evolve -r stable ~/evolve

Then build local Mercurial: make --directory ~/hg/ local

Create a directory for this project, e.g. ~/hgwebfiles/ and copy these files there:

2.2. ~/hgwebfiles/hgweb.py

# Path to repo or hgweb config to serve (see 'hg help hgweb')
config = "/home/hguser/hgwebfiles/hgweb.conf"

# Uncomment and adjust if Mercurial is not installed system-wide
# (consult "installed modules" path from 'hg debuginstall'):
import sys; sys.path.insert(0, "/home/hguser/hg/")

# Uncomment to send python tracebacks to the browser if an error occurs:
#import cgitb; cgitb.enable()

# enable demandloading to reduce startup time
from mercurial import demandimport; demandimport.enable()

from mercurial.hgweb import hgweb
application = hgweb(config)

2.3. ~/hgwebfiles/hgweb.conf

[paths]
/ = /home/hguser/repos/*

[extensions]
highlight =
evolve.serveronly = /home/hguser/evolve/hgext3rd/evolve/serveronly.py

[web]
baseurl = http://hg.mydomain/
logourl = http://hg.mydomain/
contact = Me Myself <me@mydomain>
staticurl = /static
pygments_style = tango
highlightfiles = size('<100k')
highlightonlymatchfilename = True

2.4. ~/hgwebfiles/gunicorn.conf

bind = 'unix:/home/hguser/hgwebfiles/socket'
workers = 1
worker_class = 'eventlet'
accesslog = '-'
errorlog = '-'
timeout = 30  # set to bigger values for serving bigger repos

2.5. ~/hgwebfiles/requirements.txt

eventlet
gunicorn
pygments

2.6. virtualenv

Next, do

virtualenv ~/hgwebfiles/venv/
~/hgwebfiles/venv/bin/pip install -r ~/hgwebfiles/requirements.txt

After that, you should be able to run hgweb via gunicorn manually: /home/hguser/hgwebfiles/venv/bin/gunicorn --config=/home/hguser/hgwebfiles/gunicorn.conf hgweb:application

It's time to pick your favorite launcher for running this command on system start, but here's an example config for supervisor (if also wish to use it, don't forget to apt-get install supervisor):

2.7. /etc/supervisor/conf.d/hgweb.conf

[program:hgweb]
command = /home/hguser/hgwebfiles/venv/bin/gunicorn --config=/home/hguser/hgwebfiles/gunicorn.conf hgweb:application
directory = /home/hguser/hgwebfiles/
user = hguser
stopsignal = INT

And also a minimal Nginx config:

2.8. /etc/nginx/sites-available/50hgweb

upstream hgweb {
    server unix:/home/hguser/hgwebfiles/socket;
}

server {
    server_name hg.mydomain;

    listen 80;
    listen [::]:80;

    access_log /var/log/nginx/hg.mydomain.access.log;
    error_log /var/log/nginx/hg.mydomain.error.log;

    root /home/hguser/hg/mercurial/templates/;

    location / {
        proxy_pass http://hgweb;
        proxy_redirect off;
        proxy_buffering off;
    }

    location /static {
        expires 30d;
    }
}

Don't forget to enable it by putting a symlink in /etc/nginx/sites-enabled/.

2.9. Memory consumption

That should be enough to get it up and running, but if you see your hgweb processes use too much memory after a while, then please file a bug, meanwhile you can alleviate it with these lines in gunicorn.conf:

# restart workers after serving 100 ± 5 requests
max_requests = 100
max_requests_jitter = 5

Using robots.txt to disallow some pages for bots is also recommended, see example robots.txt.

3. Reverse proxy

This is a solution I found for hosting personal projects. I was already using Nginx, didn't want to mess with (Fast)CGI, but still wanted some basic authentication for pushing. This method allows repositories to be served using hg serve, and then be cloned and accessed via the web interface anonymously, but HTTP authentication is required for pushing.

This method relies on a running "hg serve" instance. The web server is only a reverse proxy, transactions as submitted to running Mercurial instance.

The first step is to configure Nginx.

server {
    listen      80;
    server_name <your-server-name>;    # standard stuff
    access_log  /path/to/access/log;
    error_log   /path/to/error/log;

    location / {
        limit_except GET { # do this for all requests but GETS
            auth_basic           "Restricted";
            auth_basic_user_file /path/to/htpasswd/file;
            proxy_pass http://localhost:8000;
        }

    proxy_pass http://localhost:8000; # or wherever hg serve is running
    }
}

3.1. Configure htpasswd

cd into the root directory of your repos (i.e. /var/hg/repos), and run htpasswd -c <file-name> <user-name> and supply a password. If you want to give a new user commit rights, simply cd back to that directory and run htpasswd <file-name> <user-name>.

3.2. Configure hg

The final step is to configure your repository's hgrc file, located at <repo-name>/.hg/hgrc. The import things in hgrc are to make sure that, in the [web] section, push_ssl is set to false and allow_push is set to *.

3.3. Serve

Now run hg serve as normal, and you should be able to access the web interface and clone the repository with no authentication, but to push to the central repository you need to have a user name and password from the htpasswd file.


CategoryHowTo CategoryWeb

PublishRepositoriesOnNginx (last edited 2018-06-09 07:35:07 by AntonShestakov)