[PATCH 2 of 2] httpclient: import revision fc731618702a of py-nonblocking-http

[Martin Geisler]

Matt Mackall <mpm at selenic.com> writes:

Hi Matt,

> A sensible open source application does not bundle its dependencies
> nor depend on precise versions of its dependencies. This just makes
> extra pain for integrators and packagers who have to then decouple
> things or face dependency hell.

Please don't misunderstand me -- I understand perfectly well how
dependencies ought to be managed.

I'm just noting that we bundle py-nonblocking-http *right now* inside
Mercurial and that might have been an opportunity for us to learn more
about how well subrepos work. When there is a python-nonblocking-http
Debian package, then we can drop the subrepo again.

Out of curiosity, I looked through the source in an attempt to see how
many third-party libraries and files we ship today. These are the four
projects we currently bundle part:

* mercurial/byterange.py, mercurial/keepalive.py

  Homepage: http://urlgrabber.baseurl.org/

  Debian package: python-urlgrabber

* mercurial/httpclient/*

  Homepage: http://code.google.com/p/py-nonblocking-http/

* hgext/zeroconf/Zeroconf.py

  Homepage: https://github.com/paulsm/pyzeroconf

* i18n/polib.py, i18n/polib.LICENSE

  Homepage: https://bitbucket.org/izi/polib/

  Debian package: python-polib

A strict approach would have zeroconf require the user to install the
required library manually (just like highlight requires a manual install
of Pygments) but that makes the extension less useful.

For polib, I think it would be okay if we ask the translators to install
it themselves. It's only used when you build i18n/hg.pot and that is
only done by translators.


There are also some small files that were taken from other sources:

* mercurial/simplemerge.py -- from Bazaar:

  http://bazaar.launchpad.net/~bzr-pqm/bzr/bzr.dev/view/head:/bzrlib/merge3.py

* mercurial/lsprof.py -- from this Subversion repository:

  http://codespeak.net/svn/user/arigo/hack/misc/lsprof/

* mercurial/lsprofcalltree.py -- seems to be from here:

  http://codespeak.net/pypy/dist/pypy/tool/lsprofcalltree.py

I don't care much about those files since they're very isolated.

> So if it doesn't make sense to use subrepos for an open-source package
> like Mercurial, what does it make sense for? The answer is managing
> platforms and distributions.

I agree with you that subrepos will work great for this scenario!
However, when the dependencies are mostly static you might as well
maintain a simple Makefile with

  deps:
        hg clone -r 123 http://some/repo
        hg clone -r 234 http://some/other/repo

etc. This would even be clearer since 'hg diff' in the super-repo will
show you the changes you've made to the Makefile, whereas 'hg diff' does
not show the (future) changes made to .hgsubstate.

All in all, I feel that little is gained by using subrepos in this case,
but maybe you disagree?

> Similarly, people who are managing websites with multiple third-party
> components and want reproducibility of precise combinations should do
> something similar. But as a project, we -don't- want to be dependent
> on precise versions and we -don't- ship bundles of third-party
> components. So subrepos are a boat anchor we don't want to lug around.

We sometimes do want to depend on an exact version of a third-party
library: the py-nonblocking-http library is a case in point where it's
very important for us that we use the revision with the latest bugfixes
and not an older version. Luckily for us, Augie let's us know whenever
our bundled copy is out of date.

The polib library is another case: I have fixed bugs in our copy and
made sure to contribute them back to the Bitbucket repository.

> And we -also- don't want random open-source projects to follow our
> lead in using boat anchors where inappropriate.

I agree we should be a role model. Of course.

But it is also very very important that we really use the major features
we ship so that we can make them shine. The eol extension is not a core
feature so it's okay that we only have exercise it with tests, but
subrepos are a core feature.

-- 
Martin Geisler

aragost Trifork
Professional Mercurial support
http://mercurial.aragost.com/kick-start/