[PATCH stable] demandimport: add __main__ to the blacklist (because of setuptools)

[Greg Ward]

On Sun, Jan 24, 2010 at 12:34 PM, Benoit Boissinot
<benoit.boissinot at ens-lyon.org> wrote:
> On Wed, Jan 20, 2010 at 08:24:03PM -0500, Greg Ward wrote:
>> # HG changeset patch
>> # User Greg Ward <greg-hg at gerg.ca>
>> # Date 1264037016 18000
>> # Branch stable
>> # Node ID 992d832eecd81bcb22bcd889c01cbbb5431c2c68
>> # Parent  acf001ee5ef80c698b75d437d9685ed926a3e2fc
>> demandimport: add __main__ to the blacklist (because of setuptools)
>
> Couldn't we just blacklist setuptools instead? __main__ seems more
> general.

I don't think so.  I tried a couple of things to make this work, and
blacklisting __main__ was all I could come up with.  Let me explain.
I have a Python hook that does

  import pytz

because it has to do timezone calculations.  pytz/__init__.py does this:

try:
    from pkg_resources import resource_stream
except ImportError:
    resource_stream = None

where pkg_resources.py ultimately comes from setuptools.

pkg_resources.py does this:

try:
    # Does the main program list any requirements?
    from __main__ import __requires__
except ImportError:
    pass # No: just use the default working set based on sys.path
else:
    # Yes: ensure the requirements are met, by prefixing sys.path if necessary
    [...]

which, in most Python apps, would land in the "except ImportError"
code.  But in Mercurial, the "from __main__ import ..." succeeds,
putting a _demandmod object in pkg_resources' namespace.

Later, pkg_resources tries to do something with the __resources__ list
that it thought it imported from __main__.  But it's not a list, it's
a _demandmod object:

  [...]
  File "/usr/lib/python2.6/dist-packages/pkg_resources.py", line 1813,
in yield_lines
    for ss in strs:
TypeError: '_demandmod' object is not iterable

This is fairly easy to reproduce if you have any module that depends
on pkg_resources:

"""
$ python
Python 2.6.4 (r264:75706, Dec  7 2009, 18:45:15)
[...]
>>> from mercurial import demandimport
>>> demandimport.enable()
>>> import pytz
>>> pytz
<unloaded module 'pytz'>
>>> pytz.tz
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/pymodules/python2.6/mercurial/demandimport.py", line
75, in __getattribute__
[...]
  File "/usr/lib/python2.6/dist-packages/pkg_resources.py", line 1814,
in yield_lines
    for s in yield_lines(ss):
  File "/usr/lib/python2.6/dist-packages/pkg_resources.py", line 1813,
in yield_lines
    for ss in strs:
TypeError: '_demandmod' object is not iterable
"""

I tried blacklisting several module names, but only '__main__' worked.

While it seems odd to import __main__, the fact remains that this code
is out there, and works fine with Python apps other than Mercurial.
(Although I imagine it will give Bazaar's lazyimport trouble sooner or
later. ;-)  And there is certainly no cost to letting imports of
__main__ go through: by definition, it's already in memory and in
sys.modules.

Greg