[issue1774] When demandimport is disabled the 'test-profile' test fails

[Alejandro Santos]

New submission from Alejandro Santos <alejolp at alejolp.com>:

On crew and hg-stable repos, 'test-profile' test fails when demandimport is
disabled:

$ diff -u test-profile.out test-profile.err
--- test-profile.out    2009-06-16 00:23:56.000000000 -0300
+++ test-profile.err    2009-07-28 13:00:27.000000000 -0300
@@ -1,2 +1,29 @@
 % test --time
 % test --profile
+--profile failed
+** unknown exception encountered, details follow
+** report bug details to http://mercurial.selenic.com/bts/
+** or mercurial at selenic.com
+** Mercurial Distributed SCM (version 0d04d42fbab4)
+** Extensions loaded:
+Traceback (most recent call last):
+  File "/tmp/hgtests.xStzl7/install/bin/hg", line 30, in <module>
+    mercurial.dispatch.run()
+  File "/tmp/hgtests.xStzl7/install/lib/python/mercurial/dispatch.py",
line 16, in run
+    sys.exit(dispatch(sys.argv[1:]))
+  File "/tmp/hgtests.xStzl7/install/lib/python/mercurial/dispatch.py",
line 27, in dispatch
+    return _runcatch(u, args)
+  File "/tmp/hgtests.xStzl7/install/lib/python/mercurial/dispatch.py",
line 43, in _runcatch
+    return _dispatch(ui, args)
+  File "/tmp/hgtests.xStzl7/install/lib/python/mercurial/dispatch.py",
line 449, in _dispatch
+    return runcommand(lui, repo, cmd, fullargs, ui, options, d)
+  File "/tmp/hgtests.xStzl7/install/lib/python/mercurial/dispatch.py",
line 317, in runcommand
+    ret = _runcommand(ui, options, cmd, d)
+  File "/tmp/hgtests.xStzl7/install/lib/python/mercurial/dispatch.py",
line 496, in _runcommand
+    stats.pprint(top=10, file=ostream, climit=5)
+  File "/tmp/hgtests.xStzl7/install/lib/python/mercurial/lsprof.py",
line 48, in pprint
+    e.inlinetime, label(e.code)))
+  File "/tmp/hgtests.xStzl7/install/lib/python/mercurial/lsprof.py",
line 88, in label
+    for k, v in sys.modules.iteritems():
+RuntimeError: dictionary changed size during iteration
+--profile + output to file failed

While iterating over the sys.modules dict and querying all the modules
properties, if one of the modules is partially loaded there is a chance of
importing new modules while the iteration hasn't finished.

This will raise a RuntimeError since the import of new modules modifies the
sys.modules dict. In particular, this issue is caused by the email.MIMEAudio
module which is partially loaded using the email.LazyImporter class:

$ python2.6
Python 2.6.2 (r262:71600, Jun  8 2009, 09:30:33)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> import email.MIMEAudio
>>> email.MIMEAudio
<email.LazyImporter object at 0x7fd13ac31e50>
>>> print len(sys.modules)
54
>>> email.MIMEAudio.__file__
'/home/alejo/apps/local/lib/python2.6/email/mime/audio.pyc'
>>> print len(sys.modules)
113

The query of one of the module's attribute forces the LazyImporter class to
fully load the module, which in turn that module imports it's dependencies,
adding new items to the sys.modules dict.

There is some current discussion on the mercurial-devel mailing list about
the best way to fix this issue.

----------
assignedto: alejolp
messages: 10245
nosy: alejolp, djc
priority: bug
status: unread
title: When demandimport is disabled the 'test-profile' test fails

____________________________________________________
Mercurial issue tracker <mercurial-bugs at selenic.com>
<http://mercurial.selenic.com/bts/issue1774>
____________________________________________________