[PATCH 1 of 3 RFC] mercurial: implement a source transforming module loader on Python 3

[timeless]

Simon King wrote:
> I don't think that's supposed to happen, is it?

Dunno. I don't claim to be a python expert.

I do actively switch between at least py26 py27 py3 and to a lesser
extent pypy and some other flavors. So I hit all sorts of lousy edges.

> Python should automatically invalidate .pyc files based on a magic number that
> changes when the format changes:
>
> https://hg.python.org/cpython/file/2.7/Python/import.c#l31

(py26)[timeless at gcc2-power8 crew]$ make local
python setup.py  \
  build_py -c -d . \
  build_ext  -i \
  build_hgexe  -i \
  build_mo
running build_py
byte-compiling ./mercurial/byterange.py to byterange.pyc
byte-compiling ./mercurial/archival.py to archival.pyc
byte-compiling ./mercurial/crecord.py to crecord.pyc
byte-compiling ./mercurial/windows.py to windows.pyc
byte-compiling ./mercurial/lsprof.py to lsprof.pyc
byte-compiling ./mercurial/similar.py to similar.pyc
byte-compiling ./mercurial/scmwindows.py to scmwindows.pyc
byte-compiling ./mercurial/verify.py to verify.pyc
byte-compiling ./mercurial/py3kcompat.py to py3kcompat.pyc
byte-compiling ./mercurial/win32.py to win32.pyc
byte-compiling ./mercurial/__modulepolicy__.py to __modulepolicy__.pyc
byte-compiling ./mercurial/hgweb/webutil.py to webutil.pyc
byte-compiling ./mercurial/hgweb/wsgicgi.py to wsgicgi.pyc
byte-compiling ./mercurial/httpclient/socketutil.py to socketutil.pyc
byte-compiling ./hgext/convert/subversion.py to subversion.pyc
byte-compiling ./hgext/convert/convcmd.py to convcmd.pyc
byte-compiling ./hgext/convert/hg.py to hg.pyc
byte-compiling ./hgext/convert/cvsps.py to cvsps.pyc
byte-compiling ./hgext/convert/bzr.py to bzr.pyc
byte-compiling ./hgext/convert/filemap.py to filemap.pyc
byte-compiling ./hgext/convert/common.py to common.pyc
byte-compiling ./hgext/convert/gnuarch.py to gnuarch.pyc
byte-compiling ./hgext/convert/cvs.py to cvs.pyc
byte-compiling ./hgext/convert/monotone.py to monotone.pyc
byte-compiling ./hgext/fsmonitor/pywatchman/pybser.py to pybser.pyc
byte-compiling ./hgext/largefiles/wirestore.py to wirestore.pyc
byte-compiling ./hgext/largefiles/basestore.py to basestore.pyc
byte-compiling ./hgext/largefiles/lfutil.py to lfutil.pyc
byte-compiling ./hgext/largefiles/localstore.py to localstore.pyc
byte-compiling ./hgext/largefiles/proto.py to proto.pyc
byte-compiling ./hgext/largefiles/remotestore.py to remotestore.pyc
running build_ext
building 'mercurial.bdiff' extension
creating build/temp.linux-ppc64le-2.6
creating build/temp.linux-ppc64le-2.6/mercurial
gcc -pthread -fno-strict-aliasing -g -O2 -DNDEBUG -g -fwrapv -O3 -Wall
-Wstrict-prototypes -fPIC
-I/home/timeless/.localpython26/include/python2.6 -c mercurial/bdiff.c
-o build/temp.linux-ppc64le-2.6/mercurial/bdiff.o
In file included from
/home/timeless/.localpython26/include/python2.6/Python.h:125:0,
                 from mercurial/bdiff.c:13:
/home/timeless/.localpython26/include/python2.6/modsupport.h:27:1:
warning: '_PyArg_ParseTuple_SizeT' is an unrecognized format function
type [-Wformat=]
 PyAPI_FUNC(int) PyArg_ParseTuple(PyObject *, const char *, ...)
Py_FORMAT_PARSETUPLE(PyArg_ParseTuple, 2, 3);
 ^
gcc -pthread -shared build/temp.linux-ppc64le-2.6/mercurial/bdiff.o -o
/home/timeless/hg/crew/mercurial/bdiff.so
running build_hgexe
running build_mo
env HGRCPATH= python hg version
Traceback (most recent call last):
  File "hg", line 41, in <module>
    mercurial.util.setbinary(fp)
  File "/home/timeless/hg/crew/mercurial/demandimport.py", line 130,
in __getattribute__
    self._load()
  File "/home/timeless/hg/crew/mercurial/demandimport.py", line 96, in _load
    mod = _hgextimport(_import, head, globals, locals, None, level)
  File "/home/timeless/hg/crew/mercurial/demandimport.py", line 53, in
_hgextimport
    return importfunc(name, globals, *args, **kwargs)
  File "/home/timeless/hg/crew/mercurial/util.py", line 2636, in <module>
    if safehasattr(parsers, 'dirs'):
  File "/home/timeless/hg/crew/mercurial/util.py", line 136, in safehasattr
    return getattr(thing, attr, _notset) is not _notset
  File "/home/timeless/hg/crew/mercurial/demandimport.py", line 130,
in __getattribute__
    self._load()
  File "/home/timeless/hg/crew/mercurial/demandimport.py", line 96, in _load
    mod = _hgextimport(_import, head, globals, locals, None, level)
  File "/home/timeless/hg/crew/mercurial/demandimport.py", line 53, in
_hgextimport
    return importfunc(name, globals, *args, **kwargs)
  File "/home/timeless/hg/crew/mercurial/__init__.py", line 143, in load_module
    mod = imp.load_module(name, *modinfo)
ImportError: Python minor version mismatch: The Mercurial extension
modules were compiled with Python 2.7.8, but Mercurial is currently
using Python with sys.hexversion=33950192: Python 2.6.9 (unknown, Apr
13 2016, 12:40:12)
[GCC 4.9.2 20141101 (Red Hat 4.9.2-1)]
 at: /home/timeless/hg/py26/bin/python
Makefile:47: recipe for target 'local' failed
make: *** [local] Error 1

I expect someone to hand waive this as a "build system issue", or as a
"compiled library problem". But from my perspective, switching between
python versions is not fun.