[RFC] Adding Py3k support while maintaining compatibility with python <= 2.6

Matt Mackall mpm at selenic.com
Wed Jun 2 17:00:37 CDT 2010


On Wed, 2010-06-02 at 17:17 -0300, Renato Cunha wrote:
> Hi,
> 
> As some of you might already know, python 3 has introduced a few
> incompatibilities with the 2.x version of python. Also, my GSoC project is to
> provide some foundations to enable hg to run on python 3.
> 
> After the meeting that was held yesterday I still am not sure if the preferred
> method for adding this support. What would be better? To use 2to3 (a script for
> converting python 2 "to" 3) or to add py3k support while still maintaining
> compatibility with 2.4 and 2.5 in mercurial. As I mentioned in the meeting, I
> am aiming for maintaining the compatibility (even though that means a bit more
> work) with 2.4 (and 2.5).* 
> 
> The issues I've identified so far are:
>  * octal constants
>  * exception handling
>  * print statements (I won't talk about these in this email)
>  * strings/unicode/bytes (I won't talk about these in this email)
> 
> So, below I'll describe my approach for handling the octal constants and the
> exception syntax:
> 
> -*-
> 
> Octal constants have a different syntax in python 3, instead of defining an
> octal constant begging with "0", now one needs "0o" for it, so "0777" is a syntax
> error in python 3, and "0o777" is a syntax error in python < 2.6.

WTF, Guido?

> My idea for tackling with this problem is based on the following:
> 
> There are some octal constants spread on some files in mercurial source, my
> preliminary patches for util.py define a bunch of module constants to operate.
> As an example, code that once was
> 
>   st_mode &= 0666
> 
> has become
> 
>   st_mode &= MODE_666
> 
> with MODE_666 being defined as
> 
>   MODE_666 = int('666', 8)
> 
> at the beginning of util.py.
> 
> With this approach there is an advantage of knowing what is the intended use of
> the constant by reading its name. What do you think of it approach? There are
> other octal constants spread on other modules, should them all be defined on
> util.py or locally?

Step one:

diff -r 739b5e3c22c3 contrib/check-code.py
--- a/contrib/check-code.py	Tue Jun 01 11:18:57 2010 -0500
+++ b/contrib/check-code.py	Wed Jun 02 15:54:29 2010 -0500
@@ -71,6 +71,7 @@
 
 pypats = [
     (r'^\s*\t', "don't use tabs"),
+    (r'\b0\d+\b', "octal constant breaks py3k"),
     (r'\S;\s*\n', "semicolon"),
     (r'\w,\w', "missing whitespace after ,"),
     (r'\w[+/*\-<>]\w', "missing whitespace in expression"),


$ contrib/check-code.py `hg manifest`
hgext/convert/darcs.py:163:
 >         mode = (mode & 0111) and 'x' or ''
 octal constant breaks py3k
hgext/convert/gnuarch.py:214:
 >             mode = (mode & 0111) and 'x' or ''
 octal constant breaks py3k
[and a bunch more deleted]

Step 2: fix the false positives in the above approach.

Check-code is our first line of defense against compatibility problems
sneaking back in.

> -*-
> 
> As for exception handling, in Mercurial, some of the exception handlers try to
> instantiate the exception object as in the example below (ignore the print
> syntax for now):
> 
> foo.py:
>     import os
>     try:
>         os.unlink('/non/existing/path/foo')
>     except OSError, err:
>         print ('There was an error:')
>         print (err)
> 
> If you run it with 2.4 through 2.6, you get the correct output:
> 
> $ python2.4 foo.py 
> There was an error:
> [Errno 2] No such file or directory: '/non/existing/path/foo'
> 
> But, with python 3(.1):
> 
> $ python3 foo.py 
>   File "foo.py", line 7
>     except OSError, err:
>                   ^
> SyntaxError: invalid syntax

What's the correct syntax? Oh, I see:

except OSError as err:

Guido says (http://www.artima.com/weblogs/viewpost.jsp?thread=211200):

Q. I prefer to use the same source code for 2.x and 3.0; I really don't
want to have to use the 2to3 source conversion tool. Why can't you make
that work?

A. Suit yourself. The intersection of 2.6 and 3.0 is large, but there
are several things you can't do: you can't use Unicode literals (2.6
only), nor bytes literals (3.0 only). The only print syntax that works
the same in both versions is print(x). When you catch exceptions you
can't inspect their values, because the 2.6 syntax uses , while 3.0 uses
as. You can't use .iterkeys(), but .keys() works differently in 2.6 and
3.0. You can't use xrange(), but range() works differently. You can't
use metaclasses, as the syntax for specifying a metaclass is completely
changed in 3.0. And so on. Restricting yourself to the intersection of
the two versions is very painful and limited. We're not introducing
backwards compatibility syntax in 3.0, because that would defeat the
purpose (we've had backwards compatibility forever in 2.x, and the whole
point of 3.0 is to clean up the mess).


Some of these are probably fine for us (iterkeys, for instance), but
others like xrange may not be.

> Now, if instead of using a specific syntax for getting the exception value, we
> make a call to sys.exc_info in the exception handler, we get the same result
> and consistent behavior in both pythons, as in the example below:
> 
> bar.py:
>     import os, sys
>     try:
>         os.unlink('/non/existing/path/foo')
>     except OSError:
>         err = sys.exc_info()[1]
>         print ('There was an error:')
>         print (err)
> 
> If we run it, we get the expected:
> 
> $ python2.4 bar.py 
> There was an error:
> [Errno 2] No such file or directory: '/non/existing/path/foo'
> 
> and
> 
> $ python3 bar.py 
> There was an error:
> [Errno 2] No such file or directory: '/non/existing/path/foo'
> 
> The major drawback here is that the call to sys.exc_info()[1] is a bit cryptic,
> and a call to a function eventually defined in util.py would be better, like:
> 
> foobar.py:
>     def exceptionvalue():
>         return sys.exc_info()[1]
>     try:
>         os.unlink('/non/existing/path/foo')
>     except OSError:
>         err = exceptionvalue()
>         print ('There was an error:')
>         print (err)
> 
> So, is this new approach good? What about the function name (exceptionvalue)?
> Any other suggestions?
> 
> -*-
> 
> * The "recommended" way for porting a python application for py3k according to
> PEP3000 is to first port the application to python 2.6 and, then, use 2to3 plus
> the -3 switch until there's a product that runs on both python 2 and 3, though
> I suspect 2to3 would have trouble with mercurial and its byte-oriented
> operations.
> 
> Regards,
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel at selenic.com
> http://selenic.com/mailman/listinfo/mercurial-devel



-- 
Mathematics is the supreme nostalgia of our time.




More information about the Mercurial-devel mailing list