D5052: mercurial: strip function return type annotations (RFC)
indygreg (Gregory Szorc)
phabricator at mercurial-scm.org
Sat Oct 13 07:29:22 UTC 2018
indygreg created this revision.
Herald added subscribers: mercurial-devel, mjpieters.
Herald added a reviewer: hg-reviewers.
REVISION SUMMARY
THIS IS SUPER HACKY. DO NOT EVEN THINK ABOUT LANDING YET.
Now that we have a custom module importer on Python 2 and that importer
is using Python 3.7's tokenizer, it is possible for us to analyze the
source code token stream and transform it to valid Python 2.7 source
before it is compiled into a code object.
This commit teaches the token stream transformer to recognize the tokens
for a function return type annotation and to strip those tokens from the
token stream. As far as Python 2's code compiler is concerned, the original
source didn't contain a type annotation!
To prove it works, we define a type annotation in the dispatch module.
Python 2 is able to import the module just fine.
This feature means we could start adding type annotations throughout the
code base while still remaining Python 2 compatible.
But why stop at type annotations? We could also rewrite the token stream
so we can leverage other Python 3 only features. This includes "yield from"
super() without arguments, class definitions without "(object)", etc. We
could also "invert" our source transformer strategy and make the code base
native Python 3 and source transform to remain compatible with Python 2.
REPOSITORY
rHG Mercurial
REVISION DETAIL
https://phab.mercurial-scm.org/D5052
AFFECTED FILES
mercurial/__init__.py
mercurial/dispatch.py
CHANGE DETAILS
diff --git a/mercurial/dispatch.py b/mercurial/dispatch.py
--- a/mercurial/dispatch.py
+++ b/mercurial/dispatch.py
@@ -177,7 +177,7 @@
def _formatargs(args):
return ' '.join(procutil.shellquote(a) for a in args)
-def dispatch(req):
+def dispatch(req) -> int:
"""run the command specified in req.args; returns an integer status code"""
with tracing.log('dispatch.dispatch'):
if req.ferr:
diff --git a/mercurial/__init__.py b/mercurial/__init__.py
--- a/mercurial/__init__.py
+++ b/mercurial/__init__.py
@@ -308,12 +308,23 @@
import os
import struct
from hgdemandimport import py3tokenize as tokenize
+ from hgdemandimport import py3token as token
rootpath = os.path.normpath(os.path.join(os.path.dirname(__file__), '..'))
def replacetokens(tokens, fullname):
- for i, t in enumerate(tokens):
+ i = 0
+ while i < len(tokens):
+ t = tokens[i]
+
+ # Function return type annotation.
+ if t.type == token.OP and t.string == '->':
+ t2 = tokens[i + 1]
+ i += 2
+ continue
+
yield t
+ i += 1
# Header to add to bytecode files. This MUST be changed when
# ``replacetokens`` or any mechanism that changes semantics of module
To: indygreg, #hg-reviewers
Cc: mjpieters, mercurial-devel
More information about the Mercurial-devel
mailing list