D5052: mercurial: strip function return type annotations (RFC)

indygreg (Gregory Szorc) phabricator at mercurial-scm.org
Sat Oct 13 07:29:22 UTC 2018


indygreg created this revision.
Herald added subscribers: mercurial-devel, mjpieters.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  THIS IS SUPER HACKY. DO NOT EVEN THINK ABOUT LANDING YET.
  
  Now that we have a custom module importer on Python 2 and that importer
  is using Python 3.7's tokenizer, it is possible for us to analyze the
  source code token stream and transform it to valid Python 2.7 source
  before it is compiled into a code object.
  
  This commit teaches the token stream transformer to recognize the tokens
  for a function return type annotation and to strip those tokens from the
  token stream. As far as Python 2's code compiler is concerned, the original
  source didn't contain a type annotation!
  
  To prove it works, we define a type annotation in the dispatch module.
  Python 2 is able to import the module just fine.
  
  This feature means we could start adding type annotations throughout the
  code base while still remaining Python 2 compatible.
  
  But why stop at type annotations? We could also rewrite the token stream
  so we can leverage other Python 3 only features. This includes "yield from"
  super() without arguments, class definitions without "(object)", etc. We
  could also "invert" our source transformer strategy and make the code base
  native Python 3 and source transform to remain compatible with Python 2.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D5052

AFFECTED FILES
  mercurial/__init__.py
  mercurial/dispatch.py

CHANGE DETAILS

diff --git a/mercurial/dispatch.py b/mercurial/dispatch.py
--- a/mercurial/dispatch.py
+++ b/mercurial/dispatch.py
@@ -177,7 +177,7 @@
 def _formatargs(args):
     return ' '.join(procutil.shellquote(a) for a in args)
 
-def dispatch(req):
+def dispatch(req) -> int:
     """run the command specified in req.args; returns an integer status code"""
     with tracing.log('dispatch.dispatch'):
         if req.ferr:
diff --git a/mercurial/__init__.py b/mercurial/__init__.py
--- a/mercurial/__init__.py
+++ b/mercurial/__init__.py
@@ -308,12 +308,23 @@
     import os
     import struct
     from hgdemandimport import py3tokenize as tokenize
+    from hgdemandimport import py3token as token
 
     rootpath = os.path.normpath(os.path.join(os.path.dirname(__file__), '..'))
 
     def replacetokens(tokens, fullname):
-        for i, t in enumerate(tokens):
+        i = 0
+        while i < len(tokens):
+            t = tokens[i]
+
+            # Function return type annotation.
+            if t.type == token.OP and t.string == '->':
+                t2 = tokens[i + 1]
+                i += 2
+                continue
+
             yield t
+            i += 1
 
     # Header to add to bytecode files. This MUST be changed when
     # ``replacetokens`` or any mechanism that changes semantics of module



To: indygreg, #hg-reviewers
Cc: mjpieters, mercurial-devel


More information about the Mercurial-devel mailing list