D6973: mangler: stop rewriting string constants to be bytes literals

durin42 (Augie Fackler) phabricator at mercurial-scm.org
Sat Oct 5 10:06:24 EDT 2019


durin42 created this revision.
Herald added a subscriber: mercurial-devel.
Herald added a reviewer: hg-reviewers.

REVISION SUMMARY
  We've rewritten everything, so we no longer require this step.

REPOSITORY
  rHG Mercurial

REVISION DETAIL
  https://phab.mercurial-scm.org/D6973

AFFECTED FILES
  mercurial/__init__.py

CHANGE DETAILS

diff --git a/mercurial/__init__.py b/mercurial/__init__.py
--- a/mercurial/__init__.py
+++ b/mercurial/__init__.py
@@ -153,37 +153,6 @@
                 tokens[j] = st._replace(string=b"u%s" % st.string)
 
         for i, t in enumerate(tokens):
-            # Convert most string literals to byte literals. String literals
-            # in Python 2 are bytes. String literals in Python 3 are unicode.
-            # Most strings in Mercurial are bytes and unicode strings are rare.
-            # Rather than rewrite all string literals to use ``b''`` to indicate
-            # byte strings, we apply this token transformer to insert the ``b``
-            # prefix nearly everywhere.
-            if t.type == token.STRING:
-                s = t.string
-
-                # Preserve docstrings as string literals. This is inconsistent
-                # with regular unprefixed strings. However, the
-                # "from __future__" parsing (which allows a module docstring to
-                # exist before it) doesn't properly handle the docstring if it
-                # is b''' prefixed, leading to a SyntaxError. We leave all
-                # docstrings as unprefixed to avoid this. This means Mercurial
-                # components touching docstrings need to handle unicode,
-                # unfortunately.
-                if s[0:3] in (b"'''", b'"""'):
-                    yield t
-                    continue
-
-                # If the first character isn't a quote, it is likely a string
-                # prefixing character (such as 'b', 'u', or 'r'. Ignore.
-                if s[0] not in (b"'", b'"'):
-                    yield t
-                    continue
-
-                # String literal. Prefix to make a b'' string.
-                yield t._replace(string=b"b%s" % t.string)
-                continue
-
             # Insert compatibility imports at "from __future__ import" line.
             # No '\n' should be added to preserve line numbers.
             if (
@@ -252,7 +221,7 @@
     # ``replacetoken`` or any mechanism that changes semantics of module
     # loading is changed. Otherwise cached bytecode may get loaded without
     # the new transformation mechanisms applied.
-    BYTECODEHEADER = b"HG\x00\x0c"
+    BYTECODEHEADER = b"HG\x00\x0d"
 
     class hgloader(importlib.machinery.SourceFileLoader):
         """Custom module loader that transforms source code.



To: durin42, #hg-reviewers
Cc: mercurial-devel


More information about the Mercurial-devel mailing list