Type checking is an important part of making Mercurial compatible with Python 3. This page describes the workflow, recommendations and gotchas.

Setup

Mercurial source code files have to remain compatible with Python 2 for the foreseeable future. This is why we are storing type annotations in external .pyi files (called "stubs"). We then reapply them using retype for the purpose of type checking using mypy.

Types are stored in a root directory called, well, types. The output of the retype command lands in an .hgignore'd directory called typed-src.

Type hints

Never used type hinting with Python before? Start here: http://mypy.readthedocs.io/en/latest/cheat_sheet_py3.html

You will need Python 3.6. Download mypy. If you're a flake8 user, upgrade to the latest version and install the flake8-pyi plugin.

Creating a new .pyi file

The simplest way to do it is to start with the stubgen script available in the mypy GitHub repo. It is crude but gets the job done. It recreates the basic structure of the given source file in the form of a .pyi file. Note that this .pyi file is missing most annotations, and the ones that it **does** add are likely wrong (in most cases defaulting to the Any type). Your job is to fill the blanks in with actual typing information.

Our .pyi files describe Python 3.5's type system and standard library. Not Python 2.7 because the purpose of this exercise is to find all the places in which the code currently doesn't play well with Python 3. Not Python 3.6 because the initial version of Python 3 that Mercurial is targetting is 3.5. That being said, syntax-wise, you are recommended to use variable annotations in .pyi files since they are cleaner and type checkers and retype use them anyway.

Speaking of .pyi files, you might have noticed that they are syntactically valid Python, with the single difference that they natively support forward references (in other words, you can specify definitions out of order without consequence). Stylistically, .pyi files are not meant to follow PEP 8. Instead, they conform to the typeshed coding style.

Mercurial ships with mercurial.pycompat, a library providing a number of compatibility bridges between Python 2 and Python 3. While this library is imported implicitly at runtime (via a magic import hook), we don't do that for the purposes of type checking. So, if needed, add the following import in your .pyi files:

from mercurial.pycompat import delattr, getattr, hasattr, setattr, xrange, open

retype

NOTE: This tool is very new. If you hit any issues, open an issue on GitHub or contact LukaszLanga directly.

From the root of the hg repo run:

$ retype --hg --quiet mercurial

A retype run does the following:

Re-application of types requires copying typing imports and alias definitions from the .pyi file, which is why line numbers will no longer match original source code. It retains its original formatting though.

Some design principles:

Type checking

This is what you came here for, right? Once sources are translated in typed-src/, run:

$ cd typed-src/
$ mypy .


CategoryDeveloper

TypeChecking (last edited 2017-05-02 01:56:29 by JunWu)