[PATCH 1 of 5 RFC] util: add platform-agnostic absjoin function and some long-paths win utils

Gregory Szorc gregory.szorc at gmail.com
Sun Aug 13 18:35:03 EDT 2017



> On Aug 12, 2017, at 01:22, Kostia Balytskyi <ikostia at fb.com> wrote:
> 
> # HG changeset patch
> # User Kostia Balytskyi <ikostia at fb.com>
> # Date 1502481241 25200
> #      Fri Aug 11 12:54:01 2017 -0700
> # Node ID 586870aa06799e2ef49a2be4c8feab1464ad54b1
> # Parent  609606d217659e0a6c1cf6f907b6512be5340e57
> util: add platform-agnostic absjoin function and some long-paths win utils
> 
> The goal of this seies is to add support for long paths on Windows.
> Context here is that Win32 API CreateFileA (which opens a file handle and
> which is what CreateFile macro is resolved to by default) does not recognize
> paths longer than MAX_PATH constant (260 chars), unless the path starts
> with \\?\, which causes API to not do any checks or preprocessing of the
> file path. Therefore, it is posible to use long paths with this prefix,
> but it comes at a cost: forward slashes are not automatically replaced by
> baclslashes, . and .. are not automatically resolved to appropriate dirs.
> 
> Also, it is hard to change paths to use //?/ everywhere, so I propose that
> the approach is to handle both types of paths simultaneously.
> 
> Also, even if this series is not accepted as it is, I would like to start
> a discussion about solving this problem in general.
> 
> This particular patch adds some first steps in reaching this goal:
> - it adds helper functions hasntprefix and converttontpath
> - it adds absjoin function which is supposed to replace _join method
> of dirstate

Cool series! This feature is long overdue.

At the point proper path support on Windows is fully implemented in Mercurial, how much of Python 3's pathlib have we reinvented? In other words, should we start coding to the pathlib API now so all kinds of path operations "just work?"

> 
> diff --git a/mercurial/posix.py b/mercurial/posix.py
> --- a/mercurial/posix.py
> +++ b/mercurial/posix.py
> @@ -666,3 +666,9 @@ def bindunixsocket(sock, path):
>     if bakwdfd:
>         os.fchdir(bakwdfd)
>         os.close(bakwdfd)
> +
> +def absjoin(absprefix, relpath):
> +    """Join a relative path to an absolute path. This function assumes that
> +    the passed absolute path already contains a terminating directory
> +    separator."""
> +    return absprefix + relpath
> diff --git a/mercurial/util.py b/mercurial/util.py
> --- a/mercurial/util.py
> +++ b/mercurial/util.py
> @@ -149,6 +149,7 @@ testpid = platform.testpid
> umask = platform.umask
> unlink = platform.unlink
> username = platform.username
> +absjoin = platform.absjoin
> 
> try:
>     recvfds = osutil.recvfds
> diff --git a/mercurial/windows.py b/mercurial/windows.py
> --- a/mercurial/windows.py
> +++ b/mercurial/windows.py
> @@ -236,6 +236,28 @@ def normpath(path):
> def normcase(path):
>     return encoding.upper(path) # NTFS compares via upper()
> 
> +def converttontpath(path):
> +    """Prepend \\?\ prefix to a path and perform slash replacements
> +    The '\\?\' prefix tells Win32 API to not perform userland path checks,
> +    therefore allowing long paths. This also means that / are not replaced
> +    with \ by Win32, so we need to do it manually. See MSDN entry on
> +    CreateFile function for more details"""
> +    return '\\\\?\\' + os.path.normpath(path)
> +
> +def hasntprefix(path):
> +    """Check if path starts with a \\?\ prefix"""
> +    return path.startswith('\\\\?\\')
> +
> +def absjoin(absprefix, relpath):
> +    """See docblock for posix.absjoin for explanation of what this does."""
> +    if hasntprefix(absprefix):
> +        # we assume that if absprefix starts with \\?\, this means that it
> +        # was already preprocessed by converttontpath and therefore does
> +        # not need to passed to `localpath`
> +        return absprefix + localpath(relpath)
> +    else:
> +        return converttontpath(absprefix + relpath)
> +
> # see posix.py for definitions
> normcasespec = encoding.normcasespecs.upper
> normcasefallback = encoding.upperfallback
> _______________________________________________
> Mercurial-devel mailing list
> Mercurial-devel at mercurial-scm.org
> https://www.mercurial-scm.org/mailman/listinfo/mercurial-devel


More information about the Mercurial-devel mailing list