Exceeding the windows API MAX_PATH limit

[0xC0000022L]

Hi,

sorry for picking up on an older topic. Martin pointed me to this
mailing list over at StackOverflow
<http://stackoverflow.com/questions/10753991>.

> Even if you might be able to morph mercurial into using the long path
> api variants of Windows, other tools - which includes explorer on
> Windows 7 (!) [1] - won't be able to handle paths with lengths > 260 anyway.
a.) there are alternative file managers that can handle path lengths
above MAX_PATH fine and some of those do in fact implement shell
extensions. So even for the shell extension part of TortoiseHg it would
make sense. Besides, the shell extension has nothing to do but to use
the respective Unicode APIs and pass on the paths. Likely it
b.) the underlying tools (Hg itself and Python underneath that) should
support Unicode long paths.

The unfortunate thing about it is that most people will never be willing
to prefix their paths with \\?\ or even give full path names for
starters. However, this is a prerequisite. So all the burden is on the
tool that uses those APIs to prefix "unprefixed" paths itself and for
displayed paths strip the prefix as a form of pretty-printing.

For a tool such as Mercurial the build of Python would have to fully
support long path names first (which at least the version of Python I
was looking at didn't seem to do). So the question whether Mercurial
should support it is more a question of whether Python _can_ support it.

However, I think it's not right to point to inferior tools (Windows
Explorer) and point out that they can't do it either :)

> So I'd recommend to make sure that *all* your other tools can handle
> long paths _first_ before proposing to change mercurial's working copy
> handling on Windows to use the long path api's.
This will likely never happen, because too many Windows developers are
still caught in a mode of development that was common in Windows 9x
times. This is why so many woke up to Windows Vista finding their
programs didn't work the same anymore, because MS had cut off some
backward-compatibility that it was artificially maintaining on previous
versions of Windows. However, if you did your job right starting with
Windows NT, your code should work seamlessly even on Vista and 7, that
is if you didn't rely on "quirks".

Side-note: it is not enough to link against the Unicode APIs. The prefix
has to be used explicitly. If this was not the case, programs such as
Windows Explorer would be automagically capable of handling long path names.

> As a side note: explorer on Windows 7 also can't handle file names with
> reserved names (e.g. the famous file named "aux"), even though you can
> create these by using the long path api's [2].
Long file paths literally circumvent the Win32 API. Win32 is merely a
subsystem in Windows, one of potentially many (e.g. SUA is the successor
of the POSIX subsystem). What happens when you pass your path down to a
Unicode API in the prefixed (\\?\) form is that a part of NTDLL will
convert that path into the native form of \??\ (no coincidence that the
length is the same). Which is understood by the Windows object manager
and actually refers to the object namespace (which, much like in unixoid
systems, has a single root). You can use a tool such as WinObj from
Sysinternals to get an idea of how that namespace looks.

Now to the workaround:

To those having the issue with long path names, it will often work to
use junction points to work around the limitation. Junction points are a
subset of so-called reparse points and reparsing takes place long after
the path was converted to native path names. Therfore the user mode
components that see the path will never get to see the (too-long)
expanded form of the reparsed path.

So if your repo is inside <C:\Documents and
Settings\0xC0000022L\Documents\Mercurial\project\...> you could create a
junction point (symlinks with Vista and later _may_ work too) and give
that a shorter name. Of course it will only have a limited effect, but
it may be enough for many of those cases.

There is another, little-known, workaround that will allow programs to
use short path names that originate in the transition of FAT to VFAT/FAT32.

Say you have a folder such as this one:
<E:\1\ldskcnskfdlbvcsdfjkcbvsdjckvbsdjckbvsdcjsdcbvsdjcbsdchjsbdcjsdkcbdschjsdbchdk\ldskcnskfdlbvcsdfjkcbvsdjckvbsdjckbvsdcjsdcbvsdjcbsdchjsbdcjsdkcbdschjsdbchdk\ldskcnskfdlbvcsdfjkcbvsdjckvbsdjckbvsdcjsdcbvsdjcbsdchjsbdcjsdkcbdschjsdbchdk\ldskcnskfdlbvcsdfjkcbvsdjckvbsdjckbvsdcjsdcbvsdjcbsdchjsbdcjsdkcbdschjsdbchdk\ldskcnskfdlbvcsdfjkcbvsdjckvbsdjckbvsdcjsdcbvsdjcbsdchjsbdcjsdkcbdschjsdbchdk\ldskcnskfdlbvcsdfjkcbvsdjckvbsdjckbvsdcjsdcbvsdjcbsdchjsbdcjsdkcbdschjsdbchdk\ldskcnskfdlbvcsdfjkcbvsdjckvbsdjckbvsdcjsdcbvsdjcbsdchjsbdcjsdkcbdschjsdbchdk\ldskcnskfdlbvcsdfjkcbvsdjckvbsdjckbvsdcjsdcbvsdjcbsdchjsbdcjsdkcbdschjsdbchdk\ldskcnskfdlbvcsdfjkcbvsdjckvbsdjckbvsdcjsdcbvsdjcbsdchjsbdcjsdkcbdschjsdbchdk\ldskcnskfdlbvcsdfjkcbvsdjckvbsdjckbvsdcjsdcbvsdjcbsdchjsbdcjsdkcbdschjsdbchdk>

Way longer than MAX_PATH. However, each of the path segments will have a
short name in default Windows configurations, such as:

<E:\1\LDSKCN~1\LDSKCN~1\LDSKCN~1\LDSKCN~1\LDSKCN~1\LDSKCN~1\LDSKCN~1\LDSKCN~1\LDSKCN~1\ldskcnskfdlbvcsdfjkcbvsdjckvbsdjckbvsdcjsdcbvsdjcbsdchjsbdcjsdkcbdschjsdbchdk>

this is how you can convince tools that don't know how to handle long
path names to handle path names much longer than they could normally
handle. The relevant Win32 APIs are: GetLongPathName and GetShortPathName.

Just my two cents :)