Exceeding the windows API MAX_PATH limit

[Adrian Buehlmann]

On 2012-04-20 12:59, Angel Ezquerra wrote:
> Hi,
> 
> I've recently had some users that have shot themselves in the foot by
> add some files with really long paths to a mercurial repo. The file
> paths were longer than the windows API MAX_PATH, which is 260.
> 
> The problem occurred when importing some data from a clearcase
> repository into mercurial. The ClearCase repository contained some
> files with very long paths (more than 200 characters). However,
> because ClearCase forces you (in our setup) to place all your
> repositories on some folder right on your C:\ drive, the total length
> of those file paths did not exceed 260 characters.
> 
> A user created a mercurial repository right on the clearcase
> repository folder. Everything was added fine because in there all file
> paths were below MAX_PATH. He then cloned this repo into our shared
> server (into c:\mercurial\bad_repo). Again, all paths were below
> MAX_PATH.
> 
> The problem came when another user tried to clone this repo into its
> machine. This user places all his repositories in "My
> Documents\repositories" (which I think is common for windows users).
> The clone went fine until it tried to update to tip, which failed
> because mercurial could not create some files whose paths would exceed
> MAX_PATH. The user can clone the repo right into its C:\ root without
> any issues.
> 
> I think this problem is particularly tricky because the user making a
> commit cannot tell if it will fail in someone else's machine!
> 
> So far I've come up with 4 solutions, none of which is really satisfactory:
> 
> 1. Rename the files: it may be feasible in this particular case but
> there may be cases where this is just not possible (e.g. for source
> code that you do not control)
> 2. Always clone into the c:\ root or at least have all your repos in
> "c:\repos" or a similar path: This may be fine (and good advice) in
> general, but may not be ok on multi-user machines and in some cases
> you may not be able to chose where to clone into...
> 3. Use the subst command to create a temporary drive and work in
> there: I've seen this suggested online, but I don't know if this will
> cause problems since the files will live both on the "virtual drive"
> and on the original deep folder.
> 4. Use folder junctions: same issues as 3?
> 
> Do you guys have any other solution?
> 
> BTW, I've looked for info on this MAX_PATH limitation and it seems
> that this 260 MAX_PATH limit is a limitation of the windows API, not
> of the underlying NTFS file system, which can support files of up to
> 32,767 characters (with a limit of 255 per individual 'path element').
> Apparently the unicode version of the windows API _mostly_ supports
> long paths. It seems that you can access the unicode APIs by
> prepending your paths with "\\?\":
> 
> http://msdn.microsoft.com/en-us/library/aa365247%28VS.85%29.aspx#maxpath
> http://blogs.msdn.com/b/bclteam/archive/2007/02/13/long-paths-in-net-part-1-of-3-kim-hamilton.aspx
> 
> The second link is particularly interesting (specially part 2).
> Unfortunately a quick test shows that python's open function does not
> support this.

This kind of discussion pops up from time to time.

I think the most important aspect today is still this:

Even if you might be able to morph mercurial into using the long path
api variants of Windows, other tools - which includes explorer on
Windows 7 (!) [1] - won't be able to handle paths with lengths > 260 anyway.

So I'd recommend to make sure that *all* your other tools can handle
long paths _first_ before proposing to change mercurial's working copy
handling on Windows to use the long path api's.

As a side note: explorer on Windows 7 also can't handle file names with
reserved names (e.g. the famous file named "aux"), even though you can
create these by using the long path api's [2].

[1]
https://bitbucket.org/abuehl/downloads/downloads/path-too-long-Windows-7-explorer.PNG
[2]
https://bitbucket.org/abuehl/downloads/downloads/Windows7-Explorer-delete-aux-1.PNG