[PATCH] win32: improve the performance of win32.unlink() over CIFS

Kaz Nishimura kazssym at vx68k.org
Tue Feb 4 00:55:51 CST 2014


# HG changeset patch
# User Kaz Nishimura <kazssym at vx68k.org>
# Date 1381984037 -32400
#      Thu Oct 17 13:27:17 2013 +0900
# Branch stable
# Node ID be24685ef8ae89aa89964fc0b147260325b309d9
# Parent  7d269e7620c48eee93871d33e673b7a8acb503cc
win32: improve the performance of win32.unlink() over CIFS

Emulating POSIX unlink() behavior with os.rename() and os.unlink() is often slow
especially over CIFS from Windows clients due to its protocol overhead. This
patch changes win32.unlink() to try first an exclusive open with the Win32
delete-on-close flag, and if a sharing violation is detected, to fall back to
the original emulation.

This patch also removes a test with os.path.isdir() since we expect opening a
directory shall fail as os.unlink() would.

Example measurements (repeated 3-times after 1-time calibration):

(Without this patch: hg update from null to default)
127 files updated, 0 files merged, 0 files removed, 0 files unresolved
time: real 19.871 secs (user 0.328+0.000 sys 1.794+0.000)
time: real 19.622 secs (user 0.312+0.000 sys 2.044+0.000)
time: real 19.138 secs (user 0.250+0.000 sys 1.872+0.000)

(Without this patch: hg update from default to null)
0 files updated, 0 files merged, 127 files removed, 0 files unresolved
time: real 35.158 secs (user 0.156+0.000 sys 2.512+0.000)
time: real 35.272 secs (user 0.250+0.000 sys 2.512+0.000)
time: real 36.569 secs (user 0.203+0.000 sys 2.387+0.000)

(With this patch: hg update from null to default)
127 files updated, 0 files merged, 0 files removed, 0 files unresolved
time: real 17.893 secs (user 0.328+0.000 sys 1.700+0.000)
time: real 18.512 secs (user 0.265+0.000 sys 1.529+0.000)
time: real 20.238 secs (user 0.312+0.000 sys 1.685+0.000)

(With this patch: hg update from default to null)
0 files updated, 0 files merged, 127 files removed, 0 files unresolved
time: real 12.312 secs (user 0.250+0.000 sys 0.811+0.000)
time: real 12.471 secs (user 0.156+0.000 sys 0.889+0.000)
time: real 9.727 secs (user 0.125+0.000 sys 0.858+0.000)

diff -r 7d269e7620c4 -r be24685ef8ae mercurial/win32.py
--- a/mercurial/win32.py
+++ b/mercurial/win32.py
@@ -25,4 +25,5 @@
 # GetLastError
 _ERROR_SUCCESS = 0
+_ERROR_SHARING_VIOLATION = 32
 _ERROR_INVALID_PARAMETER = 87
 _ERROR_INSUFFICIENT_BUFFER = 122
@@ -60,5 +61,7 @@
 _OPEN_EXISTING = 3
 
+_FILE_FLAG_OPEN_REPARSE_POINT = 0x00200000
 _FILE_FLAG_BACKUP_SEMANTICS = 0x02000000
+_FILE_FLAG_DELETE_ON_CLOSE = 0x04000000
 
 # SetFileAttributes
@@ -345,9 +348,16 @@
     '''try to implement POSIX' unlink semantics on Windows'''
 
-    if os.path.isdir(f):
-        # use EPERM because it is POSIX prescribed value, even though
-        # unlink(2) on directories returns EISDIR on Linux
-        raise IOError(errno.EPERM,
-                      "Unlinking directory not permitted: '%s'" % f)
+    # If we can open f exclusively, no other processes must have open handles
+    # for it and we can expect its name will be deleted immediately when we
+    # close the handle unless we have another in the same process.  We also
+    # expect we shall simply fail to open f if it is a directory.
+    fh = _kernel32.CreateFileA(f, 0, 0, None, _OPEN_EXISTING,
+        _FILE_FLAG_OPEN_REPARSE_POINT | _FILE_FLAG_DELETE_ON_CLOSE, None)
+    if fh != _INVALID_HANDLE_VALUE:
+        _kernel32.CloseHandle(fh)
+        return
+    error = _kernel32.GetLastError()
+    if error != _ERROR_SHARING_VIOLATION:
+        raise ctypes.WinError(error)
 
     # POSIX allows to unlink and rename open files. Windows has serious


More information about the Mercurial-devel mailing list