[PATCH] scmutil: create directories in a race-safe way during update

[Thomas Arendsen Hein]

* Bryan O'Sullivan <bos at serpentine.com> [20130212 01:16]:
> # HG changeset patch
> # User Bryan O'Sullivan <bryano at fb.com>
> # Date 1360628112 28800
> # Node ID 4034b8d551b116b882920466e1d209e475a62721
> # Parent  f12804d3ff801b989cb2aab1aad93047a8db46f1
> scmutil: create directories in a race-safe way during update

> +def ensuredirs(name, mode=None):
> +    """race-safe recursive directory creation"""
> +    try:
> +        makedirs(name, mode)
> +    except OSError, err:
> +        if err.errno == errno.EEXIST and os.path.isdir(name):
> +            # someone else seems to have won a directory creation race
> +            return
> +        raise
> +

This is not completely race-safe as makedirs uses recursion, which
can abort too early when one worker is much faster than the other:

Concurrent runs of ensuredirs('a/b/c') and ensuredirs('a/b/d'),
initially 'a' does not exist:

Worker 1:                        | Worker 2:
---------------------------------+----------------------------
mkdir1('a/b/c')                  | mkdir1('a/b/d')
-> ENOENT                        | -> ENOENT
makedirs('a/b')                  |
        mkdir1('a/b')            |
        -> ENOENT                |
                                 | makedirs('a/b')
                                 |         mkdir1('a/b')
                                 |         -> ENOENT
                                 |         makedirs('a')
                                 |                 mkdir1('a')
                                 |                 -> OK
                                 |         mkdir2('a/b')
                                 |         -> OK
                                 | mkdir2('d')
        makedirs('a')            |
        mkdir2('a/b')            |
        -> EEXIST                |
mkdir2('a/b/c') is not executed! |
OSError with EEXIST is raised.   |

mkdir1 is the call to os.mkdir in the try part of makedirs(), mkdir2
is the call in the except part.

Regards,
Thomas

-- 
thomas at intevation.de - http://intevation.de/~thomas/ - OpenPGP key: 0x5816791A
Intevation GmbH, Neuer Graben 17, 49074 Osnabrueck - AG Osnabrueck, HR B 18998
Geschaeftsfuehrer: Frank Koormann, Bernhard Reiter, Dr. Jan-Oliver Wagner