[PATCH 1 of 3] serve: add and use portable spawnvp replacement

[Patrick Mézard]

Le 09/01/10 07:09, Steve Borho a écrit :
> On Wed, Jan 6, 2010 at 2:59 PM, Patrick Mezard <pmezard at gmail.com> wrote:
>> # HG changeset patch
>> # User Patrick Mezard <pmezard at gmail.com>
>> # Date 1239391225 -7200
>> # Node ID 92ac4240b6e0e3c1b959eb481f0e058e61ee705a
>> # Parent  3ab391dd5ec5ab1f4af06169bde7a9c850a7f4ce
>> serve: add and use portable spawnvp replacement
>>
>> There is no standard python command to really detach a process
>> under Windows. Instead we use the low level API wrapped by
>> subprocess module with all necessary options to avoid any kind
>> of context inheritance.
>>
>> Fix 1/3 for issue421
>>
>> diff --git a/mercurial/cmdutil.py b/mercurial/cmdutil.py
>> --- a/mercurial/cmdutil.py
>> +++ b/mercurial/cmdutil.py
>> @@ -580,8 +580,7 @@
>>             elif runargs[i].startswith('--cwd'):
>>                 del runargs[i:i+2]
>>                 break
>> -        pid = os.spawnvp(os.P_NOWAIT | getattr(os, 'P_DETACH', 0),
>> -                         runargs[0], runargs)
>> +        pid = util.spawndetached(runargs)
>>         os.close(wfd)
>>         os.read(rfd, 1)
>>         if parentfn:
>> diff --git a/mercurial/posix.py b/mercurial/posix.py
>> --- a/mercurial/posix.py
>> +++ b/mercurial/posix.py
>> @@ -245,3 +245,8 @@
>>         return grp.getgrgid(gid)[0]
>>     except KeyError:
>>         return str(gid)
>> +
>> +def spawndetached(args):
>> +    return os.spawnvp(os.P_NOWAIT | getattr(os, 'P_DETACH', 0),
>> +                      args[0], args)
>> +
>> diff --git a/mercurial/windows.py b/mercurial/windows.py
>> --- a/mercurial/windows.py
>> +++ b/mercurial/windows.py
>> @@ -7,7 +7,7 @@
>>
>>  from i18n import _
>>  import osutil, error
>> -import errno, msvcrt, os, re, sys, random
>> +import errno, msvcrt, os, re, sys, random, subprocess
>>
>>  nulldev = 'NUL:'
>>  umask = 002
>> @@ -59,6 +59,16 @@
>>             self.close()
>>             raise IOError(errno.EPIPE, 'Broken pipe')
>>
>> +    # Standard streams descriptors are set to -1 in child processes
>> +    # not inheriting their parent handles. And raise IOError instead of
>> +    # ignoring read/write calls.
>> +    if not hasattr(sys.stdin, 'fileno') or sys.stdin.fileno() == -1:
>> +        sys.stdin = open(nulldev, 'r')
>> +    if not hasattr(sys.stderr, 'fileno') or sys.stderr.fileno() == -1:
>> +        sys.stderr = open(nulldev, 'w')
>> +    if not hasattr(sys.stdout, 'fileno') or sys.stdout.fileno() == -1:
>> +        sys.stdout = open(nulldev, 'w')
>> +
>>  sys.stdout = winstdout(sys.stdout)
>>
>>  def _is_win_9x():
>> @@ -321,6 +331,35 @@
>>             pass
>>         os.rename(src, dst)
>>
>> +def spawndetached(args):
>> +    # No standard library function really spawns a fully detached
>> +    # process under win32 because they allocate pipes or other objects
>> +    # to handle standard streams communications. Passing these objects
>> +    # to the child process requires handle inheritance to be enabled
>> +    # which makes really detached processes impossible.
>> +
>> +    class STARTUPINFO:
>> +        dwFlags = 0
>> +        hStdInput = None
>> +        hStdOutput = None
>> +        hStdError = None
>> +        wShowWindow = 0
>> +
>> +    path = args[0]
>> +    args = subprocess.list2cmdline(args)
>> +    hp, ht, pid, tid = subprocess.CreateProcess(
>> +        path, args,
>> +        # no special security
>> +        None, None,
>> +        # Do not inherit handles
>> +        0,
>> +        # DETACHED_PROCESS
>> +        0x00000008,
>> +        os.environ,
>> +        os.getcwd(),
>> +        STARTUPINFO())
>> +    return pid
>> +
>>  try:
>>     # override functions with win32 versions if possible
>>     from win32 import *
> 
> hgtk has been using:
> 
> os.spawnve(os.P_NOWAIT, sys.executable, args, env)
> 
> to spawn new processes since about version 0.8.  Would that work for
> the http daemon?

Good question, and probably tried that too once. I tested a couple of combinations again and here are my results:

- os.spawnv(os.P_NOWAIT): works and pass the tests but since the child process is not detached, it is killed when the console where it was started terminates. This behaviour differs from unix one and prevents a future use of this for inotify-like daemon. But it might be enough for now.
- os.spawnv(os.P_DETACH): works and pass the tests with python25. With python26, something really strange happens: the process is detached and starts the cgi server, handles requests and so forth, but blocks when writing to the socket. I have no idea what's going on here.
- subprocess.CreateProcess() hack: similar to os.spawnv(os.P_DETACH)

What's interesting is the original patch version here:

http://hg.intevation.org/mercurial/pmezard/crew-w32.mq/file/7a2eef70d4f6/hg-serve-daemon-win32#l110

used the subprocess.CreateProcess() trick but ran a hg.bat batch file instead of the python interpreter. And it worked with python25 and python26 but opened an empty command shell. So, it seems python intepreter or one of its libraries does not like to be started in purely detached mode. I will try another hack to open it in detached mode, in shell mode, without a shell window.

--
Patrick Mézard