[PATCH] test-commandserver: use python instead of hg as the executable

[Adrian Buehlmann]

On 2012-06-22 00:21, Matt Mackall wrote:
> On Thu, 2012-06-21 at 23:12 +0200, Adrian Buehlmann wrote:
>> On 2012-06-21 22:10, Matt Mackall wrote:
>>> On Thu, 2012-06-21 at 02:06 +0200, Adrian Buehlmann wrote:
>>>> On 2012-06-21 00:42, Adrian Buehlmann wrote:
>>>>> On 2012-06-21 00:22, Matt Mackall wrote:
>>>>>> On Wed, 2012-06-20 at 23:10 +0200, Adrian Buehlmann wrote:
>>>>>>> On 2012-06-20 22:26, Matt Mackall wrote:
>>>>>>>> On Sun, 2012-06-17 at 11:23 +0200, Adrian Buehlmann wrote:
>>>>>>>>> # HG changeset patch
>>>>>>>>> # User Adrian Buehlmann <adrian at cadifra.com>
>>>>>>>>> # Date 1339924726 -7200
>>>>>>>>> # Node ID 20c4d99cc7e96666de519418df51b6d5b7d9fad2
>>>>>>>>> # Parent  a0f221f45f846aea982355840c614d6fced41edb
>>>>>>>>> test-commandserver: use python instead of hg as the executable
>>>>>>>>
>>>>>>>> I'm not thrilled about this approach. We certainly expect "hg" to be
>>>>>>>> executable, and it's simply a deferred bug of our Windows build process
>>>>>>>> that it doesn't result in something called "hg" you can run.
>>>>>>>>
>>>>>>>> (And it's only deferred because the traditional methods of making a
>>>>>>>> Python script executable on Windows all have rather large caveats.)
>>>>>>>>
>>>>>>>> But I don't think it's a quirk we want to enshrine in the test suite.
>>>>>>>> It'd be better if we put a single gross hack in run-tests so that it
>>>>>>>> just works. For instance, if no suitable hg is present, building an
>>>>>>>> hg.cmd and sticking it in $TMP somewhere, then adding it to the path.
>>>>>>>>
>>>>>>>> ps: Wanted: benchmark comparison of startup times for .cmd, exemaker,
>>>>>>>> and py2exe variants.
>>>>>>>>
>>>>>>>
>>>>>>> If we are forced to keep (from test-commandserver.py):
>>>>>>>
>>>>>>> 4:    cmdline = ['hg', 'serve', '--cmdserver', 'pipe']
>>>>>>> 5:    if path:
>>>>>>> 6:        cmdline += ['-R', path]
>>>>>>> 7:
>>>>>>> 8:    server = subprocess.Popen(cmdline, stdin=subprocess.PIPE,
>>>>>>> 9:                              stdout=subprocess.PIPE)
>>>>>>>
>>>>>>> as is, then I don't think it will be possible to use a hg.cmd.
>>>>>>>
>>>>>>> According to
>>>>>>>
>>>>>>>   http://docs.python.org/library/subprocess.html#popen-constructor
>>>>>>>
>>>>>>> subprocess.Popen uses CreateProcess() on Windows, which wants the name
>>>>>>> of an exe ('hg' as specified on line 4 of test-commandserver.py).
>>>>>>
>>>>>>> Then I see no way but providing a hg.exe (i.e. using exemaker or similar).
>>>>>>
>>>>>> A quick test with shell=True seems to find my test script here:
>>>>>>
>>>>>>>>> server = subprocess.Popen(['h'], shell=True, stdin=subprocess.PIPE,
>>>>>> stdout=s
>>>>>>>>> server.stdout.read()
>>>>>> 'hello\r\n'
>>>>>>
>>>>>> I think we use shell=True everywhere else in the hg code (or should) so
>>>>>> things like hooks will work.
>>>>>>
>>>>>
>>>>> Yes. If
>>>>>
>>>>> diff --git a/tests/test-commandserver.py b/tests/test-commandserver.py
>>>>> --- a/tests/test-commandserver.py
>>>>> +++ b/tests/test-commandserver.py
>>>>> @@ -5,7 +5,7 @@
>>>>>      if path:
>>>>>          cmdline += ['-R', path]
>>>>>
>>>>> -    server = subprocess.Popen(cmdline, stdin=subprocess.PIPE,
>>>>> +    server = subprocess.Popen(cmdline, shell=True, stdin=subprocess.PIPE,
>>>>>                                stdout=subprocess.PIPE)
>>>>>
>>>>>      return server
>>>>>
>>>>> would be an acceptable change, then test-commandserver.py passes here
>>>>> with a file hg.cmd in C:\Users\adi\hgrepos\hg-main containing:
>>>>>
>>>>>   @echo off
>>>>>   python C:\Users\adi\hgrepos\hg-main\hg %*
>>>>>
>>>>> (I haven't yet figured out how to reliably specify the path to the hg
>>>>> python script there - in a less hackish way).
>>>>
>>>> This one works (partly stolen from contrib/win32/hg.bat):
>>>>
>>>>   @echo off
>>>>   python "%~dp0hg" %*
>>>
>>> Yeah, something like that will usually work. Unfortunately, the "%~dp0"
>>> magic is not terribly reliable:
>>>
>>> http://stackoverflow.com/a/7689467/975761
>>>
>>> It really depends on the precise details of how the calling process sets
>>> up the subprocess environment.
>>>
>>> Also note that since "@" means "don't echo this command", this can be a
>>> one-liner.
>>>
>>
>> Thanks for that warning and the tip.
>>
>> So we either find a correct hg.cmd, or I have to resort to using hg.exe
>> again. Hmmm.
> 
> I think for the purposes here, it's fine. And I guess lots of people
> manage with such a script.
> 
> But I think there are a bunch of reasons to prefer an .exe. This "where
> am I located" issue is one.
> 
> Another, and the one that causes the most grief, is that command scripts
> must be "call"ed inside other command scripts, so you need to constantly
> be aware that "hg" is a command script when doing automation. So you
> can't do this:
> 
>   @echo off
>   hg clone x y
>   cd y
>   hg up default
>   ...
> 
> If you do this, your script will mysteriously stop at 'hg clone'. You
> need to stick "call" in front of each mention of hg (legacy behavior
> from DOS 1.0). Effectively, command scripts don't consider other command
> scripts first-class programs and it ends up biting people who forget
> that hg is 'just a script'.

That's indeed pretty bad.

> There are also issues like "the handle of the thing I launched is
> actually a command shell and not the real process" and "signal delivery
> is weird" and "why is a window popping up" and "why can't I quote or
> escape this $%@ filename" issues.

Yep. That's all true.

The "hg serve" tests still do produce pop-up windows with the hg.exe
though. So I think that might be an orthogonal problem.

> Having an exe wrapper of some sort makes hg a first-class program -from
> cmd.exe's own perspective-. I'm not saying we have to do that, just that
> it does have benefits.
> 
> Also, I think the 32-bit vs 64-bit issue is not really that big a deal.
> If we ever decide we want to make an exemaker-like component part of the
> "official" build process, we'll ship source and build it with the same
> toolchain we use to build our C extensions.

I think you are right and I rest my case about that hg.cmd (or other
modifications to the tests).

But exemaker is not exactly an extra piece of good software (although
the idea is not that bad). And I was wary about problems with those
exe-manifests, but that is probably all nothing compared to the problems
we have with trying to muddle through with hg.cmd.

So I bite the exe bullet.

Thanks for your perseverance.