Status of speed regressions...

[Greg Ward]

[please don't top-post -- it makes it much harder to follow the thread
of a conversation]

[my suggestion]
> The fix for this is simple and obvious: write an extension that
> exposes a "Mercurial shell", where you can just run "status" or
> "update" or "log" without the overhead of a new Python process.  Then
> have MacHg or MercurialEclipse control this process through a pipe or
> a socket.
>
> In fact, ISTR that someone even posted the code here a few months ago.
> It was tiny: like 20 lines or something.  If you can package that
> extension with your GUI, you're done.

[Jason's reply]
> As an aside I have also pondered doing this in the past. However, MacHg is multithreaded and launches several mercurial sessions in parallel.

Oooh, threads, yeah, tricky.  I've heard they're useful in GUIs, but
Mercurial makes no pretence of being thread-safe, so of course you
have to assume it is not.  Hmmm.

Depending on how MacHg creates threads, you could just have a separate
child "hg shell" process for each MacHg thread.  That would be simpler
than managing a pool, but not very useful if you create threads
willy-nilly in your GUI.

Another possibility is to serialize access to Mercurial: have a single
thread in MacHg that is responsible for talking to Mercurial, whether
it be by spawning a separate child for each interaction or by talking
to a mythical "hg shell" child process.  Any operation that locks the
repo (commit, pull) is going to be implicitly serialized by the lock
anyways, but this might hurt on readonly operations.

Now the *real* problem...

> I guess I could try and manage a pool of these shells, but then I have to be careful if one of these links becomes corrupted, or one of the particular python-mercurial commands leaves something in a bad state... From what I understand the normal test suite does not test for this. Eg lets say some extension or something sets some internal state in the python-mercurial session during the execution of say "hg strip somerev". Normally almost all the testing and most people who use Mercurial would have their session exit after that command. But instead this shell would continue executing the next command I give it say "hg log someargs". Who is to say that this command will not trip across some state set up by the strip command (maybe some path got set or something... I have no idea, but I gather this thing isn't regularly tested for? Maybe I am wrong here...)

Absolutely correct on all counts.  I discovered just such a bug myself
from a custom script that did a couple of hg operations in a single
process.  I don't remember the details, but I reported the bug
(issue2137), implemented a fix, sent the patch, and got it fixed.  It
distracted me from my main purpose (writing and debugging that custom
script) for several hours over a day or three, but Mercurial got a
little bit better in the process.  Everybody wins.

As for systematically testing such bugs: also correct.  It's doable,
but not trivial.  See tests/test-issue2137.t for an example.

Greg