[PATCH RFC] RFC: allow optional C++ 11 extensions with pybind11 for performance code

[Durham Goode]

On 2/8/16 1:53 PM, Sean Farley wrote:
> Laurent Charignon <lcharignon at fb.com> writes:
>
>> On 2/8/16, 1:33 PM, "Sean Farley" <sean at farley.io> wrote:
>>
>>> Laurent Charignon <lcharignon at fb.com> writes:
>>>
>>>> # HG changeset patch
>>>> # User Laurent Charignon <lcharignon at fb.com>
>>>> # Date 1454965979 28800
>>>> #      Mon Feb 08 13:12:59 2016 -0800
>>>> # Branch stable
>>>> # Node ID 3c1ae9c7e93e8556fa7470919108a02bd989d040
>>>> # Parent  61f4d59e9a0be4e25c1aa016db1a80a540a9d337
>>>> RFC: allow optional C++ 11 extensions with pybind11 for performance code
>>> I am very much against this as I am also against writing code in rust /
>>> go / etc. Mercurial is a *system tool*. When cloning code on IBM
>>> BlueGene it is annoying as hell to have random dependencies sneak in
>>> like this. For reference, IBM BlueGene has had problem with codes that
>>> use C++.
>> I understand your concern but that is not the case, it is not a random
>> dependency.
>> Like our C extensions, the C++ extensions, would be *optional*.
>> If you are not able to compile them on some system, mercurial should still
>> work without them.
> It would still be crippled (performance-wise). In academia (and HPC)
> this was very embarrassing. Git could be compiled fast on the same
> machine and was not crippled in the same way.
>
>>> C is much, much, much more portable. In fact, I would like to go even
>>> further and break out the C code we have into a shared library (for cffi
>>> sweetness).
>> It is true that C is more portable.
>> However, it is much more difficult (at least for me) to implement large
>> features like lazy manifest or tree manifest in C compared to rust / go /
>> C++.
>> How do you think we should proceed?
>> Do you suggest finding a C implementation of whatever
>> data-structure/feature that would work for us and include them in the
>> project?
> Why not? I don't C is *that* hard. My usual advice is to prototype in
> your favorite language of Python / rust / go / D / whatever and then
> convert it to C for portability and performance.
I think you underestimate the barrier of entry that C puts in the way.  
There's plenty of places that would benefit from native fast paths 
(revlog parsing, revset handling, etc), and a large reason we haven't 
attempted to do it is because the developer-time vs benefit ratio for C 
is so poor.

As for supporting super computers and solaris, if the current Mercurial 
performance has been adequate for those edge cases for 10 years, I think 
they will be fine with the existing perf for a while longer.  I don't 
think we should hinder development for the 99% platforms (linux, osx, 
windows) just so we can get those same perf improvements on edge cases.  
Hell, most repos are small enough that they won't even need the perf 
improvements we're proposing using c++ for.

That said, it's totally possible that C++ isn't an appropriate cross 
platform solution for even linux+osx+windows.  I'm relying on the 
experience of members of the community to determine the sanity of this idea.