Testing very long delta chains

Sean Farley sean at farley.io
Wed Dec 23 12:15:06 CST 2015


Gregory Szorc <gregory.szorc at gmail.com> writes:

>> On Dec 23, 2015, at 07:22, Yuya Nishihara <yuya at tcha.org> wrote:
>> 
>>> On Tue, 22 Dec 2015 23:30:20 -0800, Gregory Szorc wrote:
>>>> On Tue, Dec 22, 2015 at 9:41 PM, Matt Mackall <mpm at selenic.com> wrote:
>>>> Things we can do better:
>>>> 
>>>> - add a C decompress helper
>>>> - that works on lists of buffers
>>>> - that calls zlib directly
>>>> - that uses threads
>>>> - that uses larger buffers
>>>> - that uses a faster zlib
>>>> 
>>>> (For this last, the cloudflare fork of zlib has a faster CRC function that
>>>> seems to be worth about 20%)
>>> 
>>> From C, this will not be fun because Windows.
>>> 
>>> Half serious question: what are your thoughts on writing this in Rust? We
>>> can write a Rust library that provides a C ABI which Python can interface
>>> with (either via a Python C extension or via ctypes/cffi). Rust should be
>>> supported on all the platforms we care about. For building/distribution, we
>>> can provide wheels for Windows and OS X so `pip install` "just works" [and
>>> doesn't require a working compiler]. That leaves Linux and other Unixen.
>>> Most distributions these days offer a Rust package or are in the process of
>>> offering one. For the laggards, I assume we'll still have the pure Python
>>> fallback.
>> 
>> I like Rust, but I don't think their libraries and packaging stuff are mature
>> enough. For example, we'll need a non-std crate for zlib and its dependency
>> would be managed by cargo with an explicit version. And, it would be headache
>> to translate this sort of dependencies to system packages (e.g. .deb or .rpm).
>
> At the point we're using a faster zlib implementation, we'll likely have a zlib implementation vendored and statically linked. So, I think this concern translates to how how hard it is to support linking against the system zlib, if we want to support that at all.

Please, no rust. Building and packaging rust (even on mac) is a headache
for me and I'd rather not deal with it.

> FWIW, the way I see PyPy and Python 3 support playing out is we rewrite the existing C extensions code and produce a shared library that we call into from Python using ctypes/cffi. At that point, the underlying code could be C, C++, Rust, or anything else (or a mix thereof) capable of producing something that conforms to the C ABI.

C, please. I'm totally on board with a shared library.


More information about the Mercurial-devel mailing list