Testing very long delta chains

[Gregory Szorc]

> On Dec 23, 2015, at 07:22, Yuya Nishihara <yuya at tcha.org> wrote:
> 
>> On Tue, 22 Dec 2015 23:30:20 -0800, Gregory Szorc wrote:
>>> On Tue, Dec 22, 2015 at 9:41 PM, Matt Mackall <mpm at selenic.com> wrote:
>>> Things we can do better:
>>> 
>>> - add a C decompress helper
>>> - that works on lists of buffers
>>> - that calls zlib directly
>>> - that uses threads
>>> - that uses larger buffers
>>> - that uses a faster zlib
>>> 
>>> (For this last, the cloudflare fork of zlib has a faster CRC function that
>>> seems to be worth about 20%)
>> 
>> From C, this will not be fun because Windows.
>> 
>> Half serious question: what are your thoughts on writing this in Rust? We
>> can write a Rust library that provides a C ABI which Python can interface
>> with (either via a Python C extension or via ctypes/cffi). Rust should be
>> supported on all the platforms we care about. For building/distribution, we
>> can provide wheels for Windows and OS X so `pip install` "just works" [and
>> doesn't require a working compiler]. That leaves Linux and other Unixen.
>> Most distributions these days offer a Rust package or are in the process of
>> offering one. For the laggards, I assume we'll still have the pure Python
>> fallback.
> 
> I like Rust, but I don't think their libraries and packaging stuff are mature
> enough. For example, we'll need a non-std crate for zlib and its dependency
> would be managed by cargo with an explicit version. And, it would be headache
> to translate this sort of dependencies to system packages (e.g. .deb or .rpm).

At the point we're using a faster zlib implementation, we'll likely have a zlib implementation vendored and statically linked. So, I think this concern translates to how how hard it is to support linking against the system zlib, if we want to support that at all.

FWIW, the way I see PyPy and Python 3 support playing out is we rewrite the existing C extensions code and produce a shared library that we call into from Python using ctypes/cffi. At that point, the underlying code could be C, C++, Rust, or anything else (or a mix thereof) capable of producing something that conforms to the C ABI.

> 
> Also, 32bit MSVC ABI isn't stable yet?

They are conservative with the "stable" label. Firefox will be shipping a Rust component in 2016. If it's good enough for Firefox, it should be good enough for Mercurial.