[PATCH 3 of 3 rfc] mpath: add a cffi version of fold, patches and patchedsize

Augie Fackler raf at durin42.com
Tue Mar 8 10:25:08 EST 2016


On Mon, Mar 07, 2016 at 05:43:17PM +0200, Maciej Fijalkowski wrote:
> # HG changeset patch
> # User fijal
> # Date 1456477676 -3600
> #      Fri Feb 26 10:07:56 2016 +0100
> # Node ID 50f9e5969e282b318b4f5b7f916c6eb21bdedb95
> # Parent  e04ac5d0fefe1418e1906ec209c0a3657ad8b022
> mpath: add a cffi version of fold, patches and patchedsize
>
> This patch adds cffi-based mpatch and makes hg blame about 30%
> faster on PyPy. The cffi process is as follows:
>
> * build cffi extension for mpatch (mercurial/build_mpatch_cffi is a bad
>   place, not sure where to put it)
> * use cffi extension from mercurial/pure/mpatch via mpatch_cffi,
>   we can merge those two files (the second depending on cffi
>   the first one only conditionally importing stuff from the second)
>
> This patch is not intended for merging, only for RFC
>
> diff -r e04ac5d0fefe -r 50f9e5969e28 mercurial/build_mpatch_cffi.py
> --- /dev/null	Thu Jan 01 00:00:00 1970 +0000
> +++ b/mercurial/build_mpatch_cffi.py	Fri Feb 26 10:07:56 2016 +0100
> @@ -0,0 +1,20 @@
> +import os
> +from cffi import FFI
> +
> +ffi = FFI()
> +mpatch_c = os.path.join(os.path.dirname(__file__), 'mpatch.c')
> +ffi.set_source("_mpatch_cffi", open(mpatch_c).read(), include_dirs=["mercurial"])
> +ffi.cdef("""
> +struct flist {
> +    ...;
> +};
> +
> +struct flist *decode(const char *bin, size_t len);
> +struct flist *combine(struct flist *a, struct flist *b);
> +size_t calcsize(size_t len, struct flist *l);
> +void lfree(struct flist *a);
> +static int apply(char *buf, const char *orig, size_t len, struct flist *l);
> +size_t _patchedsize(long orig, char* bin, size_t patchlen);

Why can't we just import mpatch.h here, rather than duplicating a
bunch of the header?


> +""")
> +if __name__ == '__main__':
> +    ffi.compile()
> diff -r e04ac5d0fefe -r 50f9e5969e28 mercurial/mpatch_cffi.py
> --- /dev/null	Thu Jan 01 00:00:00 1970 +0000
> +++ b/mercurial/mpatch_cffi.py	Fri Feb 26 10:07:56 2016 +0100
> @@ -0,0 +1,33 @@
> +
> +from _mpatch_cffi import lib, ffi
> +
> +def fold(bins, start, end, all):

It bums me out to have to duplicate a third version of some of this
code (Pure, C extenstion, and now a bit of cffi glue). Is there really
no better option? I ask because if I understand right, most of
lazymanifest.c will get duplicated in cffi glue, which is tragic.

> +    if start + 1 == end:
> +        to_pass = ffi.new("char[]", str(bins[start]))
> +        all.append(to_pass)
> +        return lib.decode(to_pass, len(to_pass) - 1)
> +    lgt = (end - start) / 2
> +    return lib.combine(fold(bins, start, start + lgt, all),
> +                  fold(bins, start + lgt, end, all))
> +
> +def patches(text, bins):
> +    all = []
> +    lgt = len(bins)
> +    if not lgt:
> +        return text
> +    patch = fold(bins, 0, lgt, all)
> +    if not patch:
> +        raise Exception("failed")
> +    outlen = lib.calcsize(len(text), patch)
> +    if outlen < 0:
> +        lib.lfree(patch)
> +        raise Exception("failed")
> +    buf = ffi.new("char[]", outlen)
> +    if not lib.apply(buf, text, len(text), patch):
> +        lib.lfree(patch)
> +        raise Exception("fail")
> +    res = ffi.buffer(buf, outlen)[:]
> +    return res
> +
> +def patchedsize(orig, bin):
> +    return lib._patchedsize(orig, bin, len(bin))


More information about the Mercurial-devel mailing list