[PATCH 01 of 11] bdiff: use Python memory allocator in fixws

Jun Wu quark at fb.com
Thu Mar 16 02:15:13 EDT 2017


I think it makes sense to migrate Python modules to PyMem_*.

However, I'm also a bit conservative about the "calloc" perf issue mentioned
by Yuya. I think it's safe to take Patch 1, 2, 4, 5 where there is no calloc
change for now. We can figure out what to do with calloc / pure C code
later.

Excerpts from Gregory Szorc's message of 2017-03-09 13:59:09 -0800:
> # HG changeset patch
> # User Gregory Szorc <gregory.szorc at gmail.com>
> # Date 1489089265 28800
> #      Thu Mar 09 11:54:25 2017 -0800
> # Node ID 7a8ce919a13a9ff6e73f391fe14d228e18387f15
> # Parent  1871a1ee64ed49172b1568b89cdbab126284b309
> bdiff: use Python memory allocator in fixws
> 
> Python has its own memory allocation APIs. For allocations
> <= 512 bytes, it allocates memory from arenas. This means that
> average small allocations don't call the system allocator, which
> makes them faster. Also, arena allocations cut down on memory
> fragmentation, which can matter for performance in long-running
> processes.
> 
> Another advantage of using the Python memory allocator is that
> allocations are tracked by Python. This is a bigger deal in
> Python 3, as modern versions of Python have some decent built-in
> tools for examining memory usage, leaks, etc.
> 
> This patch converts a trivial malloc() + free() in the bdiff code
> to use the Python allocator APIs. Since the object being
> operated on is a line, chances are it will use an arena. So,
> this could have a net positive impact on performance (although
> I didn't measure it).
> 
> diff --git a/mercurial/bdiff_module.c b/mercurial/bdiff_module.c
> --- a/mercurial/bdiff_module.c
> +++ b/mercurial/bdiff_module.c
> @@ -158,7 +158,7 @@ static PyObject *fixws(PyObject *self, P
>      r = PyBytes_AsString(s);
>      rlen = PyBytes_Size(s);
>  
> -    w = (char *)malloc(rlen ? rlen : 1);
> +    w = (char *)PyMem_Malloc(rlen ? rlen : 1);
>      if (!w)
>          goto nomem;
>  
> @@ -178,7 +178,7 @@ static PyObject *fixws(PyObject *self, P
>      result = PyBytes_FromStringAndSize(w, wlen);
>  
>  nomem:
> -    free(w);
> +    PyMem_Free(w);
>      return result ? result : PyErr_NoMemory();
>  }
>  


More information about the Mercurial-devel mailing list