[PATCH] Speed up manifest parsing by 4x

Adrian Buehlmann adrian at cadifra.com
Wed Mar 26 09:58:04 CDT 2008


On 26.03.2008 14:59, Matt Mackall wrote:
> On Wed, 2008-03-26 at 11:18 +0100, Adrian Buehlmann wrote:
>> Some questions/comments below by an unwary poor C++ Windows programmer
>> who would like to learn from the *NIX gurus.
>>
>> On 26.03.2008 00:45, Bryan O'Sullivan wrote:
>>> manifest: improve parsing performance by 4x via a C extension
>> great!
>>
>>> +static PyObject *parse_manifest(PyObject *self, PyObject *args)
>>> +{
>>> +	PyObject *binascii, *unhexlify;
>>> +	PyObject *mfdict, *fdict;
>>> +	char *str, *cur, *start, *zero;
>> I admit I've never written a single line of C code for UNIX, but
>> I've learned the hard way that random pointer values found on
>> stack, shooting into random memory places, can be damn hard to track down.
>> So, I initialize stack pointers to 0 when defining them.
>> Just in case they are used in error before initialized. A zero
>> pointer deref is of magnitudes easier to debug.
>> The speedup gained when not initializing is rarely worth the potential
>> trouble.
> 
> Depends on the function. Ideally functions are made simple enough that
> such problems are obvious and/or structually impossible.
> 
>>  The alternative is to initialize when defining, which has no speed
>> penalty.
> 
> Speaking as someone who spends a lot of time doing optimization work in
> the Linux kernel: definitely not true. Initializers are code that gets
> run at every function invocation. It takes both cycles and, perhaps more
> importantly, cache footprint. Taking an extra cacheline hit is very
> expensive.

I was comparing pattern (1):

	PyObject *unhexlify = NULL;
	/* doing something here */
	unhexlify = PyObject_GetAttrString(binascii, "unhexlify");

to pattern (2):

	PyObject *unhexlify = PyObject_GetAttrString(binascii, "unhexlify");

Pattern (1) is slower than (2).

Please correct me if I'm wrong, but I think Bryan's original pattern (3):

	PyObject *unhexlify;
	/* doing something here */
	unhexlify = PyObject_GetAttrString(binascii, "unhexlify");

isn't faster than (2).

I agree that pattern (1) is slower than (3), which I try to avoid by using (2).

Thus my yet unanswered question whether declarations of variables in the
middle of a compound statement are allowed.


More information about the Mercurial-devel mailing list