[PATCH] Speed up manifest parsing by 4x

[Matt Mackall]

On Wed, 2008-03-26 at 15:58 +0100, Adrian Buehlmann wrote:
> On 26.03.2008 14:59, Matt Mackall wrote:
> > On Wed, 2008-03-26 at 11:18 +0100, Adrian Buehlmann wrote:
> >> Some questions/comments below by an unwary poor C++ Windows programmer
> >> who would like to learn from the *NIX gurus.
> >>
> >> On 26.03.2008 00:45, Bryan O'Sullivan wrote:
> >>> manifest: improve parsing performance by 4x via a C extension
> >> great!
> >>
> >>> +static PyObject *parse_manifest(PyObject *self, PyObject *args)
> >>> +{
> >>> +	PyObject *binascii, *unhexlify;
> >>> +	PyObject *mfdict, *fdict;
> >>> +	char *str, *cur, *start, *zero;
> >> I admit I've never written a single line of C code for UNIX, but
> >> I've learned the hard way that random pointer values found on
> >> stack, shooting into random memory places, can be damn hard to track down.
> >> So, I initialize stack pointers to 0 when defining them.
> >> Just in case they are used in error before initialized. A zero
> >> pointer deref is of magnitudes easier to debug.
> >> The speedup gained when not initializing is rarely worth the potential
> >> trouble.
> > 
> > Depends on the function. Ideally functions are made simple enough that
> > such problems are obvious and/or structually impossible.
> > 
> >>  The alternative is to initialize when defining, which has no speed
> >> penalty.
> > 
> > Speaking as someone who spends a lot of time doing optimization work in
> > the Linux kernel: definitely not true. Initializers are code that gets
> > run at every function invocation. It takes both cycles and, perhaps more
> > importantly, cache footprint. Taking an extra cacheline hit is very
> > expensive.
> 
> I was comparing pattern (1):
> 
> 	PyObject *unhexlify = NULL;
> 	/* doing something here */
> 	unhexlify = PyObject_GetAttrString(binascii, "unhexlify");
> 
> to pattern (2):
> 
> 	PyObject *unhexlify = PyObject_GetAttrString(binascii, "unhexlify");
> 
> Pattern (1) is slower than (2).
> 
> Please correct me if I'm wrong, but I think Bryan's original pattern (3):
> 
> 	PyObject *unhexlify;
> 	/* doing something here */
> 	unhexlify = PyObject_GetAttrString(binascii, "unhexlify");
> 
> isn't faster than (2).
> 
> I agree that pattern (1) is slower than (3), which I try to avoid by using (2).

Fair enough. In this particular instance, we should probably just bypass
the Python library and roll our own. That lets us avoid an extra
allocation and inline this piece.

> Thus my yet unanswered question whether declarations of variables in the
> middle of a compound statement are allowed.

No. While some compilers allow it as part of C99, it's still not widely
portable and will leave a lot of C programmers scratching their heads. 

-- 
Mathematics is the supreme nostalgia of our time.