[PATCH] Large clone chewing CPU

Eric M. Hopper hopper at omnifarious.org
Fri Sep 2 01:21:14 CDT 2005


The problem is much worse under Python 2.3.  Or maybe it's the Opteron
vs. the Pentium 4.  Or maybe the older kernel version.  Whichever.

I altered Bryan's class to look like this:

class stringbuffer:
    '''reasonably efficient string concatenation class.'''

    def __init__(self, *args):
        self.cat = cStringIO.StringIO()
        write = self.cat.write
        self.lens = reduce(lambda l, s: (write(s) and None) or (l + len(s)),
                           args, 0)

    def append(self, s):
        self.cat.write(s)
        self.lens += len(s)

    def __len__(self):
        return self.lens

    def splitat(self, x):
        s = self.cat.getvalue()
        self.cat.close()
        self.cat = None
        return s[:x], s[x:]

    def __str__(self):
        return self.cat.getvalue()

Somewhat unexpectedly this made things worse.  It made them about twice
as slow.  That tells me that the splits are also pretty significant as
I'm pretty certain that for simple string concatenation this code would
be faster than the original code.  The only way I can think of to speed
up the split part significantly in Python is to use the buffer type.

My original patch is still very fast under Python 2.3.

Hmmm...,
-- 
The best we can hope for concerning the people at large is that they
be properly armed.  -- Alexander Hamilton
-- Eric Hopper (hopper at omnifarious.org  http://www.omnifarious.org/~hopper) --
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 185 bytes
Desc: This is a digitally signed message part
Url : http://www.selenic.com/pipermail/mercurial/attachments/20050901/4038e66b/attachment.pgp


More information about the Mercurial mailing list