[PATCH 2 of 4] obsstore: disable garbage collection during initialisation (issue4456)

Pierre-Yves David pierre-yves.david at ens-lyon.org
Thu Dec 4 05:45:24 CST 2014



On 12/03/2014 06:00 PM, Matt Mackall wrote:
> On Thu, 2014-12-04 at 01:44 +0100, Antoine Pitrou wrote:
>> On Wed, 03 Dec 2014 17:31:25 -0600
>> Matt Mackall <mpm at selenic.com> wrote:
>>> On Sun, 2014-11-30 at 05:17 -0800, Pierre-Yves David wrote:
>>>>
>>>> On 11/30/2014 05:06 AM, Antoine Pitrou wrote:
>>>>> On Sat, 29 Nov 2014 17:57:42 -0800
>>>>> Pierre-Yves David <pierre-yves.david at ens-lyon.org> wrote:
>>>>>> # HG changeset patch
>>>>>> # User Pierre-Yves David <pierre-yves.david at fb.com>
>>>>>> # Date 1417049911 28800
>>>>>> #      Wed Nov 26 16:58:31 2014 -0800
>>>>>> # Node ID d0f3dac4ea2b4aff51946c7db0834aa4e5c3e82a
>>>>>> # Parent  04eb7e49d2b6f90f71aa85de9ad0b4d70670d688
>>>>>> obsstore: disable garbage collection during initialisation (issue4456)
>>>>>>
>>>>>> Python garbage collection is triggered by contained creation. So code that
>>>>>> creates a lot of tuple tends to trigger GC a lot. We disable the gc during
>>>>>> obsolescence marker parsing and associated initialization. The provide and
>>>>>> interesting speedup (25%).
>>>>>>
>>>>>> On my 58758 markers repo:
>>>>>> before: 0.468247 seconds
>>>>>> after:  0.344362 seconds
>>>>>
>>>>> Which Python version is that?
>>>>
>>>> Python 2.7.8 (default, Oct 18 2014, 12:50:18)
>>>> [GCC 4.9.1]
>>>
>>> Python's GC behavior while building large containers is quite
>>> questionable (aka quadratic).
>>
>> It shouldn't, see https://hg.python.org/cpython/rev/79276316b94b/
>>
>> If you have a reproducer, please open an issue at bugs.python.org.
>
> You're right, the quadratic behavior appears to be gone in 2.7. In 2.7,
> it's down to O(n): generic construction of lists of tuples gets about 4x
> slower. So it's still definitely worth disabling for any potentially
> large list or dict we might build. And obsolete markers are in that
> category.
>
> And the quadratic behavior is still present in 2.6 (aka "the current
> Python" for people running enterprise distros) which we'll probably be
> supporting for a few years still.

Interesting, lets go for disabling GC during obsmarker loading then.

-- 
Pierre-Yves David


More information about the Mercurial-devel mailing list