[PATCH 4 of 4] dirstate: use a presized dict for the dirstate
Siddharth Agarwal
sid0 at fb.com
Tue Jun 16 02:55:57 CDT 2015
# HG changeset patch
# User Siddharth Agarwal <sid0 at fb.com>
# Date 1434440761 25200
# Tue Jun 16 00:46:01 2015 -0700
# Node ID ca98a16ff9d5b1ef9b54503fde91edd928d3a88e
# Parent 7b8f6849ec9d2e5f20d7f452e448ad2720e505b7
dirstate: use a presized dict for the dirstate
This uses a simple heuristic to avoid expensive resizes.
On a real-world repo with around 400,000 files, perfdirstate:
before: ! wall 0.155562 comb 0.160000 user 0.150000 sys 0.010000 (best of 64)
after: ! wall 0.132638 comb 0.130000 user 0.120000 sys 0.010000 (best of 75)
On another real-world repo with around 250,000 files:
before: ! wall 0.098459 comb 0.100000 user 0.090000 sys 0.010000 (best of 100)
after: ! wall 0.089084 comb 0.090000 user 0.080000 sys 0.010000 (best of 100)
diff --git a/mercurial/dirstate.py b/mercurial/dirstate.py
--- a/mercurial/dirstate.py
+++ b/mercurial/dirstate.py
@@ -338,6 +338,19 @@ class dirstate(object):
if not st:
return
+ if util.safehasattr(parsers, 'dict_new_presized'):
+ # Make an estimate of the number of files in the dirstate based on
+ # its size. From a linear regression on a set of real-world repos,
+ # all over 10,000 files, the size of a dirstate entry is 85
+ # bytes. The cost of resizing is significantly higher than the cost
+ # of filling in a larger presized dict, so subtract 20% from the
+ # size.
+ #
+ # This heuristic is imperfect in many ways, so in a future dirstate
+ # format update it makes sense to just record the number of entries
+ # on write.
+ self._map = parsers.dict_new_presized(len(st) / 71)
+
# Python's garbage collector triggers a GC each time a certain number
# of container objects (the number being defined by
# gc.get_threshold()) are allocated. parse_dirstate creates a tuple
More information about the Mercurial-devel
mailing list