Performance regression on case-insensitive filesystems (Windows)

[FUJIWARA Katsunori]

At Thu, 19 Jul 2012 18:33:08 +0200,
Martin Geisler wrote:

> > BTW, should "itertools.izip" or same kind utility from scratch be used
> > instead of standard "zip()" for efficiency in resource consumption ?
> > And does avoiding large array creation also contribute performance
> > improvement ?
> 
> Good question -- using an generator adds a non-negligible overhead to
> the iteration, but that might be outweighed by the savings in up-front
> allocation. I also don't know the point where iterators pay off and I
> think we'll need to experiment to see.

OK, I'll learn about this more.

> > After this change, performance checking results are:
> >
> >   (A) hg 2.0:
> >
> >       time: real 3.428 secs (user 2.527+0.000 sys 0.905+0.000)
> >
> >   (B) 67b8cca2f12b(default) + "normcase()" for files stored into
> >       foldmap at a time:
> >
> >       time: real 4.174 secs (user 3.151+0.000 sys 0.998+0.000)
> >
> >   (C) (B) + (1) + (2) + (3):
> >
> >       time: real 4.008 secs (user 3.120+0.000 sys 0.889+0.000)
> >
> >   (D) (C) + improvement in "windows.statfiles()":
> >
> >       time: real 3.740 secs (user 2.839+0.000 sys 0.905+0.000)
> 
> My system is behaving weird yesterday and today :-/ I can no longer
> reproduce my original timings: 'hg status' takes 4.5 seconds with
> Mercurial 2.0, with 67b8cca2f12b, and with your patches.
> 
> It's quite mysterious to me and I don't really trust the timings any
> longer. I hope someone else can give the patches a try and reproduce the
> improvements you see and the regression that I saw.

I also examined again performance checking on my PC, and all
performance measurement results for each HG versions were degraded !

    (A) 3.428 => 4.055

    (B) 4.174 => 4.657

    (C) 4.008 => 4.631

    (D) 3.740 => 4.377

I confirmed that:

  - on/off of Virus Scan has no effect for these measurement results

  - according to "Windows Update" log, the latest updating is
    2012-07-11, and it is before my last measurement

  - almost all of 16GB memory on Windows7(64bit) are also free today,
    and enough disk cache should be available

  - the target repository and Mercurial installations are not moved on
    HDD, and seeking speed should not be degraded from my last
    measurement

Hummm, it's mysterious.

But it's some comfort that performance degrading and improvement can
be still measured on my PC, so I'll try more improvement.


BTW, for convenience to try performance measurement or improve further
by someone else, I attached the patch to improve "dirstate._foldmap()"
corresponded to (B) in my cases, which is not attached in your post.

Please comment on it, if this doesn't follow the image of your
"dirstate._foldmap would call util.normcase once on the filenames
joined with \0".

----------------------------------------------------------------------
[FUJIWARA Katsunori]                             foozy at lares.dti.ne.jp
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 17202.diff
Type: application/octet-stream
Size: 1056 bytes
Desc: not available
URL: <http://selenic.com/pipermail/mercurial-devel/attachments/20120720/00b235bc/attachment.obj>